sim_process_replay_event_message() now hands the decoded replay event to
the caller, which keeps it beside the translated core event and frees it
after xce_core_push_events() with driver.c's release_translated_payload()
rules: a pushed config is kept by the core, and a rusages report is kept
by the job it found. An untranslatable event is freed at once, and events
after the last session are freed at the end. The payloads were an
"accepted leak" per event; exit-time leak on the reproduction replay
75,495 -> 25,807 bytes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A batch --sim replay under ASAN exited 1 on 17.2 MB of LeakSanitizer
reports. This removes all but 75 KB:
- sched_sim_driver_fini() (called by main after a clean run) cancels and
joins the picker/allocator threads like sched_fini(), closes the replay,
alternate-events and output files, and finalizes the core. The threads'
ids now live in core.jobpicker/allocator, as in normal mode.
- xce_sched_core_fini() destroys the plugin, job, worker, queue, user and
project registries and the shared-resource arrays it creates.
- The seqjob/parjob sched_make_allocs() stand-ins set up only lock, nextid
and the aliased resmgr. Each ran a full xce_sched_core_init(), which also
replaced the process-wide string and selector tables mid-startup.
- plugin_unregister(), the plugin registry's datakiller, only runs fini_fn;
it key_del()'d from the registry being destroyed and skipped plugins
registered with plugin_register_core().
- The limit plugin's usage-instance table used kill_limit as datakiller on
xce_vres_t records (bad free on destroy); kill_vres now frees the record.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A COMPOUND/ALTER job's primaries share one ctx. The job-level prepare pass
reset only alloc_req.shared, and the allocators append their slots want
into alloc_req.host, so a later primary's host pre-filter (and a global
limit on slots) read the previous primary's last per-host want: a 1-task
primary after a 4-task one was filtered on slots:4 and pended although a
host with its one free slot existed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Gitea ignores the singular milestone= query parameter and returns the
newest issues repo-wide, so list_issues() never filtered by milestone:
phase-state updates only saw issues on the first page. Use milestones=.
sync_spec() adopted an existing milestone but always created its three
phase issues, so a run killed between creating an issue and writing
spec.json created duplicates on the next run. Reuse an issue already in
the milestone with the same title.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Add test_sim_replay_reproduces_recorded_alloc_from: a recorded DISPATCH at
an epoch-scale beginRound with a sub-second part replays at the default
simulation settings with no --sim-verify mismatch, and the live allocation's
from equals beginRound in whole seconds. Share the dispatch fixture setup
of the existing tests through two helpers.
Live: 2-round HA soak recordings (11 DISPATCH decisions) replayed with
--sim-verify and no --sim-speed report 0 mismatches.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
libxce-alloc-preason-equality (requirements, design, tasks; not yet approved):
- alloc_eq compares `from` and excludes only instid; the scheduler stamps
`from` with the deciding session's start, the same clock reading that
is recorded as beginRound, so a replay reproduces it.
- every comparator takes an optional char** why: a malloc'd description
of the first difference, owned and freed by the caller; --sim-verify
mismatch lines print it (resolves bug-1881).
- job_preason_eq is new full-match behavior, not a relocation (bug-1845
keeps the simulator main-reason-only); the three white-box test files
switch to it and keep their sim_verify.c include.
- alloc_eq stays separate from merge_job_alloc()'s same_alloc_slot()
(bug-1863), which does match on instid.
scheduler-sim-discrete-clock (initialized): remove --sim-speed so the
simulator clock moves only at recorded timestamps and a replayed
session_start equals the recorded beginRound exactly.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
make_alloc() and make_host_alloc() assigned strdup() results unchecked,
so under memory exhaustion they returned a record whose resname/reserve/
host was NULL; merge_job_alloc()'s same_alloc_slot() (bug-1863) then
strcmp()'d it (ASAN SEGV at xcealloc.c:337). make_alloc() also passed a
NULL resname straight to strdup().
- xcealloc.c: make_alloc()/make_host_alloc() reject a NULL name (EINVAL)
and check every strdup(), freeing what they built and returning NULL
(ENOMEM); on failure make_host_alloc()'s caller keeps `allocs`.
dup_host_alloc() frees its partial copy on failure instead of leaking.
- scheduler.c sched_make_allocs(): a record that cannot be built fails
the whole array (NULL, ENOMEM) instead of leaving a NULL entry.
- seqjob/parjob my_alloc(): check every constructor; on failure free
what was built (parjob also rolls back the remaining claims) and
return -ENOMEM with ctx->alloc NULL -- the caller moves ctx.alloc onto
the job whatever the rc.
Tests: new xcealloc_oom_test (8, --wrap=strdup) and
sched_make_allocs_oom_test (real scheduler_srcs + --wrap=strdup); a new
-ENOMEM test in the seqjob and parjob allocator suites. All RED before
(SEGV / "Expected NULL" / "Expected -12 Was 1"), GREEN after.
xce-build.sh all asan: 115/115 suites OK, Go tests OK.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
bug-1879: on a fresh HA cluster the config-holding ctrl-1 started its
term-1 election before its peers ran; connect failures count as "no"
votes (bug-1423), so it lost at once and became a follower with no
leader. ctrl-2/ctrl-3 hold no config and are voters only (by design),
so nothing retried until ctrl-1's 10 s standby liveness timeout: every
cold start had no primary for ~10 s.
- ha_note_leaderless() marks a config-holding node that becomes a
follower with no leader known; the liveness poll then retries the
election (PreVote) after HA_LEADERLESS_RETRY_MS = 1 s, doubling up to
heartbeat_timeout_ms. Primary contact or becoming leader clears it.
- ha_heartbeat_is_fresh() answers "no" while leaderless: last_activity
then only holds the watchdog's arm time, and the failover gates took
it for a live primary.
Live cold start: primary 1.7 s after the first node started (was ~10 s).
bug-1878: ctrlstatus_fill() reported ha->applied_lsn for every role, a
standby-only counter a primary never advances, so every primary showed
"appliedLsn: 0". A primary now reports persist->last_applied_lsn. The
controller-status design doc's "already correct in every role" note is
corrected.
Tests: 5 new in test_ha_failover_prevote.c, 1 new in test_ctrlstatus.c
(RED: expected 4900, was 0); controller suite 1256/0. HA soaks:
sequential 6 rounds PASS, chaotic 6 rounds PASS (zero divergence, zero
job loss).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- bug-1872 (worker): "Failed to find any disk statistics in
/proc/diskstats" ERROR every 15 s on an NVMe-only host. The whole-disk
filter guessed from the name and skipped anything ending in a digit or
containing 'p' -- every nvmeXnY, mmcblkN, even sdp -- so io was never
reported. diskstats.h now counts a device only when
/sys/block/<name>/device exists (real disks; not partitions, loop, dm,
md, zram), and a host with no disk warns once.
- bug-1873 (worker): "Failed to get worker pid" WARN on job start. MainPID
reads 0 until systemd has forked the new transient unit; xcecd.c already
retries, so the miss is DEBUG and the final failure names the unit.
- bug-1874 (scheduler): deprecation WARN for the legacy xce_sched_fcfs on
every start, though builtin_fcfs is the picker. Registration still wraps
a job_elect_fn plugin but no longer warns; selecting one warns once.
- bug-1875 (apiserver): "Not running as root -- PAM authentication will not
work" with the default provider config_file; now only for provider pam.
- bug-1876 (controller): standbys logged "cluster_init_phase2 ... failed
(rc=-22)" WARN every 3 s while waiting for the primary's config push,
a state the function itself calls normal. It returns -EAGAIN for that
wait; callers log it at INFO, other failures still WARN.
- bug-1877 (controller): "WAL/snapshot stored unencrypted" warned without
looking. persist_path_encrypted() checks fscrypt (statx) and dm-crypt
(sysfs dm/uuid CRYPT-, through LVM slaves); the warning stays when the
storage is not shown to be encrypted (e.g. a plain /tmp).
Tests: worker diskstats_test (2 of 4 RED on the old filter with pc13v's
/proc/diskstats), controller test_phase2_cold_boot and
test_persist_encryption, scheduler picker tests, apiserver
TestRootWarning. Full ASAN build: all 113 suites pass (controller 319/0,
harness 141/0).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
