- bug-1799/1801: fix resInUse over-commit in sim replay -- sched_account_jobs's
per-round re-derive pass now skips RECONCILING/UNAVAIL workers the same way
core_reset_workers already does (bug-1751), closing the gap that let a
worker's used slots exceed its own capacity across a resync replay
- bug-1804: revise scheduler-sim-eager-reconciliation spec (user-directed) so
a captured job's job->state is set to RUN synchronously at dispatch capture,
mirroring sched_commit_decision's normal-mode semantics (bug-1624), instead
of deferring until its separately-recorded JOB_STATE_CHANGE event replays
- bug-1800: render bytes-valued CONSUMABLE resources (mem/swp/tmp) with the
same human-scaled unit formatting already used for maxmem/maxswap/maxtmp
- bug-1802/1803: add then revert a break-on-ANY-event breakpoint kind after
live feedback that it over-fires inside busy sessions; replace it with a
prescan-based "next non-dummy session" hint in info summary instead
Full scheduler suite: 77/77 tests pass (asan profile).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RLSNJhNoULvVcUhXEFafQC
The hand-written per-flag description explaining -n's <min>[,<max>]
format was dropped when task 3.5 moved the flag list to the generated
flagspec block (which only shows short/long/required/default, with no
room for a flag's value-format semantics). Restore the explanation as
prose in USAGE_JOBSUB_HEAD, following the same pattern already used by
resupdate's HEAD (-n/-v pairing, -t's conditional meaning).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
Follow-up audit (requested: check other generated sources, specifically
packcc/PEG output) found the identical bug shape in the four PEG-parser
custom_targets: section_parser, rusage_parser, xce_resreq_parser, and
xce_select_parser each produce a .c + a paired .h (section.h, rusage.h,
xce_resreq.h, xce_select_peg.h), but libxce_src only referenced each
parser's [0] (.c) index. lib/resreq.c -- confirmed via a project-wide
grep to be the only consumer of any of these four headers -- had no
declared build-order dependency on any of the four codegen steps.
Fix: pass each of the four custom_targets unindexed into libxce_src,
same pattern as the capnp fix. No libxce_dep change needed here since
nothing outside src/lib/ consumes these headers.
A project-wide search confirms these, plus the capnp custom_target, are
the only multi-output C/H codegen steps in the whole build -- every other
custom_target() (xceadmin, apiserver, xcecli, autoscaler/providers)
produces a single Go-binary/stamp output with no equivalent hazard.
Verified: a third consecutive full from-scratch `./tools/xce-build.sh all
debug` build (1173/1173 targets) succeeded clean; a fresh `lib debug`
build plus the full lib test suite (313/313) passed; a fresh incremental
controller/ASAN rebuild passed 1331/1331 on a quiet re-run (its first,
concurrent-with-the-full-build run hit an unrelated, confirmed-flaky PTY
harness timing failure in test_smoke_launch_send_echo_capture_stdout_and_exit_code,
reproduced as system-load-induced, not caused by this change).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
The previous fix (654c6e8c) only closed the missing-dependency-edge race
inside the libxce target itself. A live user report on a second
from-scratch `./tools/xce-build.sh all debug` run showed the same race
still hitting scheduler/src/sim_record.c and core_events.c, which reach
controller.capnp.h directly via a raw -Ilib include path with no
declared dependency on the codegen step at all.
Root cause: libxce_dep = declare_dependency(link_with: libxce) -- the
dependency object controller/worker/scheduler all consume uniformly --
only orders a consumer's LINK step after libxce is built; it never
propagated a build-order dependency on the capnp custom_target to a
consumer's own COMPILE steps.
Fix: add `sources: [gen_srcs]` to libxce_dep's declare_dependency() call.
This kwarg exists specifically for generated files that must be built
before any consumer of a dependency compiles, and since every component
already lists libxce_dep in its own dependencies, this one change
propagates the missing edge everywhere with no per-component changes.
Verified: two consecutive full from-scratch `./tools/xce-build.sh all
debug` builds (1173/1173 targets, the user's own exact repro command)
both succeeded with no retry; a subsequent incremental controller/ASAN
rebuild stayed clean (1331/1331 tests, 0 ASAN diagnostics).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
libxce_src listed only gen_srcs[0] (controller.capnp.c) as a source of
the static_library('xce', ...) target, never controller.capnp.h -- which
several library files (utils.c and others) reach only via #include.
Ninja had no build edge forcing the capnp custom_target to finish before
those files compiled, so a from-scratch build could schedule them first,
intermittently failing with "controller.capnp.h: No such file or
directory". This is the same symptom recorded across 5+ prior recurrences
in buglog.json since 2026-07-16, previously worked around by retrying the
build rather than root-caused.
Fix: pass the whole gen_srcs custom_target (not gen_srcs[0]) into
libxce_src -- Meson still compiles only its .c output, but now orders the
entire target's build after both outputs are generated. Verified via two
consecutive from-scratch `xce-build.sh lib debug` builds (no retry
needed), the full lib test suite (313/313), and an incremental controller
asan rebuild (1331/1331, 0 ASAN diagnostics) to confirm no regression.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
Add one black-box regression test proving process-level help, in-session
help content, and context-scoped tab completion work correctly when
exercised together in one continuous session (context switches +
completion + help lookups), not just individually as each was already
covered by tasks 1.6/2.3/3.5. This closes the final task of the
xce-test-cli-ux implementation plan (23/23).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
Audit every printf() call site in test.c not already covered by the
flagspec-rejection or perror() paths; convert 33 genuine error/warning
messages (expect-jobs, expect-job, wait-primary, help's not-found path,
jobrun/jobsub/jobstart/jobfinish/jobfail validation, resupdate, set) from
stdout to stderr, so a command's successful result data and its
diagnostics never share a stream.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
Wire flagspec_render_help() into cmd_help via new command_t fields
(flagspec/flagspec_count/usage_tail), so every flag-accepting command's
help composes description -> generated flag block -> example/related
notes, with hand-written per-flag lines removed as a duplicate source of
truth. Add missing purpose descriptions, examples, and confirmation-gate
related notes across ~20 commands; fix bug-1711's "matchs" grammar
(Req 11.2); confirm join's corrected maxmem default (bug-1557, Req 12.2)
and the --json flag (Req 5.4) now render correctly end-to-end.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
Bring workerlist/joblist/reslist onto the shared flagspec_t library with an
identically-spelled --json flag, wired to the task-2.2 JSON list formatter
via a new pure field-mapping module (listrow_json.c/.h). Default table
output is unchanged; JSON rows carry raw numeric values (flags, timestamps,
memory sizes) rather than their humanized display strings, per Requirement
5.1's "same fields" contract.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
Gates jobkill and shutdown unconditionally, qctrl only when action is
QUEUE_CLOSE/QUEUE_INACTIVE (not ACTIVE/OPEN), hostctrl only when
HOST_CLOSE (not OPEN), and upgradeconfig unconditionally -- exactly
Requirement 9.5's enumeration, with jstart/jfinish/jfail (worker
context) correctly untouched. Each gains a -y/--yes bypass flag via
flagspec_t; cmd_set gains -y/+y calling confirm_set_skip(). A decline
maps to a plain `return 0` (success-coded, distinct from a negative
failure return) per confirm.h's own documented contract -- this means
a declined destructive command inside a source'd file under `set -e`
won't trip abort-on-failure, a real but narrow interaction only
reachable when a human is actively declining; left for task 4.3 to
test explicitly. 1321/1321 across the full controller suite (+3),
0 regressions.
Task 3.3 of .kiro/specs/xce-test-cli-ux/tasks.md.
Requirements: 9.1, 9.2, 9.3, 9.4, 9.5, 9.6, 9.7, 9.8, 9.9
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
Routes all 26 blocking xcecli_* call sites across ~24 commands through
xcetest_run_interruptible(), wires xcetest_set_active_connection() at
execute_line()'s dispatch point (active_ctx->ctx, read fresh per
command), and fixes main()'s REPL loop to distinguish a real EOF from
an EINTR-interrupted readline() so Ctrl-C at an empty prompt discards
the line instead of exiting the session.
Live-discovered and fixed bug-1794: fill_job_alloc() never initializes
xce_job_alloc_t's nres_shared/allocs_shared fields -- harmless as stack
garbage before this task, genuine heap garbage once moved into a
heap-allocated ctx block (a necessary consequence of the adapter's
heap-only precondition). Fixed by switching all ~26 call-context
allocations from malloc to calloc uniformly. Also fixed an argv-aliasing
hazard: cmd_hello/cmd_jobckpt's -m values aliased optarg, which
execute_line() frees immediately after return including on the
abandoned path -- now strdup'd into the heap block.
Also closed a safety-critical gap this task's own design introduced
(bug-1795): before this task the REPL was fully synchronous, so
leave_ctx() (called by up/quit) could never run while an RPC was
outstanding. An abandoned worker can now persist past command
completion, so leave_ctx() tearing down the connection could
use-after-free it. Fixed via xcetest_wait_connection_idle(), which
blocks on the same per-connection semaphore the adapter already
tracks, then releases it, proving no worker is active before teardown.
Proven with a deterministic white-box RED->GREEN test, since the race
itself is too timing-dependent for a black-box test to reliably force.
1361/1361 across the full controller suite (+2), 0 regressions.
Task 3.2 of .kiro/specs/xce-test-cli-ux/tasks.md.
Requirements: 1.1, 1.2, 1.3, 1.4, 1.5
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
Migrates 17 commands (cmd_hello, cmd_jobinfo, cmd_jobrun, cmd_jobsub,
cmd_resval, cmd_jobkill, cmd_jobckpt, cmd_qctrl, cmd_hostctrl,
cmd_runtasks, cmd_setpend, cmd_jobstart, cmd_jobfinish, cmd_jobfail,
cmd_resupdate, cmd_walcompare, join_worker_parse_attrs) from raw
getopt()/hand-rolled parsing to the shared flagspec_t + getopt_long
mechanism, so every one now uniformly rejects an unrecognized or
malformed flag with a usage summary and, where a close match exists,
a suggestion. cmd_walcompare's hand-rolled parser needed a leading '+'
in its optstring to preserve its pre-migration flags-only-before-
positionals behavior. Also fixes bug-1704: cmd_qctrl's uninitialized
action variable, now sentineled and rejected when unset, mirroring
cmd_hostctrl's existing pattern -- this also makes the destructive-
command gate's -i/-c detection (task 3.3) safe to rely on. Every
existing short flag's letter, value format, and default is unchanged
for a valid invocation. 1357/1357 across the full controller suite
(+4), 0 regressions.
Task 3.1 of .kiro/specs/xce-test-cli-ux/tasks.md.
Requirements: 2.1, 2.2, 2.3, 2.4, 6.1, 6.2, 6.3, 6.4, 10.1, 10.2, 10.3
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
cmd_join_worker's simulated-worker maxmem default was 64 bytes -- a
leftover from when the field was MB-scale, unfixed when maxmem became
bytes-canonical elsewhere. Now 64 * 1024 * 1024. Extracted the
argument-parsing/defaulting logic into join_worker_parse_attrs()
(join_worker_args.c), since it runs entirely before the function's
network call and is a genuine testable seam -- same precedent as
task 2.5's joblist_display.c. Verified live against a real deployed
cluster (workerlist reports MaxMem: 64.00 MB) plus a permanent 5-test
regression suite. maxswap/maxtmp have the identical stale-default bug
(logged as bug-1793) but are deliberately left unfixed here since
USAGE_JOIN_WORKER's help text currently documents their stale values
accurately -- fixing only the code would create a new docs mismatch;
a locked-in sibling-defaults test documents this for the follow-up.
1353/1353 across the full controller suite (+5), 0 regressions.
Task 2.6 of .kiro/specs/xce-test-cli-ux/tasks.md.
Requirements: 12.1
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
Eight display/text defects in test.c: worker_status_str's "UNKOWN"
typo, cmd_runtasks's "sequentail" typo (display-only, verified inert
against the real launch_type consumer), cmd_reslist's right-justified
columns and unlabeled hex Flags (now matching workerlist/state_raw's
conventions), joblist's raw-epoch timestamps (now via a format_job_time
helper delegating to the same formatter ctrlstatus already uses) and
combined NTasks column split into MinTasks/MaxTasks, job_state_str's
missing raw-state fallback for ambiguous RUN+EXIT (mirroring
print_jobinfo's existing pattern), and getconf's no-op group/config
list stubs (now genuinely implemented from already-fetched data,
matching the three sibling print functions' convention).
worker_status_str/job_state_str/format_job_time are pure functions with
no networking/REPL/global-state dependency, extracted into a new
joblist_display.c/.h module with a permanent 20-test standalone suite --
following this codebase's own resval_typeresolve.c/misc.c precedent for
testing functions that would otherwise be stranded inside untестable
test.c. cmd_reslist/joblist_query/cmd_getconf's table-printing bodies
inline fetch+print and remain untestable without a live cluster.
1348/1348 across the full controller suite (+20), 0 regressions.
Task 2.5 of .kiro/specs/xce-test-cli-ux/tasks.md.
Requirements: 11.1, 11.3, 11.4, 11.5, 11.6, 11.7, 11.8
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
cmd_source reuses rcfile_load() unmodified for line execution, adding
an explicit missing-file check (bypassing rcfile_load's silent-missing
policy, which is correct for its one existing startup-rc-file caller
but wrong for an explicit user command) and a realpath()-canonicalized
in-flight path stack that rejects direct or transitive cycles instead
of hanging. Registered in all four command contexts. Translates
rcfile_load's raw positive "aborted" return to -1 so an enclosing
source line correctly sees a transitively-aborted nested source as
failure, not success (preprocess_and_run only treats negative returns
as failure). Proven via real fixture files driven through the black-box
pty harness: a genuine two-file mutual cycle, a missing path, and
ordered execution across a non-cyclic nested source. 1310/1310
controller suite passing (+3).
Task 2.4 of .kiro/specs/xce-test-cli-ux/tasks.md.
Requirements: 13.1, 13.2, 13.3, 13.4, 13.5, 13.6
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
Registers libedit's GNU-readline-compatible rl_attempted_completion_function
under g_interactive only. The candidate generator reads active_ctx->commands[]
live on every call, so completion correctly tracks context switches rather
than freezing at registration time. Single/multi-match UI (auto-complete vs.
double-Tab candidate listing) is entirely libedit's stock behavior once given
a correct context-scoped generator. Proven via a black-box pty test verified
sound against a false-positive risk: xce_test has no fuzzy command-name
matching, so an uncompleted partial token cannot coincidentally trigger the
same effect a real completion would. Also fixes a harness bug (bug-1792):
disabling kernel ECHO for output capture was silently suppressing libedit's
own completion feedback too; added clitest_spawn_ex() with disable_echo as
an explicit parameter, clitest_spawn() unchanged for existing callers.
1307/1307 controller suite passing (+1), re-verified 8/8 for flakiness.
Task 2.3 of .kiro/specs/xce-test-cli-ux/tasks.md.
Requirements: 7.1, 7.2, 7.3, 7.4
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
listfmt_json_begin/new_row/add_string/add_number/finish wrap cJSON
(already linked via struct2json_dep) into a minimal array/object
lifecycle library with no generic "record" abstraction -- each command's
own row struct maps its fields directly when wired in later (task 3.4).
Proven via a real stdout-capture-then-reparse round trip, including a
733-character field with no truncation, and stderr staying empty.
1306/1306 controller suite passing (+9).
Task 2.2 of .kiro/specs/xce-test-cli-ux/tasks.md.
Requirements: 5.1, 5.3
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
confirm_destructive() prompts on stderr and blocking-reads a y/n answer
via a single, non-looping fgets() call (not libedit) -- the mechanism
that makes a real Ctrl-C during the prompt return EINTR and be treated
as a decline (Requirement 9.9), same as an explicit "n" or EOF.
confirm_set_skip()/confirm_is_skip_set() add a session-scoped toggle
mirroring cmd_set's existing -e/+e style. Auto-skips when input is not
interactive, via a new xcetest_is_interactive() accessor over the
existing g_interactive global. Destructive-command enumeration and
per-command wiring (Requirement 9.5) is task 3.3, not this task.
Proven with a real fork+SIGINT test. 1297/1297 controller suite
passing (+7).
Task 2.1 of .kiro/specs/xce-test-cli-ux/tasks.md.
Requirements: 9.1, 9.2, 9.3, 9.4, 9.6, 9.7, 9.8, 9.9
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
clitest_spawn/send_line/send_partial/send_tab/send_signal/wait_for_text/
wait_for_exit/kill_and_reap launch the real compiled xce_test behind a
genuine pty (so isatty/g_interactive/libedit behave as a real terminal),
with stderr routed to a separate pipe for real stream-separation testing.
All waits are poll()-bounded with an explicit timeout so a future
regression can't hang the whole test suite. This is the shared
infrastructure tasks 2.3, 3.5, and all of Validation phase (4.1-4.4)
will build their black-box scenarios on top of. One smoke test proves
launch/send/capture/exit-code/stream-separation end to end.
1290/1290 controller suite passing (+1).
Task 1.7 of .kiro/specs/xce-test-cli-ux/tasks.md.
Foundation phase (1.1-1.7) now complete.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
argp already provided --help/-?/--version with exit-without-REPL
behavior, but had no -h alias (its default short help flag is -? per
GNU argp convention) despite Requirement 4.1 naming -h explicitly.
Added a hidden -h option triggering the same help path, and extended
the doc string to describe the in-session help command. Proven against
the real compiled binary via subprocess tests, not just argp's own
in-process behavior. 1289/1289 controller suite passing (+4).
Task 1.6 of .kiro/specs/xce-test-cli-ux/tasks.md.
Requirements: 4.1, 4.2, 4.3
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
A registry of per-connection locks, acquired (interruptibly) before
spawning a worker and released by whichever side finishes last, so an
abandoned call's worker is never running concurrently with a new call
on the same connection. Uses a POSIX semaphore (sem_t), not a mutex:
review round 1 found the initial pthread_mutex_t implementation relied
on behavior POSIX classifies as undefined for a non-owning-thread
unlock (not just error-checking mutex types) -- this design's worker
thread legitimately releases a lock the caller thread acquired, which
sem_t supports with no ownership concept at all. bug-1791. Design.md
synced to reflect the semaphore choice. 1285/1285 controller suite
passing.
Task 1.5 of .kiro/specs/xce-test-cli-ux/tasks.md.
Requirements: 1.1, 1.5
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
xcetest_run_interruptible() runs a blocking call on a detached worker
thread while the caller polls xcetest_interrupted() every 50ms. A single
atomic CAS (RUNNING -> DONE_UNCLAIMED | ABANDONED) decides which side
frees the heap-allocated call block, so an abandoned call's worker
thread never double-frees or leaks once it eventually finishes. Proven
under active ASAN with a real detached thread and a real SIGINT-forced
abandonment. 1282/1282 controller suite passing (+2).
Task 1.4 of .kiro/specs/xce-test-cli-ux/tasks.md.
Requirements: 1.1, 1.2, 1.3, 1.4
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
Replaces the bare signal(SIGINT, sigbreak) install with sigaction() and
SA_RESTART cleared, so a blocking read reliably returns EINTR instead of
being silently restarted by glibc's BSD signal() semantics. Adds
xcetest_interrupted() as a read-only accessor over the existing break_out
flag for the interruptible-call adapter (tasks 1.4/1.5) to consume.
Proven by a real fork+SIGINT+blocked-read test that first demonstrates
the old bug, then the fix. 1280/1280 controller suite passing (+2).
Task 1.3 of .kiro/specs/xce-test-cli-ux/tasks.md.
Requirements: 9.9
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
flagspec_reject() now names a likely-intended flag when an unrecognized
flag closely matches exactly one entry in the descriptor table (genuine
Levenshtein DP, silent on zero or ambiguous matches). flagspec_render_help()
generates the short/long/required-optional/default flag-listing block from
a descriptor table. 1278/1278 controller suite passing (+5).
Task 1.2 of .kiro/specs/xce-test-cli-ux/tasks.md.
Requirements: 6.1, 6.2, 6.3, 6.4, 8.2, 10.1, 10.2, 10.3
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
Adds flagspec_t descriptor table, getopt_long option-table builder, and
the shared stderr-routed rejection path that every flag-accepting
command will migrate onto in later tasks. TDD'd via test_flagspec.c
(1273/1273 controller suite passing).
Task 1.1 of .kiro/specs/xce-test-cli-ux/tasks.md.
Requirements: 2.1, 2.2, 2.3, 2.4, 3.1
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
The previous fix for the "slots" regression (5b18f14a) worked but was an
unneeded special case: strcmp(vres->res->name, "slots") to fall back to
nslots directly. alloc_req->host already holds exactly that value during
the global phase -- both allocators' own unconditional "add slots" step
stages it there regardless of phase, and plugin_prepare_allocation()
never resets .host during the global phase, so by the time sched_fill_
limit_requests() runs (right after, same invocation), .host contains
exactly "slots"'s job-wide total and nothing else.
Read alloc_req->host FIRST for this combination and fall back to the
rusages-based on-demand computation only when nothing is staged there
(get_resval_by_id()'s "not found" == 0.0, safe here since "slots" is
never legitimately 0 -- seqjob only runs for maxTasks==1, parjob only
for maxTasks>1). Data-driven instead of name-based, matching bug-1789's
own earlier flag-based-dispatch philosophy, and generalizes for free to
any future resource an allocator plugin similarly pre-stages outside
its normal rusages loop.
test_fill_limit_requests_global_limit_on_slots_reads_host now pre-stages
alloc_req->host exactly as the allocators do, rusages left empty,
matching production exactly.
Full xce-build.sh all asan --incremental: 94/94 Ok. Live re-verified
against both the original gitea#46 scenario and the ug1_global_mem
scenario: byte-identical scheduler traces to every prior verification
round.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GLfB5uWRdmx2WiZSFC7Fm7
The limit-plugin-ownership refactor (fa34b038) broke gitea#46's exact
original scenario. calc_wanted_job_total_resval() scans the job's
rusages by name, but "slots" is never a declared rusage entry --
seqjob_prepare_allocation()/parjob_prepare_allocation() derive its want
directly from the job's own task count via their own unconditional
key_find("slots", ...) + store, entirely outside the rusages loop. The
refactor's on-demand lookup therefore always missed for "slots",
silently leaving the computed want at 0 -- reintroducing bug-1789's
exact original symptom (matching worked, but caps never bound).
Live-confirmed against a fresh deploy of gitea#46's own repro (9 plain
sleep jobs, user1, queue normal, no -R flag): limit1/ug1_slots/
ug1_normal_slots all matched correctly but zero unsatisfy_ lines
appeared and all 9 jobs dispatched uncapped. The refactor's own new
unit test had masked this -- it artificially added a "slots" rusages
entry that doesn't reflect production reality.
Fix: sched_fill_limit_requests() now falls back to `nslots` directly
(the same value both allocators already store as "slots"'s own
per-host want) whenever the rusages lookup misses and the resource is
named "slots" -- the one documented exception to the rusages-lookup
rule, matching this codebase's existing "slots" special-casing
convention.
New test test_fill_limit_requests_global_limit_on_slots_uses_nslots
proves the fallback with no "slots" entry in rusages at all. The
original test reworked to use "mem" instead, so it now genuinely
exercises the general on-demand path.
Full xce-build.sh all asan --incremental: 94/94 Ok. Live re-verified
against both the original gitea#46 scenario (byte-identical to the
pre-refactor trace) and the ug1_global_mem scenario -- both correct.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GLfB5uWRdmx2WiZSFC7Fm7
bug-1790's fix worked but put limit-plugin-specific logic in the wrong
place: seqjob_prepare_allocation()/parjob_prepare_allocation() (the
allocator plugins) unconditionally pre-computed a job-wide total for
every per-host resource in the job's rusages, "just in case" some
global limit needed one. That's the allocator plugins' concern leaking
into limit evaluation -- duplicated across both allocators (a third
would need it too), and wasteful (computed for resources no limit
actually references).
Moved ownership to sched_fill_limit_requests() (resource.c), called
once centrally from plugin_prepare_allocation() regardless of which
allocator ran:
- Removed calc_wanted_host_resval_job_totals() and its calls from both
allocator plugins entirely.
- Added calc_wanted_job_total_resval() (resreq.c): an on-demand,
single-resource lookup by name (no resmgr/key_find needed -- the
caller already has the resolved xce_resource_t), computing the same
job-wide-total formula only for the one (scope, underlying-resource)
combination that needs it -- a GLOBAL limit on a per-host resource.
The other three combinations are unchanged, still reading straight
out of alloc_req.
- sched_fill_limit_requests() gains rusages/nhosts/nslots parameters
back (a genuine need this time, not a re-added unused one).
plugin.c's two call sites pass obj->u.primary->resreq->rusages
(NULL for COMPOUND/ALTER, matching this codebase's existing scope
convention for those tags) and ctx->nhosts/ctx->nslots.
All 23 testsuite link-satisfying stubs mechanically updated. Removed
the two allocator-level regression tests from the first pass (the
behavior they proved no longer exists by design); rewrote
test_fill_limit_requests_global_limit_on_host_resource_reads_host to
exercise the new on-demand path directly, nslots=3 to prove the job's
WHOLE total is used.
Full xce-build.sh all asan --incremental: 94/94 Ok, zero regressions.
Live re-verified against the same ug1_global_mem deploy/scenario:
byte-identical scheduler trace to the pre-refactor result, confirming
behavior-preserving.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GLfB5uWRdmx2WiZSFC7Fm7
test_job_rusage_reconciles_mem_reservation_against_usage asserted
w->resInUse was already charged 4.0 for a JOBS_PUSH-dispatched RUN job,
immediately after the push with no round boundary simulated. That
assumption was wrong, not the scheduler: resInUse/sharedInUse is never
charged incrementally at JOBS_PUSH/dispatch time. It's fully re-derived
once per round by sched_account_jobs() (scheduler.c), which iterates
every currently-RUN job and calls sched_charge_account() on each one's
accounts -- called by both the real normal-mode driver (driver.c) and
the sim driver (sim_loop.c), always before anything else in that round
reads resInUse. Confirmed by reading the full JOBS_PUSH handler
(core_events.c) and builtin_fcfs_on_job_push(): neither ever charges or
uncharges resInUse, by design -- core_session.c's own doc comment on
core_begin_session_internal() already documents this "recompute from
source of truth each round" architecture.
Fix: the test now calls sched_charge_account(&core, tracked->accounts)
right after JOBS_PUSH, simulating the one round-boundary reconciliation
step sched_account_jobs() would perform for its single RUN job (that
function itself lives in scheduler.c, not white-box #included here, so
its single-job effect is reproduced directly via the same primitive it
calls internally). No production code changed.
Full scheduler suite: 77/77 Ok (was 76/77). Full xce-build.sh all asan
--incremental: 94/94 Ok, zero regressions.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GLfB5uWRdmx2WiZSFC7Fm7
free_host_account() (src/lib/misc.c) freed acct->accounts[] and acct
itself but never acct->host, the strdup'd string make_host_account()
(xcealloc.c) allocates -- a real leak on every free_job_account() call,
including the live production dispatch path (scheduler.c, core_events.c),
not just test fixtures. Already root-caused and the exact fix specified
in bug-1621 (logged, never applied); applied here: FREEUP(acct->host)
before FREEUP(acct), mirroring free_res_account()'s existing
FREEUP(acct->resname) pattern.
Also closes bug-1723: test_xcealloc.c's two multi-host overflow
regression tests (added by bug-1722's fix) never freed their own
locally-built fixture trees. Added free_JobAlloc_ptr(&jalloc) to
test_make_job_account_multi_host_no_overflow() and
free_HostAlloc_list(2, hallocs) to
test_make_host_allocs_multi_host_no_overflow(), exactly as bug-1723's
own root-cause note specified.
Together these take xcelib_test from a deterministic 456-byte/29-
allocation LeakSanitizer SIGABRT (masking every other libxce test result
behind it) to 1/1 Ok. Full xce-build.sh all asan --incremental: 94/94 Ok,
zero regressions.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GLfB5uWRdmx2WiZSFC7Fm7
Adds ug1_global_mem to limits.yaml: a permanent, real GLOBAL limit on
"mem" (a per-host resource, not "slots"), capping ug1's total
cluster-wide mem usage at 5GB. This is exactly the scenario bug-1790
fixed but only unit-tested until now -- no limit in the shipped example
config exercised a global cap on a non-slots per-host resource.
Live-verified via a fresh single-node deploy: 3 jobs submitted as user1
(ug1 member) each requesting rusage[mem=2000000000] (2GB) on queue
normal. Scheduler trace confirms ug1_global_mem correctly matches and,
for the first time, computes a real nonzero job-wide-total wanted value:
unsatisfy_ resource 36 total 5000000000.000000 inuse 4000000000.000000
uncomit 0.000000 wanted 2000000000.000000
unsatisfy_ unmet resource 36 (limit=ug1_global_mem) shortfall
1000000000.000000
The 3rd job (4GB inuse + 2GB wanted = 6GB > 5GB cap) correctly PENDed
until an earlier job's mem freed up. Deploy config validated with zero
new warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GLfB5uWRdmx2WiZSFC7Fm7
Closes the gap bug-1789 explicitly carved out: a GLOBAL limit built on a
per-host resource other than "slots" (e.g. a future absolute global mem
cap) needs ctx->alloc_req.host to carry that resource's job-wide total
want during the global phase, but only "slots" ever got that treatment
(via seqjob/parjob_prepare_allocation()'s own unconditional store).
Adds calc_wanted_host_resval_job_totals() (resreq.c), sharing its loop
body with calc_wanted_global_resvals() via a new static helper
parameterized only by which XCE_RESF_GLOBAL direction to match -- it
computes the SAME job-wide-total formulas (PER_TASK: value*nslots, PER_JOB:
value, PER_HOST: value*nhosts) for non-global resources instead of global
ones. seqjob/parjob_prepare_allocation()'s global-phase branch now calls
it into ctx->alloc_req.host alongside the existing calc_wanted_global_
resvals() call into .shared.
Safety was traced, not assumed: plugin_prepare_allocation() only resets
.host at the top of a per-host call, never during the global phase, and
sched_fill_limit_requests() for glimits reads .host later in the SAME
global-phase invocation -- transferring the value into the limit's
permanent .shared record before any later per-host call ever resets
.host. An earlier candidate fix (reusing calc_wanted_host_resvals()
directly) was rejected during design: its PER_HOST formula and, for
parjob, its per-bucket nslots don't give a job-wide total.
New regression tests in test_seqjob_allocator.c/test_parjob_
combinatorial.c prove a non-"slots" per-host resource ("mem") now gets
correctly populated; parjob's test uses maxTasks=3 specifically to prove
the formula uses the job's whole nslots, not a bucket's share.
Full suite (xce-build.sh all asan --incremental): 92/94, both failures
pre-existing and unrelated (bug-1762, bug-1733's already-documented
xcealloc.c leak). Not live-verified -- no limits.yaml entry exercises
this path yet.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GLfB5uWRdmx2WiZSFC7Fm7
meson/ninja/capnp/Go only exist inside the hermetic nix dev shell, but
xce-build.sh never entered it itself -- the /xce-build skill documented
wrapping every invocation in `nix develop ./src --command ...` by hand,
so a bare `bash src/tools/xce-build.sh ...` (run directly instead of
through the skill) failed opaquely with "meson: command not found". This
was a recurring, previously-logged gap that resurfaced again today mid
bug-1789 verification.
The script now checks IN_NIX_SHELL at the top and re-execs itself under
`nix develop` if unset, so every caller -- the skill, a bare invocation,
CI, a human at a terminal -- gets it automatically. A nested call (already
inside the shell) is a no-op past that check. README.md's daily-develop
section updated to match; the old "or use nix develop --command" example
is now redundant and removed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GLfB5uWRdmx2WiZSFC7Fm7
sched_fill_limit_requests() used to read a limit's underlying resource's
want from the same vector its own virtual-resource entry gets written to.
That's wrong whenever the two differ in scope -- e.g. a GLOBAL limit built
on top of a per-host resource like "slots" (limits.yaml's
ug1_normal_slots): the limit's own vres entry belongs in alloc_req->shared,
but "slots"'s want is only ever staged into alloc_req->host by
seqjob/parjob_prepare_allocation(), so the old code always read a zero and
silently never enforced the cap.
Now both the read source and write destination are derived independently
from each resource's own XCE_RESF_GLOBAL flag (vres->res for the source,
vres->vres for the destination) -- the same flag
calc_wanted_global_resvals()/calc_wanted_host_resvals() (resreq.c) already
dispatch on, so this is the existing rule applied consistently rather than
a new one. Also drops sched_fill_limit_requests()'s unused xce_scheduler_t*
parameter.
Full scheduler suite: 76/77 (the one failure, test_job_rusage_reconciles_
mem_reservation_against_usage, is bug-1762, pre-existing and unrelated).
Live-verified against a fresh single-node deploy.
Filed bug-1790 to track a known, deliberately out-of-scope residual gap:
a future global limit on a per-host resource other than "slots" would need
calc_wanted_host_resvals() to also run during the global phase for that
resource, which nothing currently does.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GLfB5uWRdmx2WiZSFC7Fm7
While live-verifying gitea#46, found and fixed a cascade of 5 previously
undetected bugs that together made every usergroup-scoped limit in
limits.yaml silently non-functional:
- Named usergroups/hostgroups (users.yaml/hosts.yaml) were never
registered into the scheduler's group managers (groupmgr_add() had zero
production callers), so any limit referencing one by name could never
resolve real membership.
- An unconfigured limit scope (e.g. no "projects:") arrived as an
empty-but-present capnp group, not NULL, so build_consumer() built a
real empty group (matching no one) instead of falling back to "all".
- Three separate call sites conflated key_find()'s negative "not found"
return with a real error, short-circuiting before ever reaching the
"instantiate on first use" logic right below them -- the same defect
class already fixed once before in this codebase (bug-1495).
- Two call sites passed &(sched->core.resmgr) where a plain handle was
expected, one level of indirection too many, causing RTTI's key_find()
to reject it as an invalid handle.
All five are live-verified: fresh cluster deploys now correctly match and
instantiate usage tracking for all 7 limits in limits.yaml on every job
submission, reproduced across multiple independent from-scratch redeploys.
A 6th, architecturally distinct issue remains open (bug-1789, logged, not
fixed): sched_fill_limit_requests() computes a job's consumption of a
slots-scoped global limit by reading the wrong request vector, so the
numeric cap itself still never binds. This needs a design decision on
intended cross-scope semantics, not a mechanical fix, so it's left for a
follow-up.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hb6Edtj4KbS8CrgEWYdA2E
Add --jobs N to xce-build.sh: for meson compile it's passed as ninja's -l N
load-average ceiling (re-checked before every job launch for the whole
build, so it adapts if a concurrent session in this shared sandbox loads
the system mid-build), and as a job-count ceiling (-j/--num-processes,
-p) for meson test/go test, which have no load-average equivalent. No
default -- omitting it preserves prior auto-parallelism behavior exactly.
The xce-build skill now computes N live at invocation time from nproc,
/proc/meminfo's MemAvailable, and /proc/loadavg, rather than a fixed
constant, and passes it through.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DrxU1dSRYNMV9CDTXHFtcR
ControlJob() set xce_jobctrl_t.action to the job_ctrl_type_t action code
(kill/stop/resume/checkpoint), but that field is actually the capnp
JobControl_u_which union discriminant -- a different enum entirely. This
made kill/resume/checkpoint hit request.c's -EINVAL default case, and made
stop collide with the checkpoint discriminant and read through the wrong
union member. Live-verified against a real deployed cluster: kill/stop/
resume/checkpoint now behave correctly with no apiserver crash.
Also additively accept a human-readable job ID ("{cluster}-{apiserver}-
{timestamp}-{seq}") on the job control endpoint, alongside the existing
numeric form xcecli already translates to -- lets a caller hit the REST
API directly without reimplementing xcecli's ID packing. Live-verified via
a direct curl call with the human-readable ID.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hb6Edtj4KbS8CrgEWYdA2E
modelJobToSubmit() never set job_startinfo_t.execuser when converting
a submitted job to the C struct sent to the controller/worker, so
every job ran as the worker's 'nobody' default and failed to exec
(exit 126). Default execuser to the submitting user via a new
resolveExecUser() pure helper, and free the newly allocated C string.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hb6Edtj4KbS8CrgEWYdA2E
sim_reconcile_unaccounted_jobs() (the mechanism that replays historical
DISPATCH decisions in --sim mode) reconstructed a job's allocation but
never called the picker's on_dispatch() hook, unlike the live allocator
path which always does. The job therefore stayed a live candidate in the
picker's own pendJobs bookkeeping, and sim_picker_thread_fn() (which
unconditionally PENDs whatever it is still offered) kept re-offering and
re-PENDing it every subsequent round -- live-observed as an already-
dispatched job receiving a bogus PEND decision (bug-1759: 22/50 jobs in
one batch; bug-1779: 5/10 array-job elements).
This directly contradicts scheduler-sim-decision-parity's own design.md,
which already documented "the picker never re-decides an already-resolved
job" as the correct assumption -- the implementation had drifted from it.
Fixed by calling on_dispatch() immediately after reconstructing the job's
allocation, mirroring sched_allocator()'s own live-dispatch ordering.
test_sim_verify_no_false_mismatch_across_round_separated_dispatch's own
fixture had baked in the stale-PEND symptom as "expected" data; updated to
match the corrected behavior (dispatch-only round 1, empty filler rounds).
Both design docs updated in place with correction notes.
info worker's assigned-jobs listing gated only on job->allocs != NULL,
so a job already at a terminal state (DONE/EXIT) could still appear as
"assigned" if its allocs pointer was ever left stale -- live-observed via
a real captured replay, three DONE jobs kept showing as assigned to a
worker despite the worker's own slot-usage accounting correctly excluding
them. Root cause of the stale allocs itself is not yet pinned (suspected
snapshot save/load inconsistency); this adds the same defensive "terminal
is permanent" exclusion job_state_regresses_terminal()/sim_reconcile_
unaccounted_jobs() already apply elsewhere, at the display layer.
NJOBS/RUN/SSUSP/USUSP (xcecli hosts list -l) and EXEC_HOST (xcecli jobs
list) always showed a hardcoded 1 slot per host regardless of a job's
actual per-host allocation -- rawJobToModel()'s ExecHosts JSONB builder
never read the real per-host "slots" resource from the C job allocation,
so the already-correct downstream slot-counting logic (computeHostSlotUsage,
bug-1743) and EXEC_HOST's "N*host" formatter never had real data to work
with. convert.go now extracts each host's real slots allocation via a new
CGO indexing helper, mirroring the scheduler's own get_nslots_in_alloc()
convention.
Also wires hosts list -l's "CURRENT LOAD USED FOR SCHEDULING" table (tmp/
swap/mem/slots and friends) to the host's real live resource values --
previously entirely hardcoded placeholder strings, never connected to any
backend data.
bug-1554: init_host_attrs() replaces hardcoded placeholder stubs with real
detection -- maxswap from sysinfo()'s totalswap (reusing the same syscall
snapshot already taken for maxmem), maxtmp from a fresh statvfs("/tmp")
total-capacity measurement. No synthetic fallback on detection failure, so
a system with swap genuinely disabled correctly reports 0.
bug-1776: the scheduler simulator's "info worker" assigned-jobs listing now
annotates a job consuming more than one "slots" unit (e.g. "jobid*2"),
using per-host allocation data that was already being walked but discarded.
Falls back to the pre-existing per-job listing when jobs on the same worker
have differing slot counts, since a single suffix on a compressed range
would be ambiguous.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hb6Edtj4KbS8CrgEWYdA2E
test_info_job_reports_state_queue_priority now expects "Tasks: min =
1, max = 1" instead of the pre-fix "min = 0, max = 0" incidental
byproduct (a fixture's parse_select() failing on an unregistered
"type" resource used to leave the primary's task-count fields at
memset-zero; apply_job_task_count() now correctly overwrites them to
the job's real submitted default of 1/1). This was the one deferred
item tracked in tasks.md's Implementation Notes since task 2.1.
Full-suite regression confirmed clean: scheduler/controller/worker
suites pass except the pre-existing, unrelated bug-1758 (predates
this feature, attributed to a separate spec). A full /xce-build all
also surfaces bug-1723, a pre-existing lib-suite fixture-teardown
leak dating to before this feature and outside Requirement 5.1's
named scope (scheduler/controller/worker) -- confirmed untouched by
any of this feature's commits.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
test_parjob_e2e_multislot.c's standing "bug-1770" assertion now
expects the correct, real primary->minTasks/maxTasks (1/JOB_MAXTASKS)
instead of the buggy always-1 value it used to document. The
PATCH_MAXTASKS_FOR_BUG_1770 block that manually overwrote the
already-ingested resreq's task-count fields is removed entirely --
the full multi-host allocation scenario (2-host commit, per-host
slots) now runs end to end through unmodified real ingest. Header
comment updated to record that bug-1770 is fixed by this feature
rather than describing a live gap.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Extends test_maxtasks_wiring_ingest.c with two tests: (1) a resource-
request update never changes a job's already-dispatched task count,
because core_events.c's live JOB_UPDATE handler never rebuilds
job->schedobj (Requirement 1.4); (2) two otherwise-identical jobs
differing only in requested task count don't collide in the
unsatisfiable-resource-request cache on the fresh-ingest path
(Requirement 5.4).
Adds a new file, test_maxtasks_wiring_sim_driver_parity.c, proving the
simulation driver's real static translator chain
(sim_process_replay_event_message()/replay_event_to_core_event())
applies this feature's wiring identically to the real driver -- a
genuine white-box test of the real sim_loop.c code (following
test_sim_event_loop.c's established #include technique), not a mirror
of its logic. This was split into its own file because
test_maxtasks_wiring_ingest.c already links the full scheduler_srcs
object list for its own task-3.1 test, which collides at link time
with white-box #include-ing core_events.c/sim_loop.c in one
translation unit; see tasks.md's Implementation Notes.
No production code changed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Adds test_maxtasks_wiring_ingest.c proving: (1) fresh ingest's
dispatched primary resreq reflects a real multi-task job's requested
count via the real core_build_job()/JOBS_PUSH path; (2) fresh ingest
and snapshot-restore ingest derive the identical task count for an
equivalent job, asserted both against expected constants and via
direct cross-comparison (Requirement 5.6); (3) a single-task job
reaches identical real host-selection/allocation behavior through the
live allocator dispatch chain, proving the feature is a true no-op
for maxTasks==1 (Requirements 3.1/3.2). No production code changed --
tasks 2.1/2.2 already wired the fix; this adds cross-call-site and
behavioral proof neither of those isolated tests attempted.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Adds white-box tests for job_start_remote_task()'s launch-type gate
(MPI types defer to their own remote-launch mechanism; default type
fans out via xcecli_runtask() across every allocated host, verified
via a #define-before-#include mock capturing taskrun/taskhost
contents) and create_mpi_cmdline()'s host-list/command construction
for openmpi and mpich. Zero production code changes -- this closes a
previously zero-coverage gap in existing, unmodified worker code
(Requirement 4.4), independent of this feature's resreq wiring work.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
snapshot_load.c's job-restore path now calls apply_job_task_count()
with the job's already-restored mintasks/maxtasks, right after its
resreq parse succeeds and before snapshot_build_schedobj() reads the
resreq's task count for its fingerprint. This is the simulation/replay
driver's only live call site (xce_sched_core_load_snapshot(), reached
via --sim), independent of and parallel to task 2.1's fresh-ingest
wiring in core_events.c.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
