Replace parjob's return-0 stub with a real multi-host allocator: for each
candidate plan, walk buckets in the given (descending) order, claiming a
distinct host per bucket via a fresh host_iter excluding hosts already
claimed elsewhere in the same plan attempt. On a bucket failure, reverse
every earlier successful claim's provisional debit via
sched_ctx_uncommit_host() before advancing to the next plan (Req 4.2, 4.4,
4.5, 4.6, 4.7).
Fix parjob_prepare_allocation() to compute each bucket's want from that
bucket's own slot count (ctx->nslots) instead of a fixed value, and reset
ctx->alloc_req.host at the top of each bucket's scan so host_iter's
pre-filter can't inherit a stale want from a differently-sized prior
bucket (safe: sched_try_alloc_on_host_() always repopulates it from the
real per-bucket want before its own authoritative check runs).
Add a maxTasks<=1 guard matching Req 4 AC1's precondition, needed because
a snapshot-restored job with unpopulated maxTasks==0 (pre-existing,
tracked separately as bug-1767) would otherwise reach parjob's now-real
allocator instead of being silently ignored as before.
On full success, merges every claimed bucket into one xce_job_alloc_t via
make_job_alloc()'s existing runtime-sized host array, satisfying task
4.3's multi-host commit scope as well.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Add the real xce_alloc_plan_iter_t for parjob: for host-count k=1..maxTasks
(ascending, most-preferred-first), generate one balanced partition of
maxTasks into k buckets (floor/ceil split, remainder to the first rem
buckets), which is descending by construction. Pure function of maxTasks
alone -- never dereferences sched -- with a real pthread_testcancel()
cancellation point and stable post-exhaustion NULL (Req 2.1-2.6, 4.1).
my_alloc()/parjob_prepare_allocation() remain untouched stubs; wiring the
iterator into real allocation is task 4.2.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Add a dedicated equivalence suite (test_seqjob_rewrap_equivalence.c)
covering multi-host fixtures the trivial single-host test doesn't reach:
select-filter-only rejection, resource-only rejection, a combined
select-then-resource-then-success ordering, and per-host debit
correctness. The iteration-order test is grounded in the deleted
pre-rewrap sched_collect_workers() code path (verified via git show
against the parent commit) and the unmodified xlist.c primitive, not
merely in host_iter.c's own traversal, so it proves equivalence to the
old behavior rather than internal self-consistency.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Restructure seqjob's my_alloc() as a trivial single-bucket case over the
shared xce_alloc_plan_iter_t/xce_host_iter_t contracts, replacing its own
inline worker-collection loop. Guard seqjob_prepare_allocation() to
maxTasks == 1, closing the fan-out gap for the parallel allocator's later
fix (Req 9.1, 4.6).
Export sched_try_alloc_on_host_() from resource.c (drop static/va_list,
plain (worker, sched, ctx) signature) so seqjob calls the real per-host
try-alloc leaf instead of a private duplicate; delete the now-dead
sched_try_alloc_on_hosts() variadic wrapper. Also wires host_iter.c into
the production scheduler_srcs list (task 2.1 only added it to testsuite).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Adds alloc_cache.c: xce_cache_key_t, cache_key_ops (the corrected
three-way, 0-means-equal .cmp -- an inverted-boolean version was a
real bug found during design review), and cache_lookup/insert/clear
against a real kvtab_t (constructed matching xce_select.c's
sel_node_table_init() precedent). Cache values are a boxed reason-
category int, not a bare hit/miss sentinel, satisfying Req 7 AC5's
requirement that a hit reproduce the original pend-reason category.
unsatisfiable_cache lives on xce_scheduler_t per Req 7 AC1.
Task 2.2 (scheduler-allocation spec). Requirements 7.1, 7.3, 7.4, 7.5.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Adds host_iter.c: the one shared xce_host_iter_t implementation,
lazily filtering sched->core.workers by select expression
(sched_meet_host_selection, unchanged) and resource availability
(sched_check_host_satisfy, unchanged), with a real cancellation
point per candidate visited. xce_alloc_plan_t gains a ctx field
(xce_sched_ctx_t*) so the iterator can reach the scheduler/select
expression/staged resource-want -- additive to task 1.1's approved
interface, NULL-guarded at init(). Test proves the core design
property this spec exists to establish: a host too small for one
search remains a valid candidate for a later, smaller search via a
fresh iterator instance over the same worker set.
Task 2.1 (scheduler-allocation spec). Requirements 3.1-3.6.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Adds sched_ctx_uncommit_host() (resource.c), the structural inverse
of sched_try_alloc_on_host_()'s existing XVEC_ADD_FLOAT commit call,
scoped to ctx->uncomitted only per requirements.md's Rollback scope
discussion. Test exercises the real sched_ctx_get_uncomit_host()
(schedobj.c) via white-box #include rather than a private
reimplementation, matching test_picker_config_fields.c's established
precedent.
Task 1.4 (scheduler-allocation spec). Requirement 4.2.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Adds xce_alloc_mode_t (XCE_ALLOC_NORMAL=0, XCE_ALLOC_PREEMPT, with
commented-out RESERVE/BACKFILL placeholders) and alloc_mode on
xce_sched_ctx_t, zero-defaulted by sched_ctx_init()'s existing
memset. Purely additive: nothing branches on the field yet -- this
spec only declares the extension point scheduler-preemption needs.
Task 1.3 (scheduler-allocation spec). Requirement 9.5.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Adds rusage_hash to xce_primary_resreq_t (computed once by
parse_resreq_primary() via a new hash_rusages() helper, hashing
rusage content rather than raw struct bytes since resname is a
non-interned strdup'd pointer), the xce_resreq_fingerprint_t value
type, and resreq_fp on xce_primary_schedobj_t, populated once at real
job-ingest time in core_build_primary_schedobj().
Task 1.2 (scheduler-allocation spec). Requirement 7.1.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Defines xce_alloc_plan_t (partition of a job's task count into
per-host slot buckets), xce_alloc_plan_iter_t, and xce_host_iter_t
(init/next/fini vtables, stack-allocate-then-init-in-place convention
matching sched_ctx_init) as reusable abstractions any allocator can
adopt. Proven usable by a trivial no-op reference implementation and
a fresh-per-bucket host_iter composition test, no allocator wired in
yet.
Task 1.1 (scheduler-allocation spec). Requirements 2.1, 3.1.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
An idle scheduler (nothing queued to publish) never noticed a dead
controller connection: sched_needs_rejoin was only ever set as a side
effect of an active client-library call, and an idle round makes none.
After any controller failover with nothing already in flight, jobs
submitted to the new primary stayed PEND indefinitely.
Adds xcecli_is_connection_broken() (libxce) -- a pure, socket-free read
of the transport's already-async-maintained dead/alive flag, set for
free by the existing RDHUP disconnect handler with zero application
calls. sched_check_pending_rejoin() now retries xcecli_join() on either
the existing reactive flag or this new passive signal, so a healthy
idle scheduler still makes zero extra network calls while a genuinely
dead connection self-heals within one ~1s round -- live-verified
against a 3-node HA cluster (rejoin fired ~300ms after new-primary
election with nothing submitted in between).
Also closes out bug-1735 (job memory/CPU usage reporting): confirmed
already fixed on this branch since 2026-08-30, just never marked in
the bug log.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
The terminal step: src/frontend/Makefile (194 lines, the second,
independently-maintained build/test path this whole spec exists to
eliminate) is deleted. Re-ran the full task 4.1 validation suite with
the file genuinely absent (build, meson test --suite apiserver, and
-- going beyond citing task 3.5 -- a real live run-resource-verify.sh
end-to-end run: VERDICT result=PASS, closing design.md's Testing
Strategy "Discovery-path regression" check specifically). Full-repo
sweep: 181 hits, zero unaccounted, byte-identical to task 4.1's
non-Makefile hits (the only delta is the Makefile's own 8
self-referential lines, now gone since the file doesn't exist).
Independently re-verified by the task reviewer.
Meson is now the single build/test system for apiserver and xcecli.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Comment-only edits in 4 files: test_ha_bootstrap.c/test_ha_ctrlhello.c
rename their "make test-integration" pointers to reference the
regression suite (tests/regression/run.sh) directly; apiserver/xcecli
cmd/main.go drop the Makefile clause from their ldflags-injection
comments, keeping only src/tools/go_build. Zero production/test
logic changed -- verified via full controller unit-test suite rerun
(1261 tests, 0 failures) and xcecli build.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Removes every live current-fact reference to src/frontend/Makefile
targets or Makefile-specific fixed paths across 11 docs (steering,
frontend/CLAUDE.md, root CLAUDE.md, deployment docs, and two skill
docs), replacing with the Meson-based equivalents established by
tasks 1-3.2: /xce-build apiserver, meson test --suite apiserver,
direct tests/regression/run.sh invocation.
Round-1 review rejected the first pass: xce-deploy-ha/SKILL.md had
over-corrected by deleting all mention of deploy-ha.sh's still-real
/tmp/apiserver legacy fallback (that script's discovery logic is
unchanged, unlike run-resource-verify.sh's, which genuinely lost its
fallback in task 3.1 -- xce-test-resources/SKILL.md's identical-
looking edit was correct for that reason). Fixed by restoring an
accurate, Makefile-free description of the real fallback at all four
touched sites. Re-reviewed and approved.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Rewords deploy-ha.sh's block comment and two fallback echo lines, and
run.sh's XCE-not-found error, to stop naming the now-deleted
Makefile's build-server/build-cli targets, pointing at /xce-build
apiserver instead. Discovery/control-flow logic in both files is
unchanged (verified line-by-line: no if/elif/else lines touched);
run.sh's XCE default placeholder is untouched by design.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Replaces both `cd frontend && make test` call sites (apiserver and
all cases) with direct `go test` invocations covering the identical
package set/flags, mirroring the xcecli case's already-Makefile-free
style. Verified: apiserver case end-to-end, matching the captured
pre-change baseline exactly; all case verified by proxy (byte-
identical replacement line, apiserver instance fully verified) since
a full /xce-build all run is currently blocked earlier in the
pipeline by two pre-existing, unrelated C-test bugs (bug-1723,
bug-1758) -- noted in Implementation Notes for task 4's awareness.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Adds a new `test('apiserver backend', ...)` in apiserver/meson.build
reusing the existing apiserver_cgo_deps-derived CGO flags (resolving
their @CURRENT_SOURCE_DIR@/@BUILD_ROOT@ placeholder tokens via
Meson's own .replace(), since this target invokes go_compiler
directly rather than through tools/go_build). Adds timeout: 300
(matching the existing controller test's convention) since a
cold-cache CGO+ASAN build exceeds Meson's 30s default.
Verified cold and warm cache; confirmed CGO_ENABLED=1 selects the
real backend/ CGO path, not the //go:build !cgo stub.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Expose the AppImage as a flake output via the nix-appimage input's
mkAppImage (passthru on the xce package) instead of requiring an
external 'nix bundle --bundler github:ralismark/nix-appimage' call.
Update README and build-deployment steering to match.
Adds test_reconcile_info_worker_and_summary_hide_completed_job_after_stray_dispatch,
mirroring bug-1764's actual live-observed shape: a worker with one job
driven to a terminal state and two jobs genuinely still RUN, all real
allocations to the SAME worker. After reconciling a stray DISPATCH
entry for the terminal job's id, info worker excludes it while still
listing the two running jobs, and info summary's running count agrees.
Round 1 review rejected an initial version for using a mismatched host
string on the stray entry ("workerarr-stray" instead of "workerarr"),
which made the info worker/info summary assertions pass vacuously
regardless of whether the fix existed, since repl_info_worker_one()
filters assigned jobs by an exact host-string match. Round 2 fixed the
host string and confirmed via an independently-reproduced mutation
test (fix reverted, the test's own accounting-level pre-check
bypassed) that the REPL-level assertions now genuinely, independently
detect the bug.
design.md's Testing Strategy is corrected: it originally said no new
test was needed, reasoning the existing display-path suites already
covered this; tasks.md's own task 2.2 correctly required a
purpose-built test for this exact scenario instead.
Full scheduler test suite: 63/64 OK, sole failure the pre-existing,
unrelated bug-1758. This is the final task of the
scheduler-sim-reconcile-terminal-guard spec; all implementation tasks
(1.1-2.2) are now complete.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
sim_reconcile_unaccounted_jobs()'s per-job trigger check (sim_loop.c)
now also excludes JOB_IS_DONE(job->state) || JOB_IS_EXIT(job->state),
alongside the existing job->accounts != NULL skip. A terminal job is
never searched for a matching recorded DISPATCH entry, so a leftover
or duplicate DISPATCH decision from a later replay round can no
longer resurrect allocs/accounts on a job that has already completed
- the exact mechanism behind info worker's assigned-jobs listing
showing a finished job as still consuming a worker's resources.
The exclusion mirrors job_state_regresses_terminal()'s existing
"terminal is permanent" invariant (core_events.c) rather than
inventing a new rule. The function's doc comment gains a paragraph
clarifying the terminal exclusion is complementary to, not a reversal
of, the existing PEND/RUN state-independence rationale that motivated
this trigger's original widening.
Both RED-phase tests (task 1.1) pass; the pre-existing PEND
eager-capture regression test and core_events.c's JOBS_PUSH
resync-merge terminal-state tests are unaffected. The analogous
resync-merge branch was investigated and confirmed already correct.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
Adds two failing tests formalizing bug-1764's reproduction: a job
driven to JOB_STAT_DONE (and, separately, JOB_STAT_EXIT) via a real
xce_core_push_events() JOB_STATE_CHANGE call correctly has its
allocs/accounts nulled by the existing end-of-life cleanup, but
sim_reconcile_unaccounted_jobs() then wrongly resurrects both fields
when presented with a stray DISPATCH entry for that same jobid, since
its trigger checks only job->accounts == NULL with no state check.
The pre-existing eager-capture regression test,
test_reconcile_captures_non_run_state_job_when_dispatch_recorded,
already covers Requirement 1.2's "PEND job's early DISPATCH still
captured" case explicitly and needed no duplicate. No production code
touched.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
The Nix build previously fetched Go modules from goproxy.cn and Python
build deps from PyPI during the build phase, relying on network leaking
into the sandbox. This is non-reproducible and breaks whenever the
sandbox resolver is unreachable (e.g. pc13v's loopback dnsmasq).
Go:
- Vendor all three modules (frontend/apiserver, frontend/xcecli,
autoscaler/providers/baidu/src/bcctool) and commit vendor/ trees.
- tools/go_build now builds with GOFLAGS=-mod=vendor and GOPROXY=off
instead of GOPROXY=https://goproxy.cn.
Python (xceadmin):
- Add setuptools + wheel to xce.nix nativeBuildInputs.
- meson-build.sh installs with pip --no-build-isolation --no-index
--no-deps so no PyPI access happens at build time.
xceadmin install script:
- Invoke meson-install.sh via find_program('bash') instead of relying
on its #!/usr/bin/env bash shebang, which fails in the Nix install
sandbox (no /usr/bin/env -> 'could not be run').
Cleanup:
- Drop now-unneeded cacert and impureEnvVars=[HOME] from xce.nix and
xcecli.nix (they only existed to enable the impure network fetch).
Docs:
- README build guide: drop '--option sandbox false' from nix build /
nix bundle commands, note the build is now hermetic, and document the
go mod vendor + git add vendor/ maintenance workflow.
Verified: nix build .#xcecore, .#xcecore_debug, .#xcecli, and .#xce all
succeed with the sandbox enabled.
repl_print_event_summary() (repl_break.c) now compares the candidate
rendered timestamp and displayed session number against a new cache
on xce_sched_repl_state_s (repl_internal.h: last_echo_ts_human[32],
last_echo_session) before printing the "[sim] t=... session=..."
preamble. It prints and updates the cache only when either value has
changed, or no preamble has been printed yet this session (the cache's
zero-initialized default, which no real echoed event can produce);
otherwise the preamble is skipped and the cache is left untouched. The
event's own content line prints unconditionally either way
(Requirements 1.1-1.4, 2.2).
All 6 RED-phase tests (task 1.1) now pass, including the critical
raw-vs-rendered nuance test (Requirement 1.3). No regressions in the
Event Echo Gate suite or the sibling spec's breakpoints-inspection
suite (test_step_echoes_events_before_session_decisions specifically
confirmed still passing under the new dedup behavior).
repl_break.c grew from 949 to 980 lines (+31, 3.3% over the residual
already accepted in scheduler-sim-repl-verbosity task 9.2). Review
judged this small growth as accepted without a fresh empirical split
test: task 9.2's own split experiment (a larger +44-line growth)
already found extracting this file's lookup-table domain reaches only
838 lines (still over cap) while requiring 22 white-box test files'
#include lines to change - disproportionate churn this smaller
increment would only re-confirm.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
Adds 6 failing tests for the Event Echo Preamble Deduplicator
(Requirements 1.1-1.4, 2.2): a second echoed event sharing its
predecessor's rendered timestamp+session omits its preamble; two
events across a session boundary both print their own preamble; the
critical raw-vs-rendered nuance (Requirement 1.3) - two events whose
raw nanosecond timestamps genuinely differ but whose rendered,
second-granularity timestamp and session are identical still dedupe,
with a fixture-premise self-check proving the two raw values really
do render identically; the first echoed event in a fresh session
always prints its preamble; a structural event under continue mode
after suppressed routine events follows the same rule; event content
and ordering are unaffected regardless of the preamble decision.
Lands in a new test_repl_step_verbosity.c (1010 lines, 6 tests) rather
than extending test_repl_event_echo.c (705 lines) - proactive split
per this file family's established practice, since appending would
have pushed that file well past the project's ~1000-line trigger.
Registered in testsuite/meson.build. No production code touched.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
Adds repl_bp_kind_names(char* buf, size_t bufsz) (repl_break.c),
comma-joining BP_KIND_TABLE's tokens in table order into a caller
buffer. repl_print_help() (repl.c) is rewritten into four labeled
sections (Execution Control, Breakpoints, State Inspection, Snapshot &
Parameters) with a single shared " %-38s%s\n" column format, info
config moved into State Inspection next to its siblings, set units
added to Snapshot & Parameters, and the break line's description now
built from repl_bp_kind_names() so it enumerates the complete event
type set. repl_cmd_break()'s unrecognized-event-type error calls the
same accessor, so help and the error message can never disagree with
each other or with BP_KIND_TABLE (Requirements 7.1-7.6). No command's
existence, argument parsing, or other output changed.
repl_break.c grew from 905 to 949 lines (18.6% over the 800-line cap,
up from task 3.2's already-accepted 13%-over residual). Review
empirically tested extracting the BP_KIND_TABLE lookup-table domain
into a new repl_bp_kind.c: it built and passed, but only reached 838
lines (still 4.75% over cap) while requiring 22 white-box test files'
#include lines to be updated - the same disproportionate-churn,
doesn't-achieve-compliance outcome already accepted at tasks 6.2/8.2.
Reverted; 949 lines accepted in place.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
Adds failing tests for Requirement 7 (Discoverable Help and Breakpoint
Event Types): help's four labeled sections in correct order, one
consistent column width across every listed command, info config's
placement alongside its info siblings, set units's placement alongside
set speed/output/verify, the break line's complete BP_KIND_TABLE-derived
event-type listing, and break's own unrecognized-event-type error
listing the same complete set. Each new test hardcodes BP_KIND_TABLE's
current 12 tokens rather than referencing the not-yet-implemented
repl_bp_kind_names() accessor, so RED comes from genuine assertion
failures against today's behavior rather than a link error.
New tests for Requirements 7.1-7.4 land in a new test_repl_help.c
rather than test_repl_execution_control.c (949 lines going in, within
reach of this spec's own ~1000-line split trigger already flagged at
task 2.1) - proactive split per this spec's established practice,
documented in design.md. break's Requirement 7.5 error-message test
lands in test_repl_breakpoints_inspection.c exactly as originally
planned, alongside the pre-existing test_break_unrecognized_event_type_rejected
(left unmodified).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
Rewrites info worker's/info resource's resource-listing loops from a
flat, unconditional fprintf into a 3-pass, section-grouped table
(Consumable/Capacity, Load Metrics, Static Hardware Attributes),
units_mode-aware value rendering (raw/human/both, including swp's
MB->bytes conversion), whole-number formatting with no trailing .00
outside the consumable value=/used= pair, and jobid-range-compression
adoption for info worker's assigned-job listing (Requirements 6.4-6.6,
4.5). All 10 of task 8.1's tests pass. Preserves every named prior-bug
invariant (bug-1727's avail_v-as-total, bug-1613's consumable/
non-consumable split, bug-1593's GLOBAL/SHARED filter, bug-1688's
.caps-bound iteration).
Fixes one pre-existing test (bug-1688, predating this spec) whose
"value=5.00" assertion for a whole-number non-consumable resource
directly conflicts with Requirement 6.5's intentional no-trailing-.00
change - same class of fix as task 4.2's pre-existing-test updates.
repl_info.c grew from 985 to 1267 lines (58% over the 800-line cap,
worse than any other file this spec touched) - but unlike repl.c/
repl_break.c, it was already 23% over cap before this spec began.
Review empirically confirmed a narrow helper-only extraction doesn't
achieve compliance (still 41% over), and a compliance-achieving split
would mean restructuring ~690 lines of pre-existing display logic
this feature doesn't otherwise touch - a larger scheduler-core-level
decision, accepted as a residual rather than folded into this task.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
Adds 10 tests in a new test_repl_resource_table.c covering info
worker's/info resource's resource-listing table/grouping layout,
units_mode-aware rendering (raw/human/both, including swp's MB->bytes
conversion and a no-mutation check on the underlying stored value),
whole-number-vs-fractional formatting, and info worker's assigned-job
listing adopting the real repl_fmt_jobid_range() (Requirements
6.4-6.6, 4.5). All 10 fail as expected against today's flat,
unconditional per-resource fprintf loop and uncompressed listing.
Found and fixed a genuine ASAN stack-use-after-return in the test's
own fixture code (not production): WORKERS_PUSH's handler moves a raw
pointer into a job's resvals rather than deep-copying, so the fixture
must heap-allocate rather than stack-allocate the structures it feeds
through that path - independently reproduced by review.
New file rather than extending test_repl_breakpoints_inspection.c:
landing this content there first pushed it to 3215 lines, 3.2x this
file family's own ~1000-line split trigger already used to justify
splitting test_repl_round_rollup.c/test_repl_event_echo.c for
structurally identical reasons.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
Implements repl_is_bytes_valued_resource()/repl_format_bytes_human()
(mirroring xce_test's confirmed is_bytes_valued_resource()/
format_bytes_human() precedent exactly) and the units_mode setting
(explicitly defaulted to both, not left to zero-init) wired into
set units raw|human|both alongside the existing set speed/output/verify
branches (Requirements 6.1-6.3). All 8 of task 7.1's tests pass.
Split repl_format_ts_human(), the Jobid Range Compressor (task 1.2),
and these two new helpers out of repl.c into a new repl_fmt.c: adding
them in place pushed repl.c to 929 lines (16.1% over the 800-line
cap), and unlike task 6.2's single-function split attempt (which
didn't even achieve compliance), bundling these three pure-formatter
groups together does - repl.c settles at 587 lines, repl_fmt.c at 402,
both comfortably under cap. Required a matching #include line in the
20 test files that white-box #include repl.c, plus extending
test_sim_isolation.c's forbidden-include/symbol scan to cover the new
file, following the repl_rollup.c split precedent from task 2.2.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
Adds 5 pure-function tests in a new test_repl_resource_formatting.c
for repl_is_bytes_valued_resource()/repl_format_bytes_human()
(classification, adaptive B/KB/MB/GB/TB/PB scaling including exact
boundary values, swp's MB->bytes conversion), and 3 tests in
test_repl_snapshot_params.c for set units raw|human|both parsing
(Requirements 6.1-6.3). All fail as expected: link failure for the
not-yet-existing helpers, compile failure for the not-yet-existing
units_mode field/enum.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
Adds repl_build_prompt(), building the readline() prompt fresh each
loop iteration: the plain "(sim) " before any session begins, else
"(sim session=<N> t=<ISO8601-ts>) " using the bare session count (not
+1, matching info session's convention) and the existing
repl_format_ts_human(). Both of task 6.1's tests pass.
repl.c grew 808->852 lines (6.5% over the 800-line cap). Review
empirically tried extracting repl_build_prompt() into a new
repl_prompt.c: it works, but requires updating 19 sibling white-box
test files' #include lists, and even then repl.c only drops to 814
lines - still over cap. Given the function is a small 15-line
formatter and the split doesn't even achieve compliance, accepted in
place rather than forcing disproportionate churn.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
Adds 2 tests in a new test_repl_prompt_context.c covering the dynamic
prompt's plain-before-first-session baseline and its session/timestamp
content in both step and continue modes (Requirement 5.5).
Required genuinely new test infrastructure: libedit's readline() only
writes its prompt text when both stdin and stdout are real ptys, which
no existing REPL test in this codebase's freopen()-from-file harness
can observe. Built a PTY-based capture technique (openpty(), ECHO
disabled via termios) instead - independently reproduced via a
standalone probe and stress-tested 15x with zero flakiness during
review.
New file rather than extending test_repl_execution_control.c (already
at 948 lines, over the 800-line cap), per this spec's established
split precedent.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
Gates sim_print_decision() so the bare per-decision line no longer
prints under continue mode, making the Round Rollup's summary the
sole source of decision output (Requirement 2.3, closing bug-1610).
step mode and non-interactive/headless runs are completely
unaffected (Requirements 5.1, 5.4). Implemented as a file-local
inline wrapper in sim_loop.c rather than repl.h, since the gated
function is static to that file and repl.h can't see repl_state's
internals without leaking them to every other repl.h consumer.
Fixes 3 pre-existing tests (predating this spec, from scheduler-core)
that pinned the now-intentionally-changed bare-line behavior under
continue mode - each fixed differently based on its actual purpose
rather than uniformly: two kept continue mode with their anchor
switched to the Round Rollup's summary (since they specifically test
continue-mode breakpoint interruption, which step mode would
trivialize), one switched to step mode (since its actual invariant -
print-before-breakpoint-check call order - is mode-independent and
continue mode no longer has any observable anchor for it).
Review caught a second, unguarded sim_print_decision() call site in
sim_reconcile_unaccounted_jobs() (added by an earlier, separately-
merged spec) that design.md's boundary never accounted for - per the
review, this is actually the dominant real-world path for DISPATCH
decisions in replay mode, so leaving it unguarded would have defeated
Requirement 2.3's "sole source of decision output" claim for the
common case. Gated it through the same wrapper; added 2 new tests.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
Adds 3 tests in a new test_sim_print_decision_gate.c covering the old
unconditional per-decision echo (sim_loop.c's sim_print_decision())
no longer duplicating the Round Rollup's per-round summary under
`continue` mode (Requirement 2.3), while `step` mode and headless
(non-interactive) batch runs stay completely unaffected (Requirements
5.1, 5.4). This is exactly the duplication bug-1610 reported.
1 of 3 tests fails as expected RED evidence (continue-mode bare
"[sim] PEND jobid=..." line still appears alongside the Round Rollup's
own summary today); the other 2 (step-mode and headless regression
checks) correctly pass since they pin already-correct, unaffected
behavior.
New file rather than extending either sibling: test_repl_round_rollup.c
is already 1022 lines (222 over the 800-line cap), and
test_repl_event_echo.c (704 lines) deliberately keeps its fixtures
job-free (bug-1595 discipline), which this task's decision-producing
fixtures cannot follow.
No production code touched (sim_loop.c/repl.c/repl_rollup.c/repl_break.c
untouched) - tests only, per this task's RED-phase boundary. The actual
gate implementation is task 4.2.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
Implements mode-aware event echo classification, job-arrival range
compression (Req 4.5), and round-collapse flush wiring at all 3
breakpoint-match/structural-event call sites (Requirements 1.1-1.4,
3.5, 3.6). All 4 of task 3.1's tests pass; also closes bug-1609
("continue is as verbose as step").
repl_break.c grew 827->905 lines (13% over the 800-line cap). Unlike
task 2.2's Round Rollup, this logic is wiring inside pre-existing
driver-facing hooks, not a relocatable subsystem - accepted in place
per design.md's established residual-tolerance precedent.
Review independently investigated a one-off ASAN stack-use-after-return
in an unrelated test (sched_sim_record_output_test) surfaced by this
change's stack-layout perturbation: traced to the same documented
pre-existing thread-lifecycle bug class as bug-1598/bug-1632 (an
unjoined picker thread reading a stack-local sim after test-function
return), confirmed via 11/11 clean reruns - logged as bug-1763, not a
regression from this task.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
Adds 4 tests in a new test_repl_event_echo.c covering mode-aware event
echo suppression and ending an active collapsed range on structural
events/breakpoint hits (Requirements 1.1-1.4, 3.5, 3.6). Job-free
fixtures (QUEUE_UPDATE/WORKERS_PUSH only) sidestep bug-1595's
jobid-reuse hazard entirely rather than merely avoiding it. 3 of 4
tests fail as expected RED evidence; the 4th (step-mode regression
check) correctly passes since it exercises unchanged behavior.
New file rather than extending test_repl_execution_control.c (already
over the 800-line cap at 937 lines) - applying the same file-size
lesson from tasks 2.1/2.2 proactively, decided before implementation
rather than after a review round.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
Implements the round-boundary decision summary, collapsing of repetitive
rounds, and the shared repl_round_collapse_flush() entry point
(Requirements 2.1, 2.2, 2.4, 3.1-3.4, 3.6). Fingerprint composition
matches design.md exactly: dispatch/preempt counts from the existing
per-round tally, pend-jobid set state-derived from core->jobs (not
decision events), so Requirement 2.4 holds independent of bug-1611.
All 7 of task 2.1's RED-phase tests now pass.
Went through three review rounds:
- Round 1 approved the logic but rejected repl.c's size (1160 lines,
45% over the 800-line cap). Remediated by splitting the Round Rollup
functions into a new repl_rollup.c, following this codebase's own
established repl.c/repl_break.c/repl_info.c/repl_params.c domain-split
precedent.
- Round 2 approved the split but found the new file wasn't added to the
"simulation isolation" regression guard (sched_sim_isolation_lib +
test_sim_isolation.c's forbidden-include/symbol scans), plus two stale
repl.c-location doc comments left over from the move. Both fixed.
- Round 3 found repl.c still 17 lines over cap post-split (808 after
trimming, from 817) and rejected on that basis. Verified empirically
that closing the remaining gap isn't achievable via comment trimming
alone (even deleting both comments entirely only reaches 801) without
another disproportionate file split for single-digit lines. Accepted
808 lines (1% over) as the final state, consistent with round 1's own
explicit judgment that a small residual is not a blocker once the
original 45%-over violation is resolved - documented in design.md.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
Adds 7 tests in a new test_repl_round_rollup.c covering per-round
decision summaries, collapsing of repetitive rounds, and the shared
repl_round_collapse_flush() entry point (Requirements 2.1-2.4, 3.1-3.4,
3.6). repl_round_collapse_flush() itself is not yet implemented (task
2.2) - build fails at link time, confirming RED.
Split into its own file rather than the originally-planned
test_repl_execution_control.c: that file was already at the project's
800-line cap before this task, and landing these tests there would
have pushed it to 1563 lines. design.md's File Structure Plan is
corrected accordingly (mirrors the existing test_repl_snapshot_params.c
/test_repl_breakpoints_inspection.c split in this same file family).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
delete <id> used atoi(), which silently coerces a non-numeric or
trailing-garbage token to 0 instead of reporting an error, e.g.
"delete abc" printed "no breakpoint with id 0" rather than rejecting
the input. Switched to strtol()+endptr validation, mirroring the same
"fail loud" discipline repl_cmd_step() already established in this
file family. Added test_delete_non_numeric_id_rejected covering
non-numeric, zero, negative, and trailing-garbage ids.
Logged as bug-1761 (buglog.json entry already committed alongside an
unrelated task's changes earlier in this branch's history).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
Implements repl_fmt_jobid_range() in repl.c (per design.md's "no new
production .c file" commitment), grouping job-array identifiers by base
and compressing contiguous index runs into [lo-hi] ranges. All 5 of task
1.1's RED-phase tests now pass.
Went through one review round: an initial GREEN implementation placed
the function in a new file (repl_jobid_range.c) to sidestep a linker
issue in the lean test target from task 1.1. Review found and verified
a cleaner fix: link the test against the full scheduler_srcs object list
(mirroring the existing sched_scheduler_connect_test precedent), which
resolves the same symbols without a new file. Remediated accordingly;
bug-1762 in buglog.json documents the investigation.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
Adds failing unit tests for repl_fmt_jobid_range() covering contiguous
ranges, non-contiguous sub-ranges, multiple array bases, mixed array/
non-array ids, and a too-small output buffer (Requirement 4, AC 4.1-4.5).
The function itself is not yet implemented (task 1.2) - build fails at
link time with undefined reference to repl_fmt_jobid_range, confirming
the RED phase.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
Extends task 4.3's post-join gating to resource_sync_thread()'s
independent periodic reporting pthread, closing the Requirement 3.8 gap
where load-indices sends could interleave with an in-flight job-state
reconciliation report. Also fixes two pre-existing latent bugs
(bug-1756, bug-1757) in resource_sync.c that blocked a truthful RED/GREEN
signal for the new tests.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019ypUeKgbAqLWUkaSJ79poN
Closes bug-1755: worker_send_resync_query() hardcoded pktlimit=0 in
its xce_job_state_query_req() call, so the controller's outbound
OPCode_jobStateQuery always sent every non-terminal job ID in one
unbounded page regardless of count - the query direction never
actually exercised the pagination mechanism task 1.4/7.4 proved
correct, unlike the report direction's real, wired
XCECD_RESYNC_REPORT_PKTLIMIT bound.
Add XCE_RESYNC_QUERY_PKTLIMIT=100, mirroring that exact precedent
(same value - a JobStateQuery entry is a bare jobid, smaller than a
JobStateReport entry), and thread it into the one call site. Existing
small-job-count behavior is unchanged; a rejoin exceeding the cap now
genuinely produces multiple wire pages, proven against the real
production call site (not a bypass) with a 250-job/3-page test.
This is the last task in worker-rejoin-resync's implementation plan.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y4M5MJM2zK6nU43da267Ku
Add test_large_job_count_pagination_correctness, proving both
OPCode_jobStateQuery and OPCode_jobStateReport genuinely span many
wire pages (1042 jobs, 11 pages each direction, including a partial
final page) with zero jobs dropped/duplicated/truncated - exact-count
assertions throughout (page continuity, no-duplicate bitmap, exact WAL
delta, exact hook-fire count), extending task 6.2's 2-page proof by an
order of magnitude. Test-only, no production changes.
Logs bug-1755: the underlying query-pagination mechanism this test
proves correct is never exercised in production - worker_send_resync_
query() hardcodes pktlimit=0, so the controller's real query send
always produces one unbounded page regardless of job count, asymmetric
with the report direction's real XCECD_RESYNC_REPORT_PKTLIMIT bound.
No hard transport-layer size cap exists, so this isn't a demonstrated
failure mode, but it contradicts design.md's explicit "both paginated"
claim in this feature's own headline mechanism. Adds task 7.5 to close
it with a small, well-scoped fix before the feature-level gate, rather
than leaving a silently-false design claim in the completed feature.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y4M5MJM2zK6nU43da267Ku
Add test_burst_recovery_unbounded_by_job_count, reproducing bug-1741's
exact original shape (7 jobs on one host, 6 lost finish reports) in
one reconciliation round and one wire page: 5 finished-success + 1
finished-failure recover correctly and distinctly, while an already-
DONE job re-reported as an unqueried extra (with a deliberately wrong
finish time) is proven structurally unchanged - exact +6 WAL LSN
delta, not >=, so a duplicate append is genuinely caught rather than
silently tolerated. Test-only, no production changes; deliberately
kept to a single page since page-boundary correctness is task 7.4's
separate concern.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y4M5MJM2zK6nU43da267Ku
Add test_failover_during_job_recovers_lost_finish, a focused proof of
the feature's own original motivating scenario (bug-1741): a job
dispatched and RUN with no finish fact ever committed anywhere
(modeling a lost fire-and-forget finish notify), recovered to DONE
with the worker-reported finish time - not stuck in RUN - via the
real rejoin/query/report/apply chain. Test-only, no production
changes; reuses task 7.1's helper infrastructure rather than
duplicating its broader four-outcome coverage.
Updates the existing bug-924 (/tmp exhaustion / flaky teardown) entry
with a 5th occurrence hit during validation, unrelated to this task's
own change.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y4M5MJM2zK6nU43da267Ku
