test(scheduler-sim-decision-parity): live-verify against a real 24-round recording (task 4.3, final)
The original motivating recording (scheduler-replay-20260820-070103.replay, named in bug-1611) no longer exists anywhere on disk -- confirmed via exhaustive filesystem search. Substituted a real, driven 24-round/12-job resource-constrained recording using actual driver.c/builtin_fcfs/resource.c code, replayed through the real, unmodified interactive REPL. This also surfaced and honestly documents a real structural limit: Recorded-Decision Reconciliation only succeeds when a job's recorded DISPATCH decision and its JOB_STATE_CHANGE-to-RUN confirmation land in the SAME replay round (reconciliation searches only that round's own decoded data). Constructing that exact coincidence from organic, continuous scheduling is structurally impossible -- a job can't be both "freshly PEND for this round's picker" and "already RUN before this round's picker runs" at once. One job (ANCHOR_ID) engineers this coincidence using entirely real constructors (matching the same precedent tasks 3 and 4.1 already established); the other 11 jobs arrive/contend/dispatch/finish with zero special-casing across all 24 real rounds. Recorded in tasks.md's Implementation Notes as a scope finding for follow-up: in a real deployment with genuine dispatch-to-confirmation latency, this narrows the feature's real-world coverage of bug-1611/1613's dominant manifestation. RED phase: reverting task 3's reconciliation call site makes this test fail with the exact bug-1611 symptom in its own assertion message, plus 3 sibling tests break too; restored and green. No production code touched (final task in the feature -- all production work landed in task 3). 50/51 scheduler tests pass; the 1 failure is pre-existing bug-1560 (unrelated). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
5d194db9b1
commit
3c4091d88b
3 changed files with 1235 additions and 1 deletions
|
|
@ -80,7 +80,7 @@
|
|||
- _Depends: 3_
|
||||
- _Boundary: Scheduler test suite (regression/verify coverage)_
|
||||
|
||||
- [ ] 4.3 Live-verify against the original motivating recording
|
||||
- [x] 4.3 Live-verify against the original motivating recording
|
||||
- Replay the real recorded session that originally surfaced the resource-accounting and decision-visibility gaps this feature fixes, through to completion
|
||||
- Observable completion: inspecting a worker mid-replay now shows nonzero resource usage for jobs independently confirmed running via their ongoing usage updates, and their dispatch decisions are now visible in step/continue output where they previously never appeared
|
||||
- _Requirements: 1.1, 1.2, 1.3, 3.1, 3.2, 3.3_
|
||||
|
|
@ -101,3 +101,4 @@
|
|||
- (task 4.1) `sim_picker_thread_fn()` is confirmed (by direct read, independently re-confirmed by the reviewer) to be structurally PEND-only -- it has no capacity-check/dispatch path for a job it evaluates fresh. This means task 1.2's fixture (or any single-round, freshly-arriving-jobs fixture) can NEVER be literally replayed-and-diffed for full dispatch/PEND parity -- sim's picker always PENDs a fresh job regardless of whether resource accounting is correct, so that comparison would be tautological for freshly-arriving jobs (though still meaningful for reconciled/already-resolved jobs). If a later task needs "real recording vs. sim replay" parity proof, build it the way task 4.1 did: a real driven normal-mode recording plus a hand-crafted sim-mode fixture modeling the identical facts (including at least one already-resolved-before-replay job to exercise reconciliation), not a literal single-file replay-and-diff. This is a real architectural limit, not a testing shortcut -- true dispatch-decision parity for freshly-arriving jobs is scheduler-allocation's (a separate, not-yet-designed spec) job, out of this feature's scope.
|
||||
- (task 4.2) `builtin_fcfs`'s pend-handling (`builtin_fcfs_on_pend`/`begin_session`) never clears a job from `pendJobs` once it's added -- a still-genuinely-PEND job recurs as a fresh PEND decision in the recorded stream EVERY round/session, not just once. This is pre-existing, unrelated-to-this-feature behavior (also visible in `test_repl_equivalence.c`'s own existing fixture). If you build a multi-round `.replay` fixture with a job that stays PEND across rounds, expect and account for that job reappearing in every round's decoded decisions -- don't assume a PEND is a one-time event.
|
||||
- (task 4.2) Not every reconciliation-adjacent test needs to be discriminating via the "disable the reconciliation call site" toggle to be worth keeping -- a test exercising the "not found, nothing to reconcile" branch through the REAL `sched_sim_driver_run()`/`xce_sched_core_load_snapshot()` path can be identical whether reconciliation runs or not, and still catch real bugs (fabrication, snapshot-load corruption, crashes) that a hand-built-core unit test (task 2.2's) can't reach. Judge a test's value by what regression classes it actually covers, not solely by one discriminating-power toggle.
|
||||
- **(task 4.3 -- IMPORTANT SCOPE FINDING, confirmed structurally by code reading, not speculation)**: Recorded-Decision Reconciliation only succeeds when a job's recorded DISPATCH decision and its recorded JOB_STATE_CHANGE-to-RUN event land in the SAME replay round (reconciliation searches only that round's own freshly-decoded `decoded_session->decisions`; each round's decode is discarded once that round finishes processing). Every test in this feature that exercises the successful-match path (`test_sim_round_boundary_reconciliation.c` task 3, `test_sim_contention_parity.c` task 4.1 "Part B", `test_sim_e2e_original_motivation.c` task 4.3's `ANCHOR_ID`) had to hand-engineer this exact coincidence -- none occurs from organic, continuously-driven scheduling, because a job cannot be "freshly PEND for this round's live picker" and "already RUN before this round's picker runs" at the same time (`builtin_fcfs`'s dispatch/state-change hooks make these mutually exclusive by construction). In any real deployment with genuine dispatch-to-worker-confirmation latency (the normal case for an async, network-distributed scheduler), a job's DISPATCH decision and its RUN confirmation will typically land in DIFFERENT rounds -- meaning this feature, as designed and implemented, likely does NOT fix bug-1611/bug-1613's dominant real-world manifestation ("a job dispatched at round 32, confirmed running by some later round, still shows resInUse=0/no visible decision when inspected at round 100"). It only fixes the narrower case of same-round dispatch+confirmation coincidence. If bug-1611/1613's common case needs fixing, the follow-up would need reconciliation to search backward across MULTIPLE prior rounds (or maintain a persistent jobid-keyed index of not-yet-consumed recorded decisions across the whole replay, not just the current round) -- a materially different, larger design than what this feature built. Flagged for the user/future spec, not something task 4.3 could fix within its validation-only boundary.
|
||||
|
|
|
|||
|
|
@ -854,6 +854,29 @@ sched_sim_contention_parity_test = executable(
|
|||
dependencies: [scheduler_test_deps],
|
||||
install: false)
|
||||
|
||||
# Live-verify for task 4.3 (Req 1.1-1.3, 3.1-3.3, FINAL task of this spec): a
|
||||
# real, driven 24-round, 12-job, resource-constrained recording (real
|
||||
# builtin_fcfs picker + a capacity-aware allocator, same technique as
|
||||
# sched_sim_resource_contention_test/sched_sim_contention_parity_test, but
|
||||
# extended far past a 1-2-round toy fixture) is replayed through the real,
|
||||
# unmodified interactive REPL. Asserts "info worker" reports the
|
||||
# arithmetically-correct, nonzero used= value at four checkpoints spanning
|
||||
# the whole replay, and that the one job modeling "already resolved before
|
||||
# this replay's own live decision-making reaches it" (Recorded-Decision
|
||||
# Reconciliation's own designed trigger, unreachable from a purely organic
|
||||
# session per this file's own header comment) has its DISPATCH decision
|
||||
# visible in step/continue output -- bug-1611's own documented symptom,
|
||||
# fixed. Substitutes for the original bug-1611/bug-1613 motivating
|
||||
# recording (`scheduler-replay-20260820-070103.replay`), confirmed absent
|
||||
# from this filesystem before this file was written -- see this file's own
|
||||
# header comment for the full transparency note.
|
||||
sched_sim_e2e_original_motivation_test = executable(
|
||||
'sched_sim_e2e_original_motivation_test',
|
||||
['test_sim_e2e_original_motivation.c'],
|
||||
include_directories:[incdir],
|
||||
dependencies: [scheduler_test_deps],
|
||||
install: false)
|
||||
|
||||
# Contract test for the interactive simulation REPL's execution-control slice
|
||||
# (scheduler-core spec, task 12.1, Req 5 AC 22): starting simulation with
|
||||
# --interactive pauses before any event is processed, and a `step N` command
|
||||
|
|
@ -1196,6 +1219,9 @@ test('scheduler sim snapshot reconcile gap test',
|
|||
test('scheduler sim contention parity test',
|
||||
sched_sim_contention_parity_test, suite: 'scheduler',
|
||||
env: ['ASAN_OPTIONS=detect_leaks=0'])
|
||||
test('scheduler sim e2e original motivation test',
|
||||
sched_sim_e2e_original_motivation_test, suite: 'scheduler',
|
||||
env: ['ASAN_OPTIONS=detect_leaks=0'])
|
||||
test('scheduler repl execution control test',
|
||||
sched_repl_execution_control_test, suite: 'scheduler',
|
||||
env: ['ASAN_OPTIONS=detect_leaks=0'])
|
||||
|
|
|
|||
1207
src/scheduler/testsuite/test_sim_e2e_original_motivation.c
Normal file
1207
src/scheduler/testsuite/test_sim_e2e_original_motivation.c
Normal file
File diff suppressed because it is too large
Load diff
Loading…
Add table
Add a link
Reference in a new issue