sim_reconcile_unaccounted_jobs() (sim_loop.c) was calling picker->picker_iface->on_dispatch() directly from the sim driver thread, violating plugin.h's documented "allocator thread only" contract for Post-allocation hooks. Dormant in practice only because builtin_fcfs.c happens to lock internally around every hook -- a defensive measure the contract text doesn't actually require of picker plugins. Since scheduler-sim-live-allocation, sim mode spawns the same real sched_job_picker()/sched_allocator() thread pair normal mode uses, so a real allocator thread now exists to route through. Added a reconcile_req/reconcile_ack cbuf pair on xce_sched_core_t and a reserved XCE_SCHED_RECONCILE_WAKE sentinel pushed through jbuf to wake the allocator thread without touching jbuf's pinned capacity-1/item-type invariant (Req 5 AC 7). sched_allocator() services the request and calls on_dispatch() itself; sim_reconcile_invoke_on_dispatch() blocks on the ack. Falls back to a direct call when no allocator thread is live for a given core (standalone unit tests), which is safe by construction. TDD: new test spawns a stand-in allocator-relay thread and asserts on_dispatch() fires there, not on the caller's thread -- confirmed RED against the old direct-call code, GREEN after. Four other test files with hand-rolled sched_allocator() stubs needed the same relay branch (found by an actual hang at meson's timeout). Full scheduler suite: 85/85, zero regressions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
30 KiB
Design Document
Overview
Purpose: This feature closes the residual gap in scheduler-sim-decision-parity's Recorded-Decision Reconciliation mechanism (sim_reconcile_unaccounted_jobs(), sim_loop.c) so a simulation-mode replay correctly captures a job's resource allocation regardless of how many rounds separate its recorded DISPATCH decision from its RUN-state confirmation — the realistic case in any real deployment, and the dominant real-world manifestation of bug-1611/bug-1613.
Users: Scheduler operators debugging or tuning via xce_scheduler -i -s <replay> (interactive) or batch --sim replay.
Impact: Changes when sim_reconcile_unaccounted_jobs() is willing to search a round's recorded decision stream for a job — from "only if the job is already RUN this round" to "whenever the job has no captured allocation yet, regardless of its current state." No other function changes.
Goals
- Capture a job's recorded DISPATCH allocation at the round it actually occurred in the original recording, independent of the job's state at that moment.
- Transition a captured job to a running state synchronously, at the SAME round its dispatch decision is captured -- mirroring normal-mode's own
sched_commit_decision()semantics (bug-1624) -- so resource-in-use accounting reflects it starting that same round viasched_account_jobs()'s existing unchangedJOB_IS_RUNgate. (Revised 2026-09-11 -- see below.) - Make the dispatch decision observable (printed, recorded, REPL-checked) at the historically accurate round.
Non-Goals
- Any change to how/when a job transitions to a NON-running state (DONE/EXIT/preemption). (The RUN transition itself is IN scope as of the 2026-09-11 revision below -- see Correction.)
- Any change to simulation mode's live picker (still PEND-only for freshly-arriving jobs, per
scheduler-allocation). - Any change to
--sim-verify's comparison semantics beyond not introducing a new false mismatch. - Reopening or modifying
scheduler-sim-decision-parity's own requirements, design, or tasks, or any normal-mode (driver.c) behavior.
Correction (2026-09-11, user-directed, live replay session): the original version of this Overview described capture and RUN-state transition as fully decoupled, with sched_account_jobs()'s existing JOB_IS_RUN gate providing "correct, state-gated" deferral until a job's SEPARATELY-recorded JOB_STATE_CHANGE confirmation event replayed -- potentially many rounds after capture. Live use surfaced the resulting operator confusion directly (info jobs showing PEND for a job [sim] DISPATCH had already printed), and investigation confirmed normal-mode's own sched_commit_decision() (scheduler.c, bug-1624) does NOT defer this way -- it sets job->state = JOB_STAT_RUN synchronously in the same operation that captures the allocation. This design is revised so simulation-mode reconciliation now does the same: sim_reconcile_unaccounted_jobs()'s capture step ALSO sets job->state = JOB_STAT_RUN, so sched_account_jobs()'s unchanged JOB_IS_RUN gate now includes the job starting the SAME round as capture, not a later one. The later, separately-recorded JOB_STATE_CHANGE/JOB_UPDATE-to-RUN event, when it replays, becomes a safe no-op re-affirmation (RUN -> RUN) rather than the sole state-transition trigger. See Requirement 1's own 2026-09-11 revision (requirements.md) and research.md's corresponding Research Log entry for the full rationale.
Correction (2026-09-07, bug-1759/bug-1779): the "live picker" Non-Goal above was never meant to license sim_reconcile_unaccounted_jobs() leaving a reconciled job as a live picker candidate — it means "do not give the live picker real resource-matching allocation logic" (still out of scope, still scheduler-allocation's territory). Whether a reconciled job stays in the picker's own pendJobs bookkeeping is a scheduler-sim-decision-parity concern (see that spec's design.md, "Integration with --sim-verify", which already documented "the picker never re-decides an already-resolved job" as the correct assumption). The implementation had drifted from that documented assumption: sim_reconcile_unaccounted_jobs() never called picker_iface->on_dispatch(), so a reconciled job stayed a live candidate and was live-observed being re-offered and re-PENDed every subsequent round (bug-1759: 22/50 already-dispatched jobs in one batch; bug-1779: 5/10 array elements). Fixed by adding that missing on_dispatch() call, mirroring sched_allocator()'s own live-dispatch ordering (scheduler.c) exactly — a correction to already-agreed-upon behavior, not a new capability, so it did not need to reopen this spec's own requirements/tasks. test_sim_verify_no_false_mismatch_across_round_separated_dispatch (test_sim_verify.c) was updated to match: rounds 2-3 of its fixture are now empty ReplaySessions (no more picker-echoed PEND for an already-reconciled job) rather than baking in the pre-fix stale-PEND behavior as "expected."
Correction (2026-09-21, bug-1838, superseding the 2026-09-11 revision above): the 2026-09-11 widening (capture "regardless of the job's current state," including PEND) turned out to be the root cause of a live-observed over-dispatch: a 22-slot host took 44 concurrent single-slot jobs during replay (bug-1836), because a PEND job's recorded DISPATCH decision was force-applied by reconciliation with no capacity check against current resource state. bug-1836 first closed the immediate hole by adding a capacity gate (sim_reconciled_alloc_fits_capacity()) to reconciliation itself — but investigation into why a PEND job could ever reach this force-apply path at all found that the underlying justification for the 2026-09-11 widening no longer holds: the since-completed scheduler-sim-live-allocation spec gave simulation mode a genuine, capacity-checked live picker/allocator thread pair (sched_job_picker()/sched_allocator(), sched_election.c) that both drivers now spawn identically — "one core, two consumers" (.kiro/steering/scheduler-driver-equivalence.md) is now actually true for dispatch, not just aspirational. A freshly-PEND job now correctly reaches that SAME live capacity-checked path simulation mode itself runs, exactly like normal mode; reconciliation force-applying a stale recorded decision for it is redundant at best and a second, competing source of truth at worst. The trigger is therefore narrowed back to RUN-only (JOB_IS_RUN(job->state), mutually excluding DONE/EXIT for free under this codebase's 4-state job model): reconciliation's remaining, genuinely necessary scope is a job already confirmed RUN (via a separately-recorded JOB_STATE_CHANGE/JOB_UPDATE event) whose allocation data was never captured — structurally invisible to the live picker, since next_job() never offers a RUN job. This does NOT reopen the 2026-09-11 revision's OTHER change (synchronous RUN-transition-at-capture for the jobs reconciliation still does handle) — that remains correct and unchanged; only the ELIGIBILITY condition for entering capture at all is revised. See sim_reconcile_unaccounted_jobs()'s own updated header comment (sim_loop.c) for the complete before/after rationale, and Requirement 1's own 2026-09-21 correction (requirements.md) for the corresponding Acceptance Criteria update.
Correction (2026-09-21, bug-1837, user-directed): the on_dispatch() call bug-1759/bug-1779's own correction above added (2026-09-07) called picker_iface->on_dispatch() directly from sim_reconcile_unaccounted_jobs()'s own caller thread (sched_sim_driver_run()'s driver thread) — but plugin.h documents on_dispatch()/on_pend() as "Post-allocation hooks (MUST) -- allocator thread, after each allocation attempt," a threading-contract violation an /xce-audit-invariants scheduler run flagged as dormant/inert (real only because builtin_fcfs.c's own internal locking happened to make every call site safe, a defensive measure plugin.h's contract text does not actually require of picker plugins in general). Now that scheduler-sim-live-allocation gives simulation mode the SAME real sched_job_picker()/sched_allocator() thread pair normal mode spawns (see the 2026-09-21 bug-1838 correction above), a real allocator thread genuinely exists to route this call through instead of calling it directly: sim_reconcile_invoke_on_dispatch() (sim_loop.c) relays the request to sched_allocator() (sched_election.c) via two dedicated cbufs on xce_sched_core_t (core.h) -- reconcile_req/reconcile_ack -- waking it through a reserved jbuf sentinel (XCE_SCHED_RECONCILE_WAKE) rather than widening jbuf's own pinned item type/capacity (test_core_jbuf_capacity_pin.c, Req 5 AC 7). A standalone caller with no live allocator thread for its core (e.g. test_sim_reconcile_unaccounted.c's own direct-call unit tests) still gets a direct call, safe by construction since only one thread exists to touch picker state in that case. This is a threading-mechanism correction only -- reconciliation's own trigger condition, capacity gate, and RUN-only eligibility (both corrections above) are unchanged.
Boundary Commitments
This Spec Owns
- The trigger condition of
sim_reconcile_unaccounted_jobs()(sim_loop.c) — which jobs are eligible to be searched for a matching recorded DISPATCH decision each round. - The test coverage proving capture now works across round separation, still respects the existing no-fabrication/no-double-capture guarantees, and does not regress
--sim-verifyor interactive/batch parity.
Out of Boundary
sched_account_jobs()'s ownJOB_IS_RUNcharging gate (scheduler.c:1258) — unchanged; it is exactly the mechanism this feature relies on to turn a captured job's RUN state (now set at capture time, per the 2026-09-11 revision above) into an actual resource charge.- The match/capture mechanics once a job is eligible (jobid matching, ownership transfer of
xce_job_alloc_t,make_job_account(), shift-down compaction, decision reporting viamake_dispatch_decision()/sim_print_decision()/xce_repl_check_decision()) — all unchanged, all already correct perscheduler-sim-decision-parity. - Normal-mode (
driver.c) behavior, simulation mode's live picker,--sim-verify's comparison algorithm itself, and any NON-running-state job transition logic (DONE/EXIT/preemption) — the RUN transition triggered by capture is now IN boundary as of the 2026-09-11 revision (see Overview Correction).
Allowed Dependencies
sim->core.jobsiteration anddecoded_session->decisions(both already owned/populated bysim_reconcile_unaccounted_jobs()'s existing implementation) — P0.sched_account_jobs()'s existingJOB_IS_RUNgate as the sole mechanism deferring actual resource charging — P0, must remain unmodified for this design to be correct.make_job_account()/make_dispatch_decision()(existing, unmodified) — P0.
Revalidation Triggers
- Any future change to
sched_account_jobs()'s charging gate (e.g. adding a new precondition beyondJOB_IS_RUN) must re-verify this feature's Requirement 1.2/1.3 still hold, since this design relies on that gate — not a new one of its own — for correct deferral. - Any future change that makes simulation mode's live picker emit a DISPATCH decision (currently PEND-only,
scheduler-allocation's stated scope) must re-verify Requirement 3.3's dedup assumption, which currently holds structurally (no live DISPATCH ever competes with a reconciled capture) rather than via an explicit check in this feature's own code. - Any change to
decoded_session's per-round decode/free lifecycle (sim_loop.c:871/:1006) must re-verify this feature's core premise (every round's own decisions are only ever visible during that round).
Architecture
Existing Architecture Analysis
sim_reconcile_unaccounted_jobs() runs unconditionally once per simulation round, strictly between xce_core_begin_session() (which zeroes resInUse/sharedInUse) and sched_account_jobs() (which re-sums them from every JOB_IS_RUN job with job->accounts != NULL) — see sim_loop.c:912-957. Its current trigger gate additionally requires JOB_IS_RUN(job->state) before considering a job at all:
if ((job == NULL) || !JOB_IS_RUN(job->state) || (job->accounts != NULL)) {
continue;
}
decoded_session (this round's decoded recorded-decision stream) is freed at the end of the same round (sim_loop.c:1006) and never persisted — so a job skipped at round N because it was still PEND has no later opportunity to be matched against round N's data, even after it reaches RUN at round N+k.
Architecture Pattern & Boundary Map
flowchart TD
RoundStart[Round begins] --> Decode[decode_ReplaySession this round]
Decode --> Begin[xce_core_begin_session resets resInUse]
Begin --> Reconcile[sim_reconcile_unaccounted_jobs]
Reconcile --> Gate{job accounts is NULL}
Gate -- no --> NextJob[skip already-captured job]
Gate -- yes --> Search[search this rounds decoded decisions for jobid]
Search --> Found{matching DISPATCH found}
Found -- yes --> Capture["capture allocs and accounts, set state=RUN synchronously (2026-09-11), report decision"]
Found -- no --> Retry[leave accounts NULL and state unchanged, retry next round]
Capture --> Charge[sched_account_jobs]
Retry --> Charge
NextJob --> Charge
Charge --> ChargeGate{job state is RUN and accounts set}
ChargeGate -- yes --> ResInUse["include in resInUse -- now true the SAME round as capture, not a later one"]
ChargeGate -- no --> Defer[exclude from resInUse]
Architecture Integration:
- Selected pattern: single-condition removal from an existing, already-correct-in-every-other-respect mechanism — not a new component.
- Domain/feature boundaries: capture (this feature,
Gate/Search/Capture/Retryabove) stays fully decoupled from charging (sched_account_jobs,ChargeGateabove, untouched) — the same separation normal mode already has betweensched_commit_decision()(capture) andsched_account_jobs()(charge). - Existing patterns preserved: per-round re-invocation, jobid-scoped matching, ownership-transfer capture, shift-down compaction, three-call decision reporting — all untouched.
- New components rationale: none — this is a boundary-widening change to one existing condition.
- Steering compliance: directly closes the gap
.kiro/steering/scheduler-driver-equivalence.mddocuments as bug-1611's residual; testing discipline (contention fixture) from that same steering doc drives the Testing Strategy below.
Technology Stack
| Layer | Choice / Version | Role in Feature | Notes |
|---|---|---|---|
| Scheduler core (C, gnu17) | Existing sim_loop.c/scheduler.c |
Trigger-condition change only | No new dependency; no schema/wire-format change |
File Structure Plan
Modified Files
src/scheduler/src/sim_loop.c— remove!JOB_IS_RUN(job->state)fromsim_reconcile_unaccounted_jobs()'s trigger condition (sim_loop.c:592); update the function's header doc comment (sim_loop.c:~505-565) to describe the new state-independent trigger and reframe the "double-application guard" description to also cover "this is now the sole capture-eligibility gate."src/scheduler/testsuite/test_sim_reconcile_unaccounted.c— replacetest_reconcile_skips_non_run_state_job(asserts the bug: a PEND job with a matching recorded entry must stay uncaptured) with an inverted-expectation test proving a PEND job is now captured; add new tests for cross-round separation,job->statetransitioning after capture, the no-recorded-origin-ever case, and the "captured-but-not-RUN contributes zero to resInUse" case (Requirement 1.2) driven throughsched_account_jobs().src/scheduler/testsuite/test_sim_contention_parity.cand/or a new contention-focused test file — add a resource-contention fixture (capacity insufficient for all candidate jobs) exercising cross-round dispatch-to-confirmation separation, perscheduler-driver-equivalence.md's explicit testing discipline that a no-contention fixture cannot prove accounting correctness.- A
--sim-verifyregression test (existingtest_sim_*verify-mode file, or a new one if none fits) spanning a multi-round-separated dispatch — proves Requirement 4.2.
No New Files
This feature introduces no new production source file — see Simplification decision in research.md.
System Flows
sequenceDiagram
participant Replay as Replay stream
participant Recon as sim_reconcile_unaccounted_jobs
participant Job as job in core jobs
participant Acct as sched_account_jobs
Replay->>Recon: round N decoded decisions include DISPATCH for job X
Note over Job: job X state is PEND at round N
Recon->>Job: capture allocs and accounts from DISPATCH; set state=RUN synchronously (2026-09-11)
Recon->>Recon: report decision at round N
Acct->>Job: check state is RUN and accounts set
Acct->>Acct: charge resInUse starting round N (same round as capture)
Replay->>Job: round N+k replays job X's separately-recorded state-change-to-RUN event
Note over Job: state already RUN -- idempotent re-affirmation, not a new transition
Recon->>Job: accounts already set, skip re-search
Key decisions not obvious from the diagram (revised 2026-09-11): the decision is reported at round N (the historically accurate round, Requirement 2.1). As of the 2026-09-11 revision, charging ALSO begins at round N, not round N+k — job->state is set to RUN synchronously with capture, mirroring normal-mode's own sched_commit_decision() exactly (bug-1624), so reporting and charging are no longer decoupled by round. The separately-recorded JOB_STATE_CHANGE/JOB_UPDATE-to-RUN event at round N+k, when it later replays, is a safe no-op re-affirmation of an already-RUN job (core_events.c's handler simply reassigns job->state = new_state, RUN → RUN, with no guard triggered) — it is preserved in the replay stream and still processed, but no longer the operative trigger for the job's own running-state transition or its resource charge.
Requirements Traceability
| Requirement | Summary | Components | Interfaces | Flows |
|---|---|---|---|---|
| 1.1 | Capture at originating round regardless of state | sim_reconcile_unaccounted_jobs |
trigger gate | Search/Capture |
| 1.2 (revised 2026-09-11) | Capture ALSO transitions job to RUN synchronously, same round | sim_reconcile_unaccounted_jobs |
Capture step (new job->state = JOB_STAT_RUN write) |
Capture |
| 1.3 (revised 2026-09-11) | resInUse includes the job starting the SAME round as capture | sched_account_jobs (unchanged, relied upon) |
JOB_IS_RUN charge gate |
ChargeGate/ResInUse |
| 1.4 (new 2026-09-11) | Later separately-recorded confirmation event is an idempotent re-affirmation | core_events.c's XCE_CORE_EVENT_JOB_STATE_CHANGE handler (unchanged, verified safe) |
n/a | n/a |
| 1.5 | Matches normal-mode's equivalent-point reporting | sched_account_jobs (shared with normal mode) |
n/a | n/a |
| 2.1 | Decision visible at historically accurate round | sim_reconcile_unaccounted_jobs (decision reporting, unchanged) |
make_dispatch_decision/sim_print_decision/xce_repl_check_decision |
Capture step |
| 2.2 | Original decision type/detail preserved | make_dispatch_decision (unchanged) |
n/a | Capture step |
| 2.3 | No duplicate report when live decision also exists | live picker (unchanged, PEND-only — structural invariant) | n/a | n/a |
| 3.1 | Never capture from a non-originating decision | jobid match (unchanged) | entry->body.dispatch->alloc->jobid check |
Search |
| 3.2 | Never re-capture once captured | job->accounts != NULL guard (unchanged, now the sole gate) |
trigger gate | Gate |
| 3.3 | Already-captured job not overwritten by a later live decision | structural (live picker never emits DISPATCH) | n/a | n/a |
| 4.1 | Same-round case still works | trigger gate (RUN is a subset of "any state") | trigger gate | Gate/Search |
| 4.2 | No new --sim-verify mismatch |
compaction ordering (unchanged), verified by new test | decoded_session compaction | Capture step |
| 4.3 | Interactive/batch parity preserved | shared sched_sim_driver_run() call site (unchanged) |
n/a | n/a |
| 4.4 | No recorded origin anywhere: report unavailable, keep going | "not found" branch (unchanged) | TRACE log, job->accounts stays NULL |
Retry |
| 4.5 | Normal mode unaffected | driver.c untouched |
n/a | n/a |
Components and Interfaces
| Component | Domain/Layer | Intent | Req Coverage | Key Dependencies (P0/P1) | Contracts |
|---|---|---|---|---|---|
sim_reconcile_unaccounted_jobs |
Scheduler sim driver | Widen capture eligibility to any not-yet-captured job, regardless of state | 1.1, 2.1, 3.1, 3.2, 4.1, 4.4 | sched_account_jobs (P0, relied on unmodified) |
Service |
Scheduler sim driver
sim_reconcile_unaccounted_jobs
| Field | Detail |
|---|---|
| Intent | Capture a job's recorded DISPATCH allocation as soon as the replay reaches the round it occurred in, independent of the job's current state |
| Requirements | 1.1, 2.1, 3.1, 3.2, 4.1, 4.4 |
Responsibilities & Constraints
- Owns only the capture-eligibility decision (which jobs are searched each round) and the search/capture/report mechanics already delivered by
scheduler-sim-decision-parity(unchanged). - Has no knowledge of resource charging — that remains
sched_account_jobs()'s exclusive responsibility. - Never fabricates data: a job with no matching recorded decision in a given round is left uncaptured and re-checked the following round (Requirement 4.4).
Dependencies
- Inbound:
sched_sim_driver_run()'s round loop — calls this once per round, unchanged call site and unchanged position relative toxce_core_begin_session()/sched_account_jobs()/xce_core_signal_go()(P0). - Outbound:
sched_account_jobs()— this feature suppliesjob->accountsearlier than before;sched_account_jobs()'s own unmodifiedJOB_IS_RUNgate determines when that data starts being charged (P0).
Contracts: Service [x] / API [ ] / Event [ ] / Batch [ ] / State [ ]
Service Interface
/* Unchanged signature. Behavior change #1 (original): a job is now
* eligible for capture whenever job->accounts == NULL, regardless of
* job->state (previously additionally required JOB_IS_RUN(job->state)).
* Behavior change #2 (2026-09-11 revision): a successful capture ALSO
* sets job->state = JOB_STAT_RUN synchronously, in the same operation --
* mirroring scheduler.c's sched_commit_decision() (bug-1624) exactly --
* so sched_account_jobs()'s own unchanged JOB_IS_RUN gate includes the
* job starting THIS round, not a later one. This function still performs
* no charging itself; it now also performs no NON-running state
* transition (DONE/EXIT/preemption remain exclusively triggered by their
* own separately-recorded events, unchanged). */
static void sim_reconcile_unaccounted_jobs(xce_sched_sim_driver_t* sim,
replay_session_t* decoded_session,
sched_record_session_ctx_t* record_ctx);
- Preconditions: unchanged —
sim/decoded_sessionNULL is a safe no-op;record_ctxmay legitimately be NULL. - Postconditions: unchanged — a matched job has
job->allocs/job->accountsset and its entry removed fromdecoded_session->decisions; an unmatched job is left exactly as it was. - Invariants (new): a job is captured at most once, at the first round (in replay order) whose decoded stream contains its originating DISPATCH decision — independent of that round's job state.
Implementation Notes
- Integration: delete
!JOB_IS_RUN(job->state)from the trigger condition atsim_loop.c:592; no other line in the match/capture/report path changes. - Validation: see Testing Strategy — contention fixture, cross-round separation,
--sim-verify, interactive/batch parity, and the invertedtest_reconcile_skips_non_run_state_jobreplacement are all required, not optional. - Risks: repeated per-round "not found" TRACE logging for a job with no recorded origin anywhere in the replay (Requirement 4.4 case) — accepted, TRACE-level, see
research.md's Risks section. - Verified safe (early capture vs. cleanup): capturing
job->accountswhile a job is still PEND is safe against every existing job-cleanup path.sched_uncharge_account()has zero production callers anywhere in the scheduler — production resource accounting is a reset-and-resum model (xce_core_begin_session()zeroesresInUse/sharedInUseevery round;sched_account_jobs()re-sums it from scratch from currently-JOB_IS_RUNjobs only). Both placesjob->accountsis freed on a non-RUN transition (scheduler.c:974-977,core_events.c:915-918) release memory only, with noresInUsemutation. A job captured while PEND that later transitions directly to a terminal state without ever passing through RUN is therefore never summed bysched_account_jobs()at any round and its accounts are freed with zero accounting impact — early capture cannot under-countresInUse. Seeresearch.md's corresponding Research Log entry for the full trace.
Error Handling
Error Strategy
- No matching recorded decision this round: not an error —
job->accounts/job->allocsstayNULL, a TRACE line is logged, the job is re-checked next round (unchanged behavior, now reached by more jobs per round since the gate widened). - No recorded origin anywhere in the replay (Requirement 4.4): the same "not found" path applies every round for the rest of the replay; the scheduler continues processing without terminating, and that job's resource usage is reported as unavailable (never fabricated) wherever it is inspected (e.g.
info worker). decoded_sessiondecode failure: unchanged — already handled upstream of this function (sim_loop.c:874-879) by skipping verification for that round; this feature does not alter that path.
Testing Strategy
Unit Tests (test_sim_reconcile_unaccounted.c)
- Replace
test_reconcile_skips_non_run_state_jobwithtest_reconcile_captures_non_run_state_job_when_dispatch_recorded— aJOB_STAT_PENDjob with a matching recorded DISPATCH entry now getsjob->allocs/job->accountscaptured and the entry consumed (Requirement 1.1, 3.1). - New:
test_reconcile_still_skips_already_captured_job_regardless_of_state— a job withjob->accountsalready set (any state) is never re-searched, proving the state-widened gate did not weaken the existing no-double-capture guard (Requirement 3.2). - New:
test_reconcile_no_match_leaves_pend_job_uncaptured_and_retries— a PEND job with no matching entry this round stays uncaptured and is eligible again next round (Requirement 4.4, state-independent retry). - New:
test_reconcile_multiple_jobs_captured_independently_across_different_rounds— several PEND jobs simultaneously outstanding, each with its own recorded DISPATCH decision spread across different rounds (some matching this round, some not yet recorded, some already captured from an earlier round); assert each job'sjob->accountsreflects only its own jobid's allocation data, captured at its own correct round, with zero cross-job interference from the wider per-round scan (Requirement 1.1, 3.1 under concurrent load — the single-job tests above cannot distinguish "correct" from "correct by coincidence, only one candidate available"). - Existing
test_reconcile_reconstructs_allocs_and_accounts_from_matching_dispatch,test_reconcile_ignores_non_dispatch_entry_with_matching_jobid,test_reconcile_compacts_via_shift_down_preserving_order,test_reconcile_is_safe_to_recheck_on_a_later_round,test_reconcile_then_release_via_job_state_change_is_asan_clean,test_reconcile_null_args_are_a_safe_noop— re-run unmodified as regression coverage (Requirement 4.1); no expected behavior change for an already-RUN job.
Integration Tests (round-boundary + charging interaction)
- Revised 2026-09-11:
test_captured_pend_job_excluded_from_resinuse_until_run— RENAMED/INVERTED totest_captured_job_included_in_resinuse_same_round_as_capture. Drivesim_reconcile_unaccounted_jobs()for a PEND job with a matching recorded DISPATCH entry, then assertjob->state == JOB_STAT_RUNimmediately (Requirement 1.2) and, after runningsched_account_jobs()in the SAME round, thatw->resInUse/core->sharedInUsealready reflect it (Requirement 1.3) — NOT deferred to a later round. Add a companion assertion that a later, separately-recordedJOB_STATE_CHANGE-to-RUN event for the same job (applied via the normalXCE_CORE_EVENT_JOB_STATE_CHANGEpath) is a safe no-op that does not double-charge or corruptresInUse(Requirement 1.4). - New (resource-contention fixture, per
scheduler-driver-equivalence.md's explicit discipline): a worker with capacity for N jobs, more than N candidate jobs recorded, dispatch decisions and RUN confirmations separated across multiple rounds — assert the live allocator's own capacity check (sched_check_host_satisfy_) sees correctresInUsestarting at the CAPTURE round (revised from "the confirmation round"), not permanently-zero capacity. A no-contention fixture is explicitly insufficient here (scheduler-driver-equivalence.md). - New:
test_sim_verify_no_false_mismatch_across_round_separated_dispatch— a--sim-verifyrun over a replay where dispatch and RUN confirmation are several rounds apart reports zero mismatches (Requirement 4.2). - Existing
test_repl_equivalence.c-style interactive/batch parity coverage re-run unmodified — this feature's call site is shared between both modes, so no new test is needed to prove Requirement 4.3, only confirmation the existing suite still passes.