myxce/.kiro/specs/scheduler-sim-decision-parity/design.md

Design Document

Overview

Purpose: Restore the scheduler's documented normal-mode/simulation-mode driver equivalence invariant (.kiro/steering/scheduler-driver-equivalence.md) for jobs already resolved before a given point in a replay — both their resource-accounting state and their decision visibility.

Users: XCE scheduler engineers and operators debugging via xce_scheduler -i -s <replay> or batch --sim, and any future scheduler spec (scheduler-allocation, scheduler-preemption) relying on simulation mode reflecting real scheduling behavior.

Impact: Changes sim_loop.c's round-boundary handling (replacing the monolithic xce_core_go() with the split calls driver.c already uses, plus a new reconciliation step that also compacts decoded_session->decisions as it consumes entries — see Recorded-Decision Reconciliation's Implementation Notes), and the signature of five accounting functions currently in resource.c (relocated to operate on xce_sched_core_t*). sim_verify.c's comparison predicates (sim_decision_eq() and its helpers) need no change — confirmed during design validation that sim_loop.c's own array bookkeeping is sufficient to keep the existing positional comparison correct. No change to core.c/core_events.c's event-application logic, to scheduling/allocation policy, or to REPL output formatting.

Goals

• Resource-in-use accounting (resInUse/sharedInUse) reflects every currently-running job in simulation mode, matching normal mode (Requirement 1).
• Simulation-mode capacity checks under contention produce the same outcome normal mode would (Requirement 2) — a direct consequence of Goal 1, since the capacity-check code path is unchanged and already reads this data correctly.
• Every job's decision (including ones resolved before this replay run started tracking them live) is visible through the existing decision-reporting channel (Requirement 3).
• Missing recorded data degrades gracefully, never fabricates or crashes (Requirement 4).
• Zero regression to REPL-vs-batch-sim equivalence, --sim-verify, or normal-mode behavior (Requirement 5).

Non-Goals

• Any change to REPL console output formatting (scheduler-sim-repl-verbosity's boundary).
• Any new inspectable data or command (scheduler-sim-diagnostics's boundary).
• Any change to scheduling/allocation policy or picker plugin behavior.
• Reopening scheduler-core.

Boundary Commitments

This Spec Owns

• The signatures of sched_account_jobs()/sched_charge_account()/sched_uncharge_account()/sched_update_account()/sched_update_res_accounts()/sched_commit_job_alloc() (currently resource.c, xce_scheduler_t*-coupled) and their relocation to xce_sched_core_t*.
• sim_loop.c's round-boundary sequence (the xce_core_go() replacement and the new reconciliation step).
• The logic that reconstructs job->allocs/job->accounts from a recorded decision for a RUN-state job that has neither, and feeds that reconstructed decision through the existing decision-reporting channel.

Out of Boundary

• driver.c's normal-mode round logic, beyond updating its one call site to the relocated accounting function signature.
• core.c/core_events.c's event-application logic (JOB_PUSH/JOB_UPDATE/JOB_STATE_CHANGE handling) — unchanged.
• repl_break.c/repl.c/repl_info.c — unchanged; they consume whatever the decision-reporting channel produces, without needing to know whether a given decision was freshly computed or reconstructed.
• --sim-verify's actual pass/fail semantics — this spec must not introduce a false mismatch, but does not redesign verification.

Allowed Dependencies

• sim_verify.c's existing sim_verify_job_alloc_eq()/sim_verify_host_alloc_eq()/sim_verify_alloc_eq() field shapes as the reference for what a reconstructed xce_job_alloc_t must contain — not necessarily calling these functions, but matching the payload shape they already validate.
• decoded_session->decisions, already decoded every round by existing sim_loop.c code — no new decode path.
• The sched_resolve_job_picker() relocation (task 4.3) as the precedent for the accounting-function signature change.

Revalidation Triggers

• Any change to xce_job_alloc_t/xce_host_alloc_t/xce_alloc_t's wire shape (controller.capnp) must be re-checked against the reconstruction logic here.
• Any change to how --sim-verify compares decisions must be re-checked against this spec's reconciliation step, and vice versa.
• If scheduler-sim-diagnostics later adds a "why did this job dispatch" trace, it can build on the now-correct job->accounts this spec establishes rather than re-deriving it.
• When scheduler-preemption lands (currently requirements-generated, not yet designed — driver.h:96 presently states "no live allocator path produces this decision type"): Requirement 3.1 here covers "dispatched, pended, or preempted" jobs, but Recorded-Decision Reconciliation's trigger condition (RUN-state + job->accounts == NULL) only ever matches a recorded DISPATCH entry — a preempted job leaves RUN-state at the moment of preemption, so it cannot trigger this same mechanism, and PEND doesn't need reconciliation because the live picker already re-derives it fresh every round. This is not a gap today (no code path emits XCE_DECISION_PREEMPT yet, so no recorded replay can contain one), but once scheduler-preemption starts producing real PREEMPT decisions, revisit whether this spec's mechanism needs a second trigger condition for "job left RUN-state via preemption within this round, no reconciled visibility yet" or whether scheduler-preemption's own design should own that instead.

Architecture

Existing Architecture Analysis

• driver.c's xce_sched_driver_run_normal_mode_round() is the reference implementation for the exact sequence this spec ports to sim_loop.c: xce_core_begin_session() → driver-specific interleaved bookkeeping (plugin_begin_session(), sched_account_jobs()) → xce_core_signal_go() → decision-drain loop (cbuf_get(&core->decisions), recording each via sched_record_session_add_decision() when recording, then core->on_decision()).
• sim_loop.c currently calls the older monolithic xce_core_go() (itself now just xce_core_begin_session()+xce_core_signal_go() back-to-back, per core_session.c) with no interleaving room, and decodes decoded_session->decisions every round but only consumes it under sim->verify.
• The accounting functions being relocated — sched_account_jobs() (scheduler.c) and its callees sched_charge_account()/sched_uncharge_account()/sched_update_account()/sched_update_res_accounts()/sched_commit_job_alloc() (resource.c) — already only touch sched->core in their bodies despite their xce_scheduler_t* signature. Confirmed by reading every function in this call chain, and independently re-confirmed with grep "sched->" resource.c | grep -v "sched->core" returning zero matches across that whole file (covers the five resource.c functions; sched_account_jobs() itself was separately read in full).

Architecture Pattern & Boundary Map

flowchart TD
    BeginSession[xce_core_begin_session] --> Reconcile[Reconcile RUN jobs with no accounts]
    Reconcile --> Lookup{Recorded decision found in decoded_session}
    Lookup -->|Yes| Rebuild[Rebuild job allocs and accounts from recorded decision]
    Lookup -->|No| Unavailable[Report usage unavailable, continue]
    Rebuild --> Compact[Free consumed entry, compact decoded_session.decisions]
    Compact --> Report[Feed decision through existing reporting channel]
    Report --> AccountJobs[sched_account_jobs on core]
    Unavailable --> AccountJobs
    AccountJobs --> SignalGo[xce_core_signal_go]
    SignalGo --> Picker[Live picker and allocator run for still-pending jobs]
    Picker --> DrainLoop[Decision drain loop -- same reporting channel for fresh decisions]

Architecture Integration:

• Selected pattern: extend the existing round-boundary interleaving slot (the same one driver.c already uses for its own bookkeeping) with one additional step, rather than a new hook or a second round-processing pass.
• Domain/feature boundaries: reconciliation logic lives in sim_loop.c only; the relocated accounting functions live in resource.c/core.h and are shared by both drivers; driver.c gets a one-line call-site update.
• Existing patterns preserved: the job->accounts == NULL check sched_account_jobs() already performs is reused as the reconciliation trigger and the double-application guard — no new state.
• New components rationale: reconciliation must run before sched_account_jobs() (so the same round's re-charge picks up newly-reconstructed jobs) and before xce_core_signal_go() (so the live picker/allocator's capacity checks for still-pending jobs see correct usage this round, satisfying Requirement 2).
• Steering compliance: sim_loop.c remains at its documented interleaving-slot discipline (no new call sites added to the core's own API surface); resource.c's functions gain no new behavior, only a new valid signature.

Technology Stack

Layer	Choice / Version	Role in Feature	Notes
Scheduler daemon (C, gnu17)	existing src/scheduler/ toolchain	All logic in existing driver/core/verify translation units	No new dependency; matches tech.md

File Structure Plan

Modified Files

• src/scheduler/src/resource.c — relocate sched_charge_account(), sched_uncharge_account(), sched_update_account(), sched_update_res_accounts(), sched_commit_job_alloc() from xce_scheduler_t* to xce_sched_core_t* parameters (body unchanged — every reference is already sched->core.*).
• src/scheduler/include/scheduler.h — remove/update the now-relocated declarations.
• src/scheduler/include/core.h — add the relocated declarations, mirroring sched_resolve_job_picker()'s existing placement and doc-comment precedent.
• src/scheduler/src/scheduler.c — relocate sched_account_jobs() itself the same way (it is defined here, not in resource.c; body likewise unchanged — every reference is already sched->core.*); update this file's own two call sites for the new signatures: sched_commit_decision()'s call to sched_commit_job_alloc() (pass &(sched->core)) and sched_begin_session()'s call to sched_account_jobs() likewise.
• src/scheduler/src/driver.c — update xce_sched_driver_run_normal_mode_round()'s sched_account_jobs(sched) call to sched_account_jobs(&(sched->core)).
• src/scheduler/src/sim_loop.c — replace the xce_core_go(&(sim->core)) call with xce_core_begin_session() → reconciliation step (new static function(s)) → sched_account_jobs(&(sim->core)) → xce_core_signal_go(); reconciled decisions feed through the same decision-reporting call the existing drain loop uses, and each consumed recorded decision is removed/compacted from decoded_session->decisions in the same pass so the existing --sim-verify positional walk (di against cbuf_get(&(sim->core.decisions), ...), unchanged) stays correctly aligned.
• src/scheduler/src/sim_verify.c — no change (confirmed during design validation; see sim_loop.c entry above and the Recorded-Decision Reconciliation component's Implementation Notes).

New/Modified Test Files

• src/scheduler/testsuite/test_driver_session_ordering.c (or a new test_sim_session_ordering.c mirroring it) — proves sim_loop.c now upholds the same "picker never observes zeroed-but-not-yet-resummed accounting" property driver.c's own test already proves.
• A new resource-contention fixture (multiple jobs competing for insufficient capacity) added to the scheduler's simulation testsuite, per scheduler-driver-equivalence.md's testing discipline — required because existing fixtures evidently don't exercise this path.
• src/scheduler/testsuite/test_sim_equivalence.c (existing, decision-identity coverage) — extended with a case where a job is already resolved before the replay's live picker runs, verifying decision AND accounting-state identity, not just decision identity.

System Flows

See the Architecture Pattern & Boundary Map diagram above — it is the single non-trivial flow this feature introduces (the round-boundary reconciliation decision tree). No sequence diagram is needed beyond it; the decision-drain loop's own flow is unchanged.

Requirements Traceability

Requirement	Summary	Components	Interfaces	Flows
1.1, 1.2, 1.3, 1.4	Accurate resource-usage reporting	Core-Level Resource Accounting, Sim Driver Round-Boundary Accounting	relocated sched_account_jobs	Round-boundary flow
2.1, 2.2	Correct capacity awareness under contention	Core-Level Resource Accounting (consequence of 1.x being correct)	sched_check_host_satisfy_/sched_check_shared_satisfy_ (unchanged, already reads this data)	Round-boundary flow
3.1, 3.2, 3.3	Decision visibility for pre-resolved jobs	Recorded-Decision Reconciliation	decision-reporting channel	Round-boundary flow
4.1, 4.2	Graceful degradation	Recorded-Decision Reconciliation	n/a (error path)	Round-boundary flow
5.1, 5.2, 5.3	No regression	all components	n/a	n/a

Components and Interfaces

Component	Domain/Layer	Intent	Req Coverage	Key Dependencies (P0/P1)	Contracts
Core-Level Resource Accounting	Scheduler core / resource.c, core.h	Re-derive resource-in-use state from currently-running jobs' known allocations, callable by either driver	1.1-1.4, 2.1-2.2	xce_sched_core_t (P0)	Service
Sim Driver Round-Boundary Accounting	Sim driver / sim_loop.c	Call the core-level accounting function at the correct point in the round boundary, mirroring the normal-mode driver	1.3, 1.4, 5.3	Core-Level Resource Accounting (P0)	State
Recorded-Decision Reconciliation	Sim driver / sim_loop.c	Reconstruct allocation/accounting and decision visibility for a RUN-state job with no accounts, from this round's recorded decisions	3.1-3.3, 4.1-4.2	decoded_session->decisions (P0), decision-reporting channel (P0)	Service, State

Scheduler Core (resource.c / core.h)

Core-Level Resource Accounting

Field	Detail
Intent	Re-derive resInUse/sharedInUse from every currently-running job's known resource allocation, usable identically by both drivers
Requirements	1.1, 1.2, 1.3, 1.4, 2.1, 2.2

Responsibilities & Constraints

• Body unchanged from today's sched_account_jobs()/callees — this is a signature relocation (xce_scheduler_t* → xce_sched_core_t*), not a logic change, since every reference in the existing bodies is already sched->core.*.
• Both drivers call it at the identical relative position: after xce_core_begin_session()'s resval reset, before xce_core_signal_go() unblocks the picker/allocator.
• Requirement 2 (capacity awareness) is satisfied as a direct consequence — sched_check_host_satisfy_()/sched_check_shared_satisfy_() already read resInUse/sharedInUse unconditionally; once those fields are correctly populated in simulation mode, the existing (unmodified) capacity-check code path produces the same outcome normal mode would.

Contracts: Service [x] / State [ ]

Service Interface

/*
 * Re-charges every currently-running job's known resource allocation
 * into core->sharedInUse and each worker's resInUse. Skips a job whose
 * allocation is not yet known (job->accounts == NULL) -- reconciling
 * that is the Recorded-Decision Reconciliation component's job, which
 * must run first in the same round-boundary window for this call to
 * see a fully up-to-date job set.
 */
int sched_account_jobs(xce_sched_core_t* core);

• Preconditions: core non-NULL; called strictly after xce_core_begin_session()'s resval reset for the current round.
• Postconditions: every RUN-state job with known accounts is reflected in resInUse/sharedInUse; a RUN-state job with unknown accounts is silently skipped (not an error — see Recorded-Decision Reconciliation for why that's expected before this call runs, and Requirement 4 for the case where it stays unknown).
• Invariants: identical result given identical core state, regardless of which driver calls it.

Sim Driver (sim_loop.c)

Sim Driver Round-Boundary Accounting

Field	Detail
Intent	Call Core-Level Resource Accounting at the same relative round-boundary position driver.c already uses
Requirements	1.3, 1.4, 5.3

Responsibilities & Constraints

• Replaces the current single xce_core_go(&(sim->core)) call with the split sequence, matching xce_sched_driver_run_normal_mode_round()'s documented ordering exactly.
• Must not change xce_core_go()'s existing error-handling contract (a failed xce_core_begin_session() still skips the round's decision drain, per today's sim_loop.c error path) — the split must preserve this, not simplify it away.
• Headless (non-interactive) and interactive (-i) simulation runs both go through this same call site — no divergence between them (Requirement 5.1 depends on this).

Contracts: State [x]

Recorded-Decision Reconciliation

Field	Detail
Intent	For a RUN-state job with no known allocation, reconstruct it from this round's recorded decision and report it through the existing decision channel
Requirements	3.1, 3.2, 3.3, 4.1, 4.2

Responsibilities & Constraints

• Runs once per round, after xce_core_begin_session(), before sched_account_jobs().
• Trigger condition: a job in sched->core.jobs (via the existing core.h iteration idiom) that is RUN-state and has job->accounts == NULL.
• On trigger, search this round's decoded_session->decisions for a matching jobid. If found, reconstruct job->allocs/job->accounts from the recorded payload (same field shape sim_verify_job_alloc_eq() already validates) and feed the reconstructed decision through the same reporting call the existing decision-drain loop uses for a freshly-drained decision (Requirement 3.1).
• If not found (Requirement 4): report the job's usage as unavailable — do not fabricate a value, do not treat this as fatal, continue processing the rest of the replay. This is the expected case for a job already running at the moment the replay's starting snapshot was taken (see Risks).
• Because job->accounts is set as a side effect, a job reconciled in round N is never reconsidered in round N+1 — the same NULL-check that identifies the trigger also prevents double-application (Requirement 3.3), including against a job that separately receives a fresh live decision in the same round (that path sets job->accounts too, via the unchanged sched_commit_decision(), so this component simply finds nothing to do for it).

Contracts: Service [x] / State [x]

Service Interface

/*
 * For every RUN-state job in `sim->core.jobs` with job->accounts ==
 * NULL, attempts to reconstruct its allocation from a matching
 * decision in this round's decoded_session->decisions and reports it
 * through the same channel a freshly-drained decision uses. A job with
 * no matching recorded decision is left with job->accounts == NULL
 * (Requirement 4 -- reported as unavailable elsewhere, not an error
 * here) and is safe to call again next round.
 *
 * MUTATES *decoded_session: each consumed entry is freed and removed
 * (shifted down, not swap-removed -- see Implementation Notes) from
 * decisions[]/n_decisions, so decoded_session cannot be `const` here --
 * this is deliberate, not an oversight: the caller's own subsequent
 * --sim-verify positional walk over the same decoded_session depends on
 * consumed entries already being gone by the time it runs.
 *
 * record_ctx is nullable and mirrors the existing decision-drain loop's
 * own `if (record_ctx != NULL)` gate (this reporting path must recording-
 * capture identically to a freshly-drained decision, and recording is
 * optional per run) -- CONFIRMED during implementation this third
 * parameter is necessary, not avoidable: record_ctx exists only as a
 * stack-local inside sched_sim_driver_run()'s per-ReplaySession block
 * (no field on xce_sched_sim_driver_t can supply it), and the round-
 * boundary call site this function is wired into (task 3) sits in that
 * exact same lexical scope, so passing it through is the natural,
 * unambiguous plumbing rather than a workaround.
 */
void sim_reconcile_unaccounted_jobs(xce_sched_sim_driver_t* sim,
                                    replay_session_t* decoded_session,
                                    sched_record_session_ctx_t* record_ctx);

• Preconditions: sim/decoded_session non-NULL (NULL is a safe no-op, matching this codebase's established "never load-bearing" REPL-hook convention). record_ctx may be NULL (recording not active for this run).
• Postconditions: every RUN-state job either has job->accounts set (reconstructed) or is left NULL (genuinely unavailable this round). decoded_session->decisions/n_decisions reflect the removal of every consumed entry (freed, not merely uncounted -- see memory-ownership note below), leaving the remaining entries in their original relative order.
• Invariants: never double-applies (idempotent per job, per the job->accounts gate); never fabricates allocation data not present in the recording; never leaves a freed pointer reachable through job->allocs or through the remaining decoded_session->decisions entries.

Implementation Notes

• Integration with --sim-verify (CONFIRMED during design validation, resolving the prior "read the comparison driver at implementation time" placeholder): sim_loop.c's verify branch (current code: the while ((cbuf_get(&(sim->core.decisions), &d) == 0) ...) loop) walks decoded_session->decisions positionally via an index di, comparing each live-drained decision d against decoded_session->decisions[di] and advancing di once per live decision. A recorded decision belonging to a job this round's reconciliation step consumes (i.e., a job whose allocation/decision visibility is being restored from the recording rather than freshly re-derived by the live picker) will never produce a corresponding live-drained d — the picker never re-decides an already-resolved job. Comparing positionally without accounting for this would misalign every subsequent entry and produce false mismatches (Requirement 5.2 violation), independent of any bug in the comparison logic itself.
  • Resolution: the reconciliation step removes each recorded decision it consumes from decoded_session->decisions before the verify drain loop runs for that round — it already has the exact jobid-matched entry in hand from its own lookup, so this is a natural side effect of the same pass, not a second scan. This keeps the remaining (unconsumed) recorded entries positionally aligned with the live-drained stream exactly as they are today, when no reconciliation was needed for a job. No change to sim_decision_eq()/sim_verify_job_alloc_eq()/sim_verify_job_preason_eq() (the comparison predicates in sim_verify.c) is required — only sim_loop.c's bookkeeping of which array entries reach that comparison.
  • Memory ownership (must not be overlooked): decoded_session->decisions/n_decisions (replay_session_t, typemap.h) are a plain heap-owned array — free_ReplaySession_ptr() frees it via free_ReplayDecision_list(d->n_decisions, d->decisions), which walks and frees exactly the first n_decisions entries, then frees the array itself. Consequently, simply decrementing n_decisions to "drop" a consumed entry (without separately freeing that entry) is not safe compaction — it silently leaks that entry's payload every round a reconciliation happens, since the round-end free walk will never reach it again. The reconciliation step must instead: (1) extract the data it needs (see ownership sub-point immediately below — this is NOT a plain read for a DISPATCH entry), (2) call the existing free_ReplayDecision(entry) + free(entry) (the same two calls free_ReplayDecision_list() already makes per-element) on the one consumed entry, then (3) compact the array by shifting the remaining entries down over the removed slot (memmove), and decrement n_decisions by one. Shift-down, not swap-with-last: swap-with-last is the usual O(1) removal trick, but it would relocate the last entry into the removed slot and change its position relative to the other remaining entries — corrupting the very relative ordering the positional di comparison in the point above depends on. decoded_session is decoded fresh once per round regardless, so this compaction is confined to the current round with no cross-round effect.
  • Ownership hazard in step (1) above — a DISPATCH entry's alloc must be moved, not aliased, before step (2) frees the entry: the established live-path precedent (scheduler.c:1440, sched_commit_decision()) is job->allocs = decision->u.alloc; — a pointer takeover, not a deep copy; job->accounts (make_job_account(job->allocs)) is then built directly on top of that same, now job-owned, memory. If reconciliation reconstructs job->allocs the same way from a recorded DISPATCH entry — job->allocs = entry->body.dispatch->alloc; (sched_dispatch_decision_t.alloc, an xce_job_alloc_t*, typemap.h) — that pointer is now aliased by BOTH job->allocs and the entry about to be freed in step (2): free_ReplayDecision() → free_DispatchDecision() → free_JobAlloc_ptr(&d->alloc) would free that exact xce_job_alloc_t, leaving job->allocs dangling immediately (and a double-free whenever the job's own allocation is later released through its normal lifecycle). Required fix: immediately after the takeover assignment, null out the source field — entry->body.dispatch->alloc = NULL; — before calling free_ReplayDecision(entry); every _ptr-style free helper in this codebase (including free_JobAlloc_ptr) already no-ops on NULL, so this makes the subsequent free of the rest of the entry safe without a special case. This is DISPATCH-specific and, by construction, the only case this component needs to handle: the reconciliation trigger is a RUN-state job, and a job can only reach RUN-state via a recorded DISPATCH decision — a recorded PEND (or PREEMPT) entry for the same job in the same round is structurally impossible to match here, since PEND/PREEMPT are exactly the decisions that do not put a job into RUN-state. (Independently confirmed no job->preason/equivalent field exists to alias in the first place — job->accounts/job->allocs are DISPATCH-only concepts, consistent with this.) No PEND/PREEMPT ownership case needs to be designed or implemented.
  • This also fixes a latent problem that exists independently of this feature: --sim-verify against any real replay containing already-resolved jobs likely already misaligns positionally today (the recorded stream has entries the live stream never independently reproduces), so this correction is a net improvement to verify-mode accuracy, not merely a regression guard.
• Validation: must be exercised by a resource-contention fixture, not only a fixture where every job fits, per scheduler-driver-equivalence.md.
• Snapshot-restored jobs (CONFIRMED during design validation): xce_sched_core_load_snapshot() (snapshot_load.c) restores job->state from snapshot data but never touches job->accounts/job->allocs. A job already RUN-state at snapshot-load time therefore always has job->accounts == NULL with no in-window recorded decision to reconcile from — this is unambiguously Requirement 4's graceful-degradation case, not a separate path requiring its own handling.

Data Models

Domain Model

No new persistent entity. job->accounts/job->allocs (existing fields on xce_sched_job_t, xce/types.h) become the single source of truth for "has this job's allocation been established, by either the live path or reconciliation" — reused, not extended.

Logical Data Model

No new fields. The reconciliation step reads decoded_session->decisions (already-decoded, existing structure) and writes job->allocs/job->accounts (existing fields) exactly as the live sched_commit_decision() path already does for a fresh dispatch — same target shape, different source.

No Physical Data Model, API Data Transfer, or Event Schema sections apply — no persistent storage, no new wire format, no new wire-level contract. This stays inside the existing .replay stream shape.

Error Handling

Error Strategy

Consistent with this codebase's established "never load-bearing, degrade to unattended" discipline (repl_state_ensure()'s own precedent): a reconciliation failure for one job never blocks the rest of the replay.

Error Categories and Responses

• No matching recorded decision for a RUN-state, unaccounted job (Requirement 4.1, 4.2): report usage as unavailable, continue. Not an error condition — expected for jobs already running at snapshot time.
• Allocation failure while reconstructing job->accounts from a recorded decision: fall back to leaving the job unaccounted (same as "not found") rather than a partial/corrupt reconstruction — preserves Requirement 4's "never fabricate" guarantee.
• xce_core_begin_session() failure: unchanged from today's sim_loop.c behavior — the round's decision drain is skipped, matching the existing error path this feature's split-call migration must preserve exactly.

Monitoring

No new logging framework — existing WARN/TRACE conventions apply if reconciliation needs to report a skip; no daemon-level behavior change.

Testing Strategy

Unit Tests

• Core-Level Resource Accounting produces identical resInUse/sharedInUse output whether called with a core obtained from xce_scheduler_t or from xce_sched_sim_driver_t (Requirement 1.4).
• Recorded-Decision Reconciliation reconstructs job->allocs/job->accounts correctly from a synthetic recorded DISPATCH decision (Requirement 3.1, 3.2).
• Recorded-Decision Reconciliation is a no-op (does not double-apply) for a job whose job->accounts is already set (Requirement 3.3).
• Recorded-Decision Reconciliation leaves a job unaccounted, without fabricating data or erroring, when no matching recorded decision exists (Requirement 4.1, 4.2).
• Reconciling a DISPATCH entry, then later releasing the job's allocation through its normal end-of-life path, must be ASAN-clean (no double-free, no leak) — this directly exercises the move-then-null-then-free ownership handling in the Implementation Notes above, run under the project's standard ASAN build per tech.md/build-deployment.md rather than relying on a plain functional assertion to catch it.

Integration Tests

• Resource-contention fixture (new, required per scheduler-driver-equivalence.md): multiple jobs competing for insufficient capacity, replayed in simulation mode; assert resInUse matches what the equivalent normal-mode recording produced at every round, and that no job is dispatched in simulation mode that normal mode would have pended (Requirement 2.1, 2.2).
• test_repl_equivalence.c (existing) re-run unmodified to confirm no regression to REPL-vs-batch-sim equivalence (Requirement 5.1).
• --sim-verify run against a recording containing already-resolved jobs, confirming no false mismatch is reported as a result of this feature (Requirement 5.2).
• A replay starting from a snapshot with jobs already running at snapshot time, confirming graceful degradation (Requirement 4) rather than a crash or fabricated value.

E2E

Live-verify against the real 108-round replay capture that originally motivated bug-1611/bug-1613: after the fix, info worker shows nonzero used= for jobs confirmed running via JOB_RUSAGE ticks, and their DISPATCH decisions become visible in step/continue output.

Optional Sections

Security Considerations, Performance & Scalability, and Migration Strategy are omitted: this is an internal scheduler-daemon correctness fix with no new attack surface, no new persistent data, and no schema/data migration. Performance is bounded by existing per-round job iteration this codebase already tolerates (sched_account_jobs()'s own existing full-job-table walk).

Supporting References

None beyond research.md.