myxce/.kiro/specs/scheduler-sim-decision-parity/brief.md
Rongsong Shen 0b2c856d8b
docs(scheduler-sim-decision-parity): finalize design and generate tasks
Design went through 3 review rounds (fixed 6 real issues: verify-loop
positional alignment, snapshot-restored-job ambiguity, a leak, a
double-free/UAF from move-semantics aliasing, a const-qualified interface
contradicting its own mutation requirement, and a file-location error in
the accounting-function relocation plan). Tasks generated and passed an
independent sanity review after one repair round (missing call-site update,
missing (P) boundary tag). All phases approved, ready for implementation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-22 05:30:49 +00:00

44 lines
7.9 KiB
Markdown

# Brief: scheduler-sim-decision-parity
## Problem
Operators debugging scheduling behavior via `xce_scheduler -i -s <replay>` (or batch `--sim`) cannot trust two things about jobs that were already resolved (dispatched) in the *original* recorded run: (1) their resource usage is never reflected in `w->resInUse`/`core->sharedInUse` (`info worker` always shows `used=0.00` regardless of actual load — bug-1613), and (2) their `DISPATCH` decision is never visible in decision output, even under `step` (bug-1611). Both undermine the entire premise of simulation mode: that replaying a recording reproduces what actually happened, closely enough to debug and tune against.
## Current State
- `scheduler-core` (implementation-complete, closed) built the core/driver split: `xce_sched_core_t` is mode-agnostic; `driver.c` (normal mode) and `sim_loop.c` (simulation mode) are separate driver implementations of the same session/decision-callback protocol.
- **Confirmed (bug-1613)**: `sched_account_jobs()` — re-derives `resInUse`/`sharedInUse` from currently-RUN-state jobs' already-built `job->accounts`, required after every session's resval reset — is called by `driver.c`'s `xce_sched_driver_run_normal_mode_round()` but never by `sim_loop.c`, which still uses the monolithic `xce_core_go()` (no interleaving point for it). The function is also currently signatured on `xce_scheduler_t*` (normal-mode-only struct), which `xce_sched_sim_driver_t` deliberately does not have (Req 5 AC 17 simulation-isolation boundary) — it cannot be called from `sim_loop.c` today without a signature change.
- **Confirmed (this session, new finding)**: `xce_sched_driver_run_normal_mode_round()`'s decision-drain loop shows every decision the normal-mode scheduler makes — including `DISPATCH`, with its full `xce_job_alloc_t` allocation payload — is written into the `.replay` file via `sched_record_session_add_decision()`. `sim_loop.c` already decodes this full `decisions` list every round (`decoded_session->decisions`, via `decode_ReplaySession_ptr()`), but only ever reads it when `sim->verify` is set, purely to compare against freshly-drained live decisions. It is never applied to reconstruct `job->allocs`/`job->accounts` for a job the live picker in *this* replay run doesn't independently re-decide — which is the normal case for any job already resolved before the replay began. This is very likely the single root cause behind both bug-1611 and bug-1613: the data needed already exists in every replay stream and is already decoded every round; it is simply never applied outside verify-mode comparison.
- `.kiro/steering/scheduler-driver-equivalence.md` (created this session) documents the constraint this spec restores: replaying a normal-mode recording must reproduce identical decisions AND identical resource-accounting state, not just matching final decisions on fixtures with no contention.
## Desired Outcome
Replaying the exact recording of a normal-mode run reproduces, at every point in the replay: the same resource-accounting state (`resInUse`/`sharedInUse`/`job->accounts`) and the same decision visibility (a `DISPATCH`/`PEND`/`PREEMPT` line is observable for every job, whether the live picker re-derived it fresh this run or it was already resolved in the original recording) that the normal-mode run itself produced.
## Approach
Two components, one spec:
1. **Accounting re-derivation parity**: relocate `sched_account_jobs()` and its callees (`sched_charge_account`/`sched_uncharge_account`/`sched_update_account`/`sched_update_res_accounts`) from `xce_scheduler_t*` to `xce_sched_core_t*` — every one of their bodies already only touches `sched->core`, so this is a signature change, not a logic change, following the exact precedent `sched_resolve_job_picker()` already established (task 4.3) for the identical reason. Then call it in `sim_loop.c` at the same relative position `driver.c` does: replace the monolithic `xce_core_go()` with the split `xce_core_begin_session()` → `sched_account_jobs(&(sim->core))` → `xce_core_signal_go()`.
2. **Recorded-decision application**: for a round where the live picker/allocator does not independently produce a decision for a job that the recording shows as already resolved, apply the corresponding recorded decision from `decoded_session->decisions` directly — reconstructing `job->allocs`/`job->accounts` (mirroring the field data `sim_verify_job_alloc_eq()` already knows how to compare) and feeding it through the same decision pipeline `sim_print_decision()`/`xce_repl_on_decision()` consume, so it is both charged and visible.
## Scope
- **In**: Accounting-state re-derivation parity between the two drivers; applying recorded decisions for already-resolved jobs during replay (both interactive and batch `--sim`); avoiding double-application when a job is both recorded-resolved and somehow also independently re-decided; resource-contention test fixtures proving the fix (per `scheduler-driver-equivalence.md`'s testing discipline) since no-contention fixtures cannot distinguish correct from broken accounting.
- **Out**: Any change to REPL console output/formatting (that's `scheduler-sim-repl-verbosity`, a sibling spec, explicitly excluded from touching simulation state by its own Non-Goals). Any change to `--sim-verify`'s actual comparison semantics beyond ensuring recorded-decision application doesn't produce a spurious mismatch or double-count against it. Reopening or modifying `scheduler-core` itself. New scheduling/allocation policy.
## Boundary Candidates
- Accounting relocation (`resource.c`/`scheduler.h`/`core.h`, signature-only change) — small, mechanical, low-risk, could land independently of the second piece.
- Recorded-decision application (`sim_loop.c`, and `sim_verify.c` for the double-count/verify-interaction question) — larger, the real design surface: what "already resolved, not independently re-decided" means precisely, and how it composes with `--sim-verify`'s existing comparison.
## Out of Boundary
- `scheduler-sim-repl-verbosity`'s REPL output/formatting work.
- `scheduler-sim-diagnostics`'s future new inspection commands (limit usage, pend-reason) — this spec restores existing data's correctness, it does not add new inspectable data.
- Whether `sched_check_host_satisfy`/`sched_meet_host_selection` read `resInUse` for live capacity decisions — CONFIRMED during design (see `research.md`'s Research Log): they do, unconditionally. This is existing `scheduler-core` behavior this spec does not change, only relies on.
## Upstream / Downstream
- **Upstream**: `scheduler-core`'s core/driver split and its now-documented equivalence invariant (`.kiro/steering/scheduler-driver-equivalence.md`).
- **Downstream**: `scheduler-sim-repl-verbosity`'s round-rollup dispatch counts (currently designed to read from the existing decision tally) become accurate once this lands, rather than permanently under-reporting dispatches for already-resolved jobs. `scheduler-allocation`/`scheduler-preemption` (both requirements-generated, not yet designed) inherit a scheduler core with a restored, tested equivalence guarantee to build on.
## Existing Spec Touchpoints
- **Extends**: none formally — `scheduler-core` is closed; this spec restores its own documented invariant rather than reopening it.
- **Adjacent**: `scheduler-sim-repl-verbosity` (sibling, not yet implemented — benefits from this fix but does not depend on it to ship), `scheduler-sim-diagnostics` (sibling, brief only).
## Constraints
- Must not regress REPL-vs-batch-sim decision equivalence (`test_repl_equivalence.c`) — that invariant is solid today and must stay solid.
- Must be validated against a resource-contention fixture (multiple jobs competing for insufficient capacity), per `scheduler-driver-equivalence.md`'s testing discipline — a no-contention fixture cannot prove this fix works.
- `sched_account_jobs()`'s relocation must be a pure signature change (no behavior change) to keep the accounting-parity piece low-risk and independently reviewable from the larger recorded-decision-application piece.