Elevates the normal-mode/simulation-mode driver equivalence guarantee from a one-line architectural description in CLAUDE.md to an enforced steering constraint (scheduler-driver-equivalence.md), backed by bug-1611/bug-1613 as concrete evidence of violation. Scopes the fix as its own spec (scheduler-sim-decision-parity, requirements + design) rather than reopening the closed scheduler-core spec or folding it into scheduler-sim-repl-verbosity's explicitly narrower boundary. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
6.5 KiB
Requirements Document
Introduction
The scheduler's .replay-driven simulation mode (--sim/--interactive) exists so operators can debug and tune scheduling behavior against recorded production data, trusting that a replay reproduces what actually happened. That trust currently does not hold for any job already resolved (dispatched, pended, or preempted) before a given point in the replay: its resource usage is never reflected in worker/shared resource-in-use totals, and its decision is never visible in decision output — because the recorded decision stream, though fully decoded every round, is only ever used for optional --sim-verify comparison, never applied to reconstruct state or report the decision otherwise. This feature restores decision and resource-accounting state parity between the normal-mode and simulation-mode drivers, per the constraint documented in .kiro/steering/scheduler-driver-equivalence.md.
Boundary Context
- In scope: resource-in-use accounting correctness during simulation-mode execution (interactive and batch); capacity-awareness correctness of simulation-mode scheduling decisions under resource contention; decision visibility (through existing decision-reporting channels) for jobs already resolved in the original recording; graceful behavior when recorded decision data for a job is unavailable.
- Out of scope: any change to REPL console output formatting (owned by the sibling
scheduler-sim-repl-verbosityspec); any new inspectable data or REPL command (owned by the siblingscheduler-sim-diagnosticsbrief); any change to--sim-verify's comparison logic itself, beyond not producing a false mismatch as a side effect of this feature; any change to scheduling/allocation policy; any change to normal-mode (non-simulation) behavior. - Adjacent expectations: this feature restores an invariant
scheduler-core(implementation-complete) was already expected to uphold; it does not reopen or modify that spec.scheduler-sim-repl-verbosity's round-summary decision counts depend on decisions being reported through the same channel this feature restores, but that spec does not depend on this one to ship.
Requirements
Requirement 1: Accurate resource-usage reporting in simulation mode
Objective: As an operator inspecting simulation-mode resource state, I want per-worker and shared resource usage to reflect every currently-running job, so that what I inspect during a replay is trustworthy.
Acceptance Criteria
- While a job is in a running state during simulation-mode execution, the scheduler shall include that job's resource consumption in its worker's resource-in-use total.
- While a job is in a running state during simulation-mode execution, the scheduler shall include that job's resource consumption in any shared resource's resource-in-use total it consumes from.
- When a scheduling round completes, the scheduler shall refresh resource-in-use totals to reflect every currently-running job, not only jobs whose resource consumption was already reflected in an earlier round.
- When replaying a recorded run in simulation mode, the scheduler shall report the same resource-in-use totals as normal-mode execution reported at the equivalent point in the original run.
Requirement 2: Correct capacity awareness under resource contention in simulation mode
Objective: As an operator relying on simulation mode to reflect realistic scheduling outcomes, I want the scheduler's capacity checks during simulation-mode execution to account for resources already consumed by running jobs, so that simulated scheduling behavior matches live behavior under contention.
Acceptance Criteria
- While resource capacity is already consumed by running jobs, when the scheduler evaluates whether a job fits on a worker during simulation-mode execution, the scheduler shall treat that already-consumed capacity as unavailable, the same way normal-mode execution does.
- The scheduler shall not make a simulation-mode scheduling decision that normal-mode execution of the same recorded state would not also make.
Requirement 3: Decision visibility for jobs resolved before replay began
Objective: As an operator watching simulation-mode decision output, I want to see a decision for every job the replay reaches, including jobs already resolved in the original run, so I have a complete picture of scheduling activity.
Acceptance Criteria
- When a job was dispatched, pended, or preempted in the original recorded run and simulation-mode replay reaches the point that event occurred, the scheduler shall make that decision observable through the same decision-reporting channel used for a freshly-computed decision.
- The scheduler shall preserve the original decision's type and resource-allocation detail when reporting it during replay.
- When simulation-mode independently computes a fresh decision for a job in a given round, the scheduler shall report only that one decision for the job in that round, not also a duplicate from the recorded stream.
Requirement 4: Graceful degradation when recorded decision data is unavailable
Objective: As an operator running simulation mode against an incomplete or older recording, I want the scheduler to degrade gracefully rather than fail, so a data gap in one job's history doesn't block inspection of the rest of the replay.
Acceptance Criteria
- If a job's originating decision cannot be found in the recorded data when simulation-mode needs it to report or account for that job, the scheduler shall report that job's resource usage as unavailable rather than fabricating a value.
- If a job's originating decision cannot be found in the recorded data, the scheduler shall continue processing the remainder of the replay without terminating.
Requirement 5: No regression to existing driver-equivalence and verification guarantees
Objective: As a maintainer relying on existing scheduler guarantees, I want this fix to preserve them, so existing verification workflows and tests continue to hold.
Acceptance Criteria
- The scheduler shall continue to produce identical decisions between interactive (step-by-step) and batch simulation execution modes.
- The scheduler shall continue to support decision verification mode (comparing freshly-computed decisions against recorded ones) without reporting a mismatch caused by this feature's own behavior.
- While simulation mode is not enabled, the scheduler shall not change its existing resource-accounting or decision-reporting behavior.