myxce/.kiro/specs/scheduler-sim-eager-reconciliation/tasks.md

Implementation Plan

• 1. Core: widen reconciliation's capture eligibility to any not-yet-captured job

• 1.1 Prove and implement state-independent capture of a job's recorded dispatch allocation

  • Replace the existing test that locks in the bug (asserting a not-yet-running job must never be reconciled) with one asserting the opposite: a job whose recorded dispatch decision is found this round gets its allocation captured regardless of whether it has reached a running state yet
  • Run the new/updated test against the current code first to confirm it fails for the expected reason (the old state-gated behavior), then remove the state precondition from the capture-eligibility check so only "not yet captured" remains
  • Update the function's own explanatory comments so they describe the new state-independent trigger instead of the old state-gated one
  • Observable: the updated unit test suite shows the not-yet-running job's allocation is captured and its recorded entry consumed, where it previously stayed uncaptured
  • Requirements: 1.1, 3.1
  • Boundary: sim_reconcile_unaccounted_jobs

• 1.2 Extend unit coverage for capture-safety guarantees under the widened eligibility

  • Add a test proving a job whose allocation was already captured is never re-searched or re-captured on a later round, regardless of its state
  • Add a test proving a job with no matching recorded decision this round stays uncaptured and remains eligible to be found on a later round
  • Add a test with several simultaneously outstanding jobs whose recorded dispatch decisions land in different rounds, proving each job's captured allocation reflects only its own recorded decision with no cross-job interference
  • Observable: the unit test suite demonstrates no-double-capture, retry-on-no-match, and per-job isolation all hold under the widened eligibility check
  • Requirements: 3.1, 3.2, 4.1, 4.4
  • Boundary: sim_reconcile_unaccounted_jobs

• 2. Integration: prove captured-but-not-running allocations stay excluded from resource-in-use accounting until the job actually runs

• 2.1 Verify resource-in-use accounting defers to the job's real running state, not its capture time

  • Drive a captured, not-yet-running job's allocation through the resource-in-use recomputation and confirm it contributes nothing to worker or shared resource usage
  • Transition that same job to a running state and drive resource-in-use recomputation again, confirming its allocation now contributes from that point forward
  • Observable: a test demonstrates the same captured job's resource usage is zero before its running-state transition and correctly non-zero after, using the exact allocation captured earlier
  • Requirements: 1.2, 1.3, 1.4
  • Boundary: sim_reconcile_unaccounted_jobs, sched_account_jobs
  • Depends: 1.1

• 3. Validation: end-to-end proof under contention, decision-verification mode, and full regression

• 3.1 (P) Verify correct capacity accounting under resource contention with realistic dispatch-to-confirmation separation

  • Build a replay fixture where a worker's capacity is insufficient for every candidate job, with dispatch decisions and their running-state confirmations recorded several rounds apart
  • Confirm the live capacity check sees the correct, non-zero resource-in-use total at each round rather than treating capacity as permanently free
  • Observable: a resource-contention test demonstrates the capacity check's view of resource usage matches what actually happened in the original recording at each round, not a permanently-zero reading
  • Requirements: 1.1, 1.2, 1.3, 1.4
  • Boundary: sim_reconcile_unaccounted_jobs, sched_account_jobs
  • Depends: 1.1

• 3.2 (P) Verify decision-verification mode reports no new mismatch across a round-separated dispatch

  • Run decision-verification mode over a replay where a job's dispatch decision and its running-state confirmation are recorded several rounds apart
  • Confirm the verification pass reports zero mismatches for that job's decision
  • Observable: a decision-verification test run over a multi-round-separated dispatch scenario reports zero mismatches
  • Requirements: 2.1, 2.2, 2.3, 4.2
  • Boundary: sim_reconcile_unaccounted_jobs
  • Depends: 1.1

• 3.3 Confirm full regression: existing reconciliation, interactive/batch parity, and the full scheduler suite are unaffected

  • Re-run the existing reconciliation unit tests that were not modified by this feature and confirm their already-running-job scenarios still pass unchanged
  • Re-run the existing interactive-versus-batch simulation parity coverage and confirm it still passes unchanged
  • Run the full scheduler test suite and confirm no new failures beyond any already-tracked, unrelated pre-existing ones
  • Observable: a full scheduler test suite run shows the same pass/fail baseline as before this feature, with only the new tests added by this feature as net-new coverage
  • Requirements: 3.3, 4.1, 4.3, 4.5
  • Depends: 3.1, 3.2

Implementation Notes

• Task 1.2: the pre-existing test_recordstart_symlink_escape_replies_error failure (bug-1560, test_admin_request.c) is unrelated to this spec and appears in every task's full-suite run — do not treat it as a regression.
• Task 3.1: any NEW xce_sched_sim_driver_t fixture passed to sched_sim_driver_run() in a scheduler test file MUST be declared static, not stack-local — sched_sim_driver_run() spawns a picker thread that is never joined and keeps reading the struct for the life of the process, so a stack-local instance is a latent stack-use-after-return once the test function returns (already-documented class, bug-1598/bug-1632; this spec's own task 3.1 round-1 review caught a live recurrence). sched_sim_driver_start() unconditionally memset()s the struct first, so static storage is safe as long as each fixture is written by exactly one call site. Caveat (task 3.2): this static fix is NOT safe for a helper called multiple times per process (e.g. run_sim_and_capture_stdout()) — an earlier call's never-joined zombie picker thread could still be alive when a later call's memset() reuses the same static storage. Leave multiply-invoked fixtures stack-local; only single-invocation fixtures should convert.
• Task 3.2: sim mode's live picker (sim_picker_thread_fn + builtin_fcfs) never calls on_dispatch() — it unconditionally PENDs every candidate, including a job whose allocation sim_reconcile_unaccounted_jobs() already captured. Such a job stays in the picker's pendJobs and gets re-PENDed every round until its recorded JOB_STATE_CHANGE event actually removes it. Any --sim-verify/decision-stream fixture spanning a captured-but-not-yet-RUN job's intervening rounds must include a matching PEND(jobid) entry at each of those rounds, or the fixture will produce a false mismatch (or mask a real one) unrelated to the behavior under test.