Rongsong Shen
f854d9facd
Formalizes the fix for bug-1764 through this project's TDD process: sim_reconcile_unaccounted_jobs() (sim_loop.c:664) triggers on job->accounts == NULL alone, with no check on job->state -- a deliberate widening from scheduler-sim-eager-reconciliation to catch a PEND job's early-recorded DISPATCH before its RUN confirmation replays, but one that never excluded jobs already at a terminal state (DONE/EXIT). Since xce_core_push_events() correctly frees allocs/ accounts on a terminal transition (leaving accounts == NULL, the same trigger signal), a later round's leftover/duplicate DISPATCH entry for that job id gets misread as something to reconcile, resurrecting the allocation on an already-dead job -- directly observable via info worker's assigned-jobs listing showing a completed job as still consuming a worker's resources. Requirement 1 adds a terminal-state exclusion to the trigger check, mirroring job_state_regresses_terminal()'s existing "terminal is permanent" invariant (core_events.c). Requirement 2 captures the resulting REPL-visible outcome without requiring any REPL code change. The JOBS_PUSH resync-merge branch (core_events.c, bug-1734/bug-1738) was investigated during design and confirmed to already guard against this exact problem -- explicitly out of this spec's scope. Generated via kiro-spec-quick with an inline requirements review gate, an inline design review gate, and an independent task-graph sanity review (one NEEDS_FIXES round, repaired: task 2.2's REPL-level verification mechanism was underspecified, now pins down the exact capture_dispatch()-driven harness and stray-DISPATCH-injection technique to use). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
4.5 KiB
Requirements Document
Introduction
The simulation driver's decision-reconciliation logic (sim_reconcile_unaccounted_jobs()) exists to catch a job whose DISPATCH decision was recorded in a replay round before that job's RUN confirmation has been decoded — a deliberate, state-independent widening introduced by scheduler-sim-eager-reconciliation so an early-recorded dispatch is never missed. That widening did not account for jobs that have since moved all the way to a terminal state (DONE or EXIT): once a job completes, its resource allocation is correctly released, but a later round's replay stream can still carry a leftover or duplicate DISPATCH entry for that same job id (for example, from an array job with retried dispatch attempts). Because reconciliation currently triggers on "no allocation recorded" alone, it cannot distinguish "not yet allocated" from "already completed and released," and resurrects the allocation on the completed job. This is directly observable through info worker's assigned-jobs listing, which then shows a job as still consuming a worker's resources long after it finished — bug-1764, confirmed via a live interactive session and independently reproduced with a targeted probe. This feature closes that gap: reconciliation must never act on a job that has already reached a terminal state, while continuing, unchanged, to capture the early-dispatch case for jobs that have not yet terminated.
Boundary Context
- In scope: adding a terminal-state check to the simulation driver's reconciliation logic (
sim_reconcile_unaccounted_jobs()) so it never populates or modifies a job's allocation/accounting state once that job has reached DONE or EXIT. - Out of scope: any change to how
info worker/info summaryrender a job's assignment (the display code is a correct, unmodified consumer of the job's actual allocation state — the defect is upstream of it); any change to how a job's state actually transitions to DONE/EXIT; any change to the replay/decision-stream format; the JOBS_PUSH resync-merge branch's own terminal-state handling incore_events.c, which is believed to already be correct for an analogous problem and is confirmed, not modified, by this feature unless a genuine gap is found (in which case it is reported, not silently folded into this fix). - Adjacent expectations: this feature depends on the job-state-change event handling that already frees a job's allocation on a terminal transition continuing to behave exactly as it does today — this feature does not change when or how a job's allocation is released, only what reconciliation is allowed to do to a job whose allocation is already gone because it terminated.
Requirements
Requirement 1: Terminal-State Guard on Job Reconciliation
Objective: As an operator relying on the simulation driver's internal job/resource bookkeeping to reflect reality, I want reconciliation logic to never re-establish a resource allocation for a job that has already finished, so that a completed job can never be treated as still holding resources.
Acceptance Criteria
- When the simulation driver's reconciliation logic considers a job whose current state is DONE or EXIT, the simulation driver shall not populate or modify that job's allocation or accounting state, regardless of whether a matching recorded dispatch entry exists in the replay stream.
- While a job's state is not terminal, the simulation driver's reconciliation logic shall continue to capture a matching recorded dispatch entry for that job exactly as it does today, unaffected by this feature.
- The simulation driver shall treat a job's terminal state as permanent for reconciliation purposes, consistent with how terminal states are already treated as permanent elsewhere in this driver's job-state handling.
Requirement 2: Assigned-Resource Display Consistency
Objective: As an operator reading info worker's assigned-jobs listing, I want it to never show a job that has already completed, so the listing stays consistent with the running/pending counts info summary reports.
Acceptance Criteria
- When a job has reached a terminal state, the REPL's assigned-jobs listing for the worker that job was dispatched to shall not include that job, even if a later replay round's decoded session carries a recorded dispatch entry for that job id.
- The REPL shall require no code change of its own to satisfy this requirement — the listing already derives its content solely from the job's actual allocation state, and shows only what Requirement 1 keeps correct.