myxce/.kiro/specs/scheduler-sim-repl-verbosity/research.md

Research & Design Decisions

Summary

• Feature: scheduler-sim-repl-verbosity
• Discovery Scope: Extension (existing system) — light discovery
• Key Findings:
  • Two independent, unconditional stdout print paths cause continue-mode volume to match step-mode volume: xce_repl_on_event() (repl_break.c:708-731, gated only by sim->interactive, not by rs->mode) and sim_print_decision() (sim_loop.c:388-411, called directly from sched_sim_driver_run()'s main loop with no sim->interactive/mode gate at all).
  • xce_repl_post_session_check() (repl.c), called once per completed round from sched_sim_driver_run() right after each round's decisions are drained, is the correct integration point for round-boundary rollup/collapse logic — it already runs exactly once per round when interactive, and already exists purely for the step-budget pause check.
  • xce_repl_on_decision() (repl_break.c:776-799) already tallies last_n_dispatch/last_n_pend/last_n_preempt per round, but that tally is decision-event-driven — real replay data shows it may never observe a fresh DISPATCH decision in replay mode even when jobs demonstrably transition to running (bug-1611). The pend-jobid set for the round rollup must therefore be derived by directly querying core->jobs for JOB_IS_PEND state (the same iteration idiom repl_info_jobs() already uses), not solely from the decision tally, to satisfy Requirement 2.4 regardless of bug-1611's outcome.
  • Existing test file test_repl_execution_control.c already drives sched_sim_driver_run() end-to-end via a child-process harness (child_continue_then_inspect, child_continue_twice_after_complete) that captures real process output — this is the correct home for behavioral tests of Requirements 1-3 (mode gating, rollup, collapsing), since it observes actual printed text, not just internal state.
  • No new external dependency is needed — all formatting is plain C string/stdio work consistent with the rest of repl.c/repl_break.c.

Research Log

Extension point analysis: where does continue-mode output actually originate?

• Context: Requirements 1-3 need continue mode to suppress/summarize output that today prints unconditionally.
• Sources Consulted: repl_break.c (xce_repl_on_event, xce_repl_on_decision), sim_loop.c (sim_print_decision), repl.c (xce_repl_post_session_check, repl_should_pause), repl.h (interactive-gated inline wrappers), repl_internal.h (xce_sched_repl_state_s), CodeGraph call-graph for sched_sim_driver_run.
• Findings:
  • xce_repl_check_event() (repl.h, static inline) gates xce_repl_on_event() on sim->interactive only — inside xce_repl_on_event(), repl_print_event_summary() fires unconditionally before any breakpoint check, with no reference to rs->mode.
  • sim_print_decision() has no interactive or mode gate at all; sched_sim_driver_run() calls it directly for every drained decision, the same call site that also drives xce_repl_check_decision() (which does the tally) and sched_record_session_add_decision() (recording).
  • xce_repl_post_session_check() fires once per completed round, strictly after that round's decisions are drained and before the next round's events begin — confirmed via call-graph (sched_sim_driver_run → xce_repl_post_session_pause).
• Implications: Requirement 1's gate belongs inside xce_repl_on_event()/repl_print_event_summary(). Requirement 2/3's rollup and collapsing logic belongs at the xce_repl_post_session_check() boundary, reading state that xce_repl_on_decision()'s existing tally and a direct core->jobs query both feed. Requirement 2/3 also requires suppressing sim_print_decision()'s direct per-decision print when sim->interactive && rs->mode == XCE_REPL_MODE_CONTINUE, since that print path is the one bug-1610 identified as unconditional.

Job identifier range compression: existing formatting conventions

• Context: Requirement 4 needs contiguous array-job indices displayed as ranges.
• Sources Consulted: lib/utils.c (fmt_jobid/fmt_jobid_ex), repl_break.c's repl_print_event_summary() (current JOB_PUSH jobid list rendering).
• Findings: fmt_jobid_ex() already separates the packed 64-bit jobid into (cluster, apiserver, timestamp, job[, idx]) components via EXTRACT_* macros; an array job's idx is already isolated during formatting. There is no existing range-compression helper anywhere in the REPL formatting code — every current jobid list (the JOB_PUSH line, and any future round-rollup pend list) is one fmt_jobid() call per id.
• Implications: A new helper is needed; the natural base for grouping is the already-decoded (cluster, apiserver, timestamp, job) tuple (identical for every element of one array submission), with only idx varying. No change to fmt_jobid/fmt_jobid_ex themselves — the new helper wraps a list of ids, not a single id.

Architecture Pattern Evaluation

Option	Description	Strengths	Risks / Limitations	Notes
Gate at existing hook boundaries (chosen)	Add mode/collapse logic inside xce_repl_on_event, sim_print_decision's call site, and xce_repl_post_session_check	Reuses every existing per-round/per-event integration point; no new hook call sites needed (sim_loop.c already at its documented line-budget ceiling per repl.h's own header comment)	Requires touching two files (repl_break.c, sim_loop.c) instead of one	Matches this codebase's own precedent (repl.h's inline-wrapper gating)
New dedicated "output policy" module	Introduce a separate repl_output.c owning all print decisions, called from a single new hook	Cleaner separation of concerns	Adds a 6th call site into sim_loop.c's round loop, which repl.h's own header comment says there is no line budget for; larger diff for a formatting-only feature	Rejected — over-engineered for the boundary this spec owns

Design Decisions

Decision: Round rollup's pend-jobid set is state-derived, not decision-tally-derived

• Context: Requirement 2.4 requires the round summary to reflect real job status changes even though bug-1611 shows the decision tally may never register a fresh DISPATCH in replay mode.
• Alternatives Considered:
  1. Trust xce_repl_on_decision()'s existing last_n_dispatch/last_n_pend/last_n_preempt tally directly for the pend-jobid set — simplest, but inherits bug-1611's blind spot if it turns out to be a real gap rather than benign replay semantics.
  2. Derive the pend-jobid set each round by directly iterating core->jobs for JOB_IS_PEND state (same idiom as repl_info_jobs()), independent of the decision tally.
• Selected Approach: Option 2 for the pend-jobid set; the existing tally is still used for the dispatch/preempt counts shown in the rollup header, since bug-1611 only questions dispatch visibility and preempt is unaffected.
• Rationale: Requirement 2.4 must hold regardless of how bug-1611 resolves. A state query is authoritative by construction; a decision-tally is only as complete as whatever decisions the replay happens to (re-)generate.
• Trade-offs: One extra core->jobs iteration per round in continue mode (bounded by job count, not a hot path — replay/step already tolerates equivalent iteration cost in repl_info_jobs()/repl_count_jobs()). If bug-1611 is later confirmed a real gap, the dispatch count shown in the rollup may still under-report until that separate bug is fixed — flagged as a known limitation, not silently masked.
• Follow-up: bug-1611 should be investigated (separately from this spec) before or during implementation; if it proves to be a real visibility gap rather than benign replay semantics, a follow-up task should switch the dispatch count to the same state-diff approach used for the pend set.

Decision: Collapse fingerprint keys on (dispatch_count, pend_jobid_set, preempt_count), not raw decision objects

• Context: Requirement 3.2 needs to detect when consecutive rounds are "the same" to collapse them.
• Alternatives Considered:
  1. Compare full decision-object lists round to round.
  2. Compare a compact fingerprint: dispatch count + preempt count + the state-derived pend-jobid set (as an ordered/canonical list).
• Selected Approach: Option 2.
• Rationale: The real replay validated this exact fingerprint shape — 65 consecutive rounds shared an identical (0, {8 jobids from base A + 6 jobids from base B}, 0) fingerprint and collapsed correctly by hand-simulation. Comparing raw decision objects would be more expensive and no more precise, since the pend-jobid set already captures every observable difference the operator cares about.
• Trade-offs: A change in why a job is pending (a future pend-reason feature under scheduler-sim-diagnostics) would not by itself break the collapse, since the fingerprint doesn't include pend reason. Acceptable for this spec's boundary — pend-reason display is explicitly out of scope here.
• Follow-up: If scheduler-sim-diagnostics later adds pend-reason display, revisit whether the fingerprint needs to include it (a revalidation trigger, see design.md Boundary Commitments).

Decision: Round Rollup exposes a shared flush entry point instead of owning every flush trigger itself

• Context: An independent task-graph sanity review (run before task generation) found that breakpoint-hit detection and structural-event classification both live in repl_break.c (xce_repl_on_event, xce_repl_on_begin_round, xce_repl_on_decision, each calling repl_command_loop() directly on a match) — not in repl.c where the initial design placed all Round Rollup logic. Requirements 3.5 and 3.6 cannot be satisfied by round-boundary logic alone.
• Alternatives Considered:
  1. Duplicate collapse-flush logic inside repl_break.c as well as repl.c — rejected, violates single-ownership of the collapse state and risks divergent flush behavior.
  2. Move all breakpoint/structural-event detection into repl.c so Round Rollup owns every trigger directly — rejected, a much larger and riskier change to existing, already-tested breakpoint-matching code with no benefit to this spec's boundary.
  3. Round Rollup exposes one flush entry point (repl_round_collapse_flush); repl_break.c's existing hooks call it before their existing structural-event print / breakpoint-triggered command-loop entry.
• Selected Approach: Option 3.
• Rationale: Keeps collapse-state ownership entirely inside repl.c (single writer, no duplicated logic) while letting the two other legitimate trigger sites (repl_break.c's breakpoint/structural-event detection, and repl.c's own xce_repl_on_replay_complete) call a small, idempotent, safe-on-NULL interface — consistent with this codebase's existing "safe no-op on NULL/not-yet-allocated state" convention already documented on every other REPL hook.
• Trade-offs: Event Echo Gate (repl_break.c) now has a real outbound dependency on Round Rollup's interface, removing the parallel-safety the initial task draft assumed between those two components' implementation work — tests for both can still be written independently, but implementation must be sequenced.
• Follow-up: none — resolved before task generation; design.md and the task graph both reflect this.

Risks & Mitigations

• bug-1611 (dispatch visibility in replay mode) is unresolved — Mitigation: pend-set correctness (Requirement 2.4) does not depend on its resolution (see Design Decision above); dispatch-count accuracy is called out as a known, separately-tracked limitation rather than silently assumed correct.
• sim_print_decision()'s exact call site inside sched_sim_driver_run() was confirmed via call-graph but its surrounding loop body was not fully read — Mitigation: implementation task must re-confirm the precise insertion point and existing control flow around it before editing; this is a normal implementation-time verification, not a design ambiguity (the gating contract — suppress under sim->interactive && continue mode — is unambiguous regardless of exact line placement).
• No existing test coverage on xce_repl_on_event/sim_print_decision text output (per CodeGraph) — Mitigation: Requirement-driven tests added in test_repl_execution_control.c (child-process output capture) before implementation, per this repo's TDD mandate.

References

• .wolf/buglog.json bug-1609, bug-1610, bug-1611 — the three findings that motivated this spec, each with file:line root causes already recorded.
• .kiro/specs/scheduler-sim-diagnostics/brief.md — sibling brief; Requirement 2.4's "why did it pend" is explicitly out of this spec's boundary and belongs to that brief's seed capability 2 if formalized later.