lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

test(scheduler-sim-decision-trace): close validation coverage gaps; sync design.md

Browse source 
Feature validation round 1: tests for trace off keeping recall (1.5),
no logging from REPL trace sources (8.7), reconstructed records in the
trace file (8.1) and first in continue mode (7.3), output after trace
off matching an untraced run (10.4), and tracing enabled only from the
REPL (11.1). design.md now reflects the REPL-side file split, the added
API, the dropped file tag and the test files; duplicate trace
declarations removed from repl_internal.h.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-08 17:33:26 +00:00
parent 7f1c7953a1
commit 5ca48206b2
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
4 changed files with 609 additions and 39 deletions
Download patch file Download diff file Expand all files Collapse all files

61
.kiro/specs/scheduler-sim-decision-trace/design.md
View file

@ -140,22 +140,42 @@ src/scheduler/
│ │ # reconstructed-dispatch record. No I/O.
│ ├── sched_trace_internal.h # Helpers shared by the sched_trace*.c files (task 2.4 split to
│ │ # stay under the 800-line cap); not installed.
│ ├── repl_trace.c # `trace on|off|status` parsing (raw line), settings lifecycle
│ │ # (next/cur references), session begin/end hooks, file open/close,
│ │ # dev/inode conflict checks, inline emission, shutdown
│ ├── repl_trace.c # `trace on|off|status` parsing (raw line), refcounted settings
│ │ # (next/cur references), file open/close, dev/inode conflict
│ │ # checks, shutdown
│ ├── repl_trace_emit.c # Session begin/end hooks, inline emission (live, trace-only and
│ │ # reconstructed records), file write-error reporting
│ ├── repl_trace_store.c # jobid -> latest record index, capture-order eviction at 64 MiB,
│ │ # discarded marks, `info trace` (job, array element, array)
│ └── repl_trace_render.c # Record -> lines on one or two FILE* sinks; units mode; jobid fmt;
│ # hierarchy/indent; summary vs full host section; outcome section
│ ├── repl_trace_render.c # Record -> lines on one or two FILE* sinks; units mode; jobid fmt;
│ │ # hierarchy/indent; summary vs full host section
│ ├── repl_trace_render_outcome.c # Renderer, part 2: outcome section (50-host cap), pend reasons,
│ │ # no-decision and reconstructed endings (800-line split)
│ └── repl_trace_render.h # Helpers shared by the two renderer files; not installed
└── testsuite/
├── test_trace_model.c # sched_trace.c + sched_trace_scan.c unit tests
├── test_trace_capture.c # capture helpers against fixture core/schedobj/workers
├── test_decision_trace.c # XCE_DECISION_TRACE, make_trace_decision, decision->trace transport
├── test_trace_hooks_gate.c # SCHED_TRACE no-evaluation proof; live/non-interactive never trace
├── test_trace_hooks_alloc_loop.c # sched_allocator hooks: selection, limits, no-decision records
├── test_trace_hooks_resreq.c # requirement-branch hooks (single, compound, alternatives)
├── test_trace_host_hooks.c # host pre-filter, host scan and satisfaction-check row hooks
├── test_trace_hooks_runner.c # alloc_runner hooks: plans, positions, hosts, rollback, commit
├── test_trace_render.c # renderer output per section, units modes, truncation, outcome cap
├── test_trace_store.c # store replace/evict/discarded marks, info trace variants
├── test_repl_trace_cmd.c # trace command grammar, errors, status, lifecycle, file rules
├── test_repl_trace_baseline.c # pre-feature REPL transcript baseline (tracing off, 10.4)
├── test_repl_trace_cmd.c # trace command grammar, errors, status, lifecycle
├── test_repl_trace_file.c # trace file rules: open errors, conflicts, replace, shutdown
├── test_repl_trace_emit.c # session hooks, emission order per mode, file copy, write errors
├── test_repl_trace_wiring.c # repl.c/repl_info.c/repl_params.c routing, interactive-only gates
├── test_sim_trace_driver.c # sim driver wiring: trace-only path, breakpoints, reconstructed
│ # records, trace off retention and output (1.5, 10.4), 11.3
├── test_repl_trace_e2e.c # step/continue/breakpoint integration over replay fixtures
│ # (+ test_repl_trace_e2e_fixture.c, test_repl_trace_e2e.h)
├── test_trace_error_paths.c # fault injection into the real allocator thread
├── test_trace_bounds.c # line bound and 64 MiB store bound under large traces
└── test_trace_equivalence.c # trace on vs off: decisions, --sim-verify, --sim-output, accounting
# (+ test_trace_equivalence_cmp.c, test_trace_equivalence.h)
```
### Modified Files
@ -188,8 +208,8 @@ src/scheduler/
- reconstructed-dispatch trace in `sim_reconcile_unaccounted_jobs`' report block;
- drain loop: `XCE_DECISION_TRACE` handled first, then `xce_repl_check_trace(sim, d)` after `sim_print_decision_gate`;
- `xce_repl_trace_session_end(sim)` after the drain loop.
- `src/scheduler/include/repl.h`: `extern` declarations of the forwarded entry points (`repl_trace_session_begin/end`, `repl_trace_on_decision`, `repl_trace_on_trace_only`), following the existing `xce_repl_on_*` pattern so `repl.h` stays self-contained. It also adds inline gates `xce_repl_trace_session_begin/end`, `xce_repl_check_trace` and `xce_repl_check_trace_only`, all forwarding only when `sim->interactive`.
- `src/scheduler/src/repl_internal.h`: `xce_repl_trace_state_t trace` member in `xce_sched_repl_state_t`; declarations for the `repl_trace*` entry points and moved formatting helpers.
- `src/scheduler/include/repl.h`: `extern` declarations of the forwarded entry points (`repl_trace_session_begin/end`, `repl_trace_on_decision`, `repl_trace_on_trace_only`, `repl_trace_on_reconstructed`), following the existing `xce_repl_on_*` pattern so `repl.h` stays self-contained; this is their only declaration site. It also adds inline gates `xce_repl_trace_session_begin/end`, `xce_repl_check_trace`, `xce_repl_check_trace_only` and `xce_repl_trace_reconstructed`, all forwarding only when `sim->interactive`.
- `src/scheduler/src/repl_internal.h`: `xce_repl_trace_state_t trace` member in `xce_sched_repl_state_t`; declarations for the other `repl_trace*` entry points (command, store, renderer, settings file, conflicts, shutdown) and moved formatting helpers.
- `src/scheduler/src/repl.c`: dispatch `trace` (passing the raw line); help gains a trailing "Trace" section and an `info trace` line; `repl_cmd_quit` and EOF call `repl_trace_shutdown`.
- `src/scheduler/src/repl_info.c`: `info trace` route (one branch); usage string updated; units helpers moved out.
- `src/scheduler/src/repl_fmt.c`: receives `translate_units_mode`, `repl_format_numeric_resource_value`, `repl_format_consumable_value_and_used` and `repl_current_units_mode` (behaviour unchanged, now non-static).
@ -449,6 +469,9 @@ void sched_trace_host_begin(xce_sched_trace_rec_t*, const xce_sched_obj_t* obj,
const xce_sched_worker_t* w); /* per-host scope */
void sched_trace_pos_scan_error(xce_sched_trace_rec_t*, int err); /* host scan stopped */
void sched_trace_host_verdict(xce_sched_trace_rec_t*, unsigned verdicts, int err);
/* added in implementation (Implementation Notes 1.5, 2.2): */
void sched_trace_row(xce_sched_trace_rec_t*, const sched_trace_row_t* row); /* CHECK_ROW or held HOST_ROW */
void sched_trace_host_limit_hit(xce_sched_trace_rec_t*, const char* limit, const char* scope); /* summary tally */
void sched_trace_rollback_host(xce_sched_trace_rec_t*, const char* host); /* one call per released claim */
void sched_trace_commit(xce_sched_trace_rec_t*); /* committed plan = the last plan_begin */
/* end of sched_alloc_search_: one budgeted line; also updates decision-wide counters */
@ -582,13 +605,15 @@ When rc > 0 or rc < 0 there is no statement; the commit or internal-error entrie
#### SimIntegration (`sim_loop.c`, `repl.h` gates)
- `xce_repl_trace_session_begin(sim)` runs after `xce_core_begin_session` has succeeded, which means after its `rc < 0` check (`sim_loop.c:1620-1631`), whose failure branch `continue`s without draining. It runs before reconciliation. A failed session begin therefore publishes nothing and holds no settings reference. It resets `rs->trace.round_traced` and `rs->trace.round_flushed` to false in every mode, so a round that ended in step mode (for example continue → breakpoint → `step`) cannot leak the flag into the next continue-mode round. It then sets `cur = next`, then sets `ctl->session_no = rs->session_count` and `ctl->position = 0`, and publishes `sim->core.trace_ctl = cur ? &cur->ctl : NULL`.
- In the reconstructed-dispatch report block (`sim_loop.c:1309-1329`), when `trace_ctl` wants the job, the block builds a `RECONSTRUCTED` record and sets `d->trace = rec` right after `make_dispatch_decision`. The order is then: record add → `sim_print_decision_gate` → `xce_repl_check_trace(sim, d)` → `xce_repl_check_decision(sim, d)` → `sched_trace_rec_free(d->trace)` → `free(d)`. This is the same order as the drain loop, so 7.5 holds for reconstructed dispatches too (2.5). This runs before `signal_go`, so these traces precede the round's live decisions (7.3).
- In the reconstructed-dispatch report block (`sim_loop.c:1309-1329`), when `trace_ctl` wants the job, the block builds a `RECONSTRUCTED` record and sets `d->trace = rec` right after `make_dispatch_decision`, through the gate `xce_repl_trace_reconstructed(sim, d, job)` (`repl_trace_on_reconstructed` in `repl_trace_emit.c`). The order is then: record add → `sim_print_decision_gate` → `xce_repl_check_trace(sim, d)` → `xce_repl_check_decision(sim, d)` → `sched_trace_rec_free(d->trace)` → `free(d)`. This is the same order as the drain loop, so 7.5 holds for reconstructed dispatches too (2.5). This runs before `signal_go`, so these traces precede the round's live decisions (7.3).
- Drain loop:
- `if (d->op == XCE_DECISION_TRACE) { xce_repl_check_trace_only(sim, d); sched_trace_rec_free(d->trace); free(d); continue; }` comes first.
- `xce_repl_check_trace(sim, d)` sits between `sim_print_decision_gate` and `xce_repl_check_decision`. It is a no-op when `d->trace == NULL`. `sched_trace_rec_free(d->trace)` is called just before the existing `free(d)`.
- `xce_repl_trace_session_end(sim)` runs after the drain loop, before `plugin_end_session`. It sets `sim->core.trace_ctl = NULL`, drops `cur` (closing its file if it is no longer `next`), and clears the in-progress flag.
#### ReplTraceControl (`src/repl_trace.c`)
#### ReplTraceControl (`src/repl_trace.c`, `src/repl_trace_emit.c`)
`repl_trace.c` holds the command, status, settings, file and shutdown code; `repl_trace_emit.c` holds the session hooks and inline emission.
**Responsibilities & Constraints**
- Parses the raw input line for `trace` (1.9, 1.12):
@ -619,13 +644,18 @@ When rc > 0 or rc < 0 there is no statement; the commit or internal-error entrie
**Contracts**: Service [x] / State [x]
```c
bool repl_trace_cmd(xce_sched_sim_driver_t* sim, const char* raw_line); /* returns false: re-prompt */
/* repl_trace.c (declared in repl_internal.h) */
bool repl_trace_cmd(xce_sched_sim_driver_t* sim, const char* raw_line); /* returns false: re-prompt */
FILE* repl_trace_settings_file(const xce_repl_trace_settings_t* s); /* NULL: no file */
bool repl_trace_file_conflicts(const xce_sched_sim_driver_t* sim, const char* path);
void repl_trace_shutdown(xce_sched_sim_driver_t* sim); /* clears trace_ctl, releases cur/next, store_fini */
/* repl_trace_emit.c (declared in repl.h, called through its interactive-only gates) */
void repl_trace_session_begin(xce_sched_sim_driver_t* sim);
void repl_trace_session_end(xce_sched_sim_driver_t* sim);
void repl_trace_on_decision(xce_sched_sim_driver_t* sim, xce_decision_t* d); /* takes d->trace */
void repl_trace_on_trace_only(xce_sched_sim_driver_t* sim, xce_decision_t* d);
bool repl_trace_file_conflicts(const xce_sched_sim_driver_t* sim, const char* path);
void repl_trace_shutdown(xce_sched_sim_driver_t* sim);
void repl_trace_on_reconstructed(xce_sched_sim_driver_t* sim, xce_decision_t* d,
const xce_sched_job_t* job); /* attaches a RECONSTRUCTED record (2.5) */
```
`trace status` output (1.7) shows a block for `cur` ("in effect", when a session is in progress) or `next` ("next session", otherwise), plus a second "replaces at next session" block when `next` differs from `cur`. Each block has enabled, selection (rendered with `repl_fmt_jobid_range`), level, line bound and file. A final line gives stored-trace memory `used / 64 MiB`.
@ -644,6 +674,7 @@ void repl_trace_shutdown(xce_sched_sim_driver_t* sim);
```c
void repl_trace_store_put(xce_repl_trace_state_t* ts, xce_sched_trace_rec_t* rec);
size_t repl_trace_store_bytes(const xce_repl_trace_state_t* ts);
void repl_trace_store_fini(xce_repl_trace_state_t* ts); /* frees all records; called by repl_trace_shutdown */
bool repl_cmd_info_trace(xce_sched_sim_driver_t* sim, char tokens[][XCE_REPL_TOKEN_LEN], int ntok);
/* repl_internal.h: the state block embedded in xce_sched_repl_state_t */
@ -660,7 +691,7 @@ typedef struct {
} xce_repl_trace_state_t;
```
#### TraceRenderer (`src/repl_trace_render.c`)
#### TraceRenderer (`src/repl_trace_render.c`, `src/repl_trace_render_outcome.c`, private `src/repl_trace_render.h`)
**Responsibilities & Constraints**
- `repl_trace_render(const xce_sched_trace_rec_t* rec, xce_repl_units_mode_t units, FILE* out1, FILE* out2 /* nullable */)`.
@ -734,7 +765,7 @@ Summary only; no new boundaries.
- **TraceSettings** (REPL-owned, reference-counted): `all`, selectors, level, line bound, file (path + `FILE*` + dev/inode), and the `xce_sched_trace_ctl_t` it publishes. It is referenced by `next` and/or `cur` (see Trace settings lifecycle).
- **TraceControlBlock** (`xce_sched_trace_ctl_t`, REPL-owned, published to the core for one session): an immutable copy of the selection, level and bound, plus `session_no`, `policy_name` and the allocator-owned `position`.
- **TraceRecord** (aggregate root, one per traced selection):
- **Header**: jobid, kind, level, session_no, position, line_bound, the file tag of the settings it was captured under.
- **Header**: jobid, kind, level, session_no, position, line_bound. (The planned "file tag of the settings it was captured under" was dropped in implementation, Implementation Note 4.2: `info trace` is console-only (8.1) and emission writes through the session's `cur` settings, so a record needs no file reference.)
- **Selection block**: policy, queue name and priority, job priority, `queue_time`, cluster-wide limits list.
- **Entries**: an ordered list. Each entry has a type, a depth, and a payload of copied strings and floats with resource flags.
- **Counters**: `plans_tried`, `plans_discarded`, `reqs_none_generated`, `reqs_all_discarded`, `reqs_none_placed`, `host_evals[6]`, `omitted_lines`.