lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

refactor(scheduler-sim-decision-trace): split trace sources under the 800-line cap

Browse source 
Task 2.4: outcome capture moves to sched_trace_outcome.c and the
position/host/summary/rollback half of the model to sched_trace_scan.c;
shared helpers go behind sched_trace_internal.h (sti_ prefix). Code is
moved unchanged; every target that compiles the trace sources gets all
four, and the core isolation scan covers the new files.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-08 10:16:46 +00:00
parent eacb1fa599
commit 144a01982e
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
40 changed files with 1355 additions and 1094 deletions
Download patch file Download diff file Expand all files Collapse all files

33
.kiro/specs/scheduler-sim-decision-trace/design.md
View file

@ -27,7 +27,7 @@
## Boundary Commitments
### This Spec Owns
- The trace record model, its arena storage, entry budget and counters (`sched_trace.c`), and the allocator-side capture helpers that read scheduler state (`sched_trace_capture.c`).
- The trace record model, its arena storage, entry budget and counters (`sched_trace.c`, `sched_trace_scan.c`), and the allocator-side capture helpers that read scheduler state (`sched_trace_capture.c`, `sched_trace_outcome.c`), with their shared internal header `sched_trace_internal.h`.
- Two gate fields on `xce_sched_core_t`: `trace_ctl` (session settings snapshot) and `trace_rec` (record under construction). It also owns the `SCHED_TRACE` hook macro.
- The `trace` field on `xce_decision_t`, the `XCE_DECISION_TRACE` op and `make_trace_decision()`.
- The hook call sites in `sched_election.c`, `scheduler.c`, `alloc_runner.c`, `host_iter.c` and `resource.c`. Each is a single gated statement that never changes control flow or return values.
@ -44,7 +44,7 @@
### Allowed Dependencies
- Core side (`sched_trace*.c`): libxce containers and types (`xvec`, `xlist`, `zarray`, `xce/types.h`), `include/sched/alloc_iter.h` (for `xce_alloc_plan_t`), `src/include/utils.h` (only `array_jobid`/`array_idx` for selector matching), `include/sched/types.h`, `include/core.h`, and `include/preason.h`. These files perform no stdio, file or network I/O and do not include REPL or sim headers.
- REPL side (`repl_trace*.c`): `include/sched/trace.h` (read-only use of records), `repl_internal.h`, `repl_fmt.c` helpers, libxce `htab`, and `parse_jobsel`/`array_jobid`/`fmt_jobid` (`utils.h`). It must not call scheduler mutation APIs.
- Dependency direction: `trace.h` types → `sched_trace.c` (model) → `sched_trace_capture.c` (allocator-side readers) → hook sites. Separately, `trace.h` → `repl_trace_render.c` → `repl_trace_store.c` → `repl_trace.c` → `repl.c`/`repl_info.c`/`repl_params.c`/`repl_rollup.c`/`sim_loop.c` call sites. Core-side files never include REPL-side headers.
- Dependency direction: `trace.h` types → `sched_trace.c`/`sched_trace_scan.c` (model) → `sched_trace_capture.c`/`sched_trace_outcome.c` (allocator-side readers) → hook sites. Separately, `trace.h` → `repl_trace_render.c` → `repl_trace_store.c` → `repl_trace.c` → `repl.c`/`repl_info.c`/`repl_params.c`/`repl_rollup.c`/`sim_loop.c` call sites. Core-side files never include REPL-side headers.
### Revalidation Triggers
- Any change to `xce_decision_t` or `xce_decision_op_t` (shared with `scheduler-preemption`).
@ -128,11 +128,18 @@ src/scheduler/
├── include/sched/
│ └── trace.h # Record, entry, settings types; SCHED_TRACE macro; model + capture API
├── src/
│ ├── sched_trace.c # Record model: arena, entry append, budget, counters, verdict tallies,
│ │ # jobsel predicate, record free/size. No I/O, no scheduler-state reads.
│ ├── sched_trace.c # Record model: arena, entry append, budget, requirement/plan/check
│ │ # entries, jobsel predicate, record free/size. No I/O, no
│ │ # scheduler-state reads.
│ ├── sched_trace_scan.c # Record model, part 2: positions, host verdicts and rows, counters,
│ │ # verdict tallies, rollback, commit, search end. No I/O.
│ ├── sched_trace_capture.c # Allocator-side capture helpers that read scheduler structures
│ │ # (job/queue keys, glimits, resource flags, vres lookup, alloc/pend
│ │ # outcome copy with 50-host cap). No I/O.
│ │ # (job/queue keys, glimits, resource flags, vres lookup, check
│ │ # rows, attempt reasons). No I/O.
│ ├── sched_trace_outcome.c # Capture, part 2: alloc/pend outcome copy with 50-host cap,
│ │ # reconstructed-dispatch record. No I/O.
│ ├── sched_trace_internal.h # Helpers shared by the sched_trace*.c files (task 2.4 split to
│ │ # stay under the 800-line cap); not installed.
│ ├── repl_trace.c # `trace on|off|status` parsing (raw line), settings lifecycle
│ │ # (next/cur references), session begin/end hooks, file open/close,
│ │ # dev/inode conflict checks, inline emission, shutdown
@ -141,7 +148,7 @@ src/scheduler/
│ └── repl_trace_render.c # Record -> lines on one or two FILE* sinks; units mode; jobid fmt;
│ # hierarchy/indent; summary vs full host section; outcome section
└── testsuite/
├── test_trace_model.c # sched_trace.c unit tests
├── test_trace_model.c # sched_trace.c + sched_trace_scan.c unit tests
├── test_trace_capture.c # capture helpers against fixture core/schedobj/workers
├── test_trace_hooks_gate.c # SCHED_TRACE no-evaluation proof; live/non-interactive never trace
├── test_trace_render.c # renderer output per section, units modes, truncation, outcome cap
@ -192,9 +199,9 @@ src/scheduler/
- `src/scheduler/include/sim.h`, `src/sim.c`, `src/main.c`: `log_path` plumbing. `sim.h` also declares `sim_print_decision`, which `sim_loop.c` makes non-static (format unchanged) so continue-mode trace emission reuses the exact echo line.
- `src/scheduler/meson.build`:
- add the new sources to the scheduler sources;
- add `sched_trace.c` and `sched_trace_capture.c` to `test_core_isolation.c`'s source-text scan list (forbidden headers and symbols, plus a new no-stdio check for these two files only). They are not added to `sched_core_isolation_lib`, whose `nm` check covers the four core state-machine files; the capture file references scheduler symbols outside that archive's scope;
- add the trace sources (`sched_trace.c`, `sched_trace_scan.c`, `sched_trace_capture.c`, `sched_trace_outcome.c` and `sched_trace_internal.h`) to `test_core_isolation.c`'s source-text scan list (forbidden headers and symbols, plus a new no-stdio check for these files only). They are not added to `sched_core_isolation_lib`, whose `nm` check covers the four core state-machine files; the capture file references scheduler symbols outside that archive's scope;
- add `repl_trace*.c` to `sched_sim_isolation_lib`.
- `src/scheduler/testsuite/meson.build`, `test_core_isolation.c`, `test_sim_isolation.c`, plus every white-box test that `#include`s a hooked or calling source: register the new tests, scan the new files, and add `#include` lines for `sched_trace*.c` / `repl_trace*.c`. `SCHED_TRACE` bodies are compiled even when they never run, so a test that includes a hooked file must also include (or link) `sched_trace.c` and `sched_trace_capture.c`.
- `src/scheduler/testsuite/meson.build`, `test_core_isolation.c`, `test_sim_isolation.c`, plus every white-box test that `#include`s a hooked or calling source: register the new tests, scan the new files, and add `#include` lines for `sched_trace*.c` / `repl_trace*.c`. `SCHED_TRACE` bodies are compiled even when they never run, so a test that includes a hooked file must also include (or link) `sched_trace.c`, `sched_trace_scan.c`, `sched_trace_capture.c` and `sched_trace_outcome.c`.
- **Hooked sources**: `resource.c`, `host_iter.c`, `alloc_runner.c`, `scheduler.c`, `sched_election.c` and `driver.c`. These are included by `test_alloc_runner*.c`, `test_host_iter.c`, `test_alloc_equivalence_*.c`, `test_parjob_combinatorial.c`, `test_core_events.c`, `test.c` and others (about 20).
- **Calling sources**: `repl.c` and `sim_loop.c`, included by about 26 tests.
- The exact list is produced by `grep -l '#include "../src/\(resource\|host_iter\|alloc_runner\|scheduler\|sched_election\|driver\|repl\|sim_loop\).c"' testsuite/*.c` and updated in one dedicated task right after the new files exist.
@ -325,7 +332,7 @@ A settings object (selection, level, bound, open file) is referenced by `rs->tra
### Core Side
#### TraceModel (`include/sched/trace.h`, `src/sched_trace.c`)
#### TraceModel (`include/sched/trace.h`, `src/sched_trace.c`, `src/sched_trace_scan.c`)
**Responsibilities & Constraints**
- Owns `xce_sched_trace_rec_t`: an append-only sequence of typed entries in a per-record arena, plus fixed-size header, selection, counters and outcome blocks. A record is built by exactly one thread at a time and is immutable once attached to a decision.
@ -464,7 +471,7 @@ void sched_trace_finalize_failed(xce_sched_trace_rec_t*, int err);
- Validation: `test_trace_model.c` covers budget edges (bound-1, bound, bound+1), summary tallies with multi-kind hosts, counters under truncation, and `ctl_wants` for plain, element and array selectors.
- Risk: arena growth uses `malloc` on the allocator thread. This happens only for traced jobs, never in live.
#### TraceCapture (`src/sched_trace_capture.c`)
#### TraceCapture (`src/sched_trace_capture.c`, `src/sched_trace_outcome.c`)
**Responsibilities & Constraints**
- Reads scheduler state on the thread that owns it at the moment of capture, and writes entries through TraceModel:
@ -489,7 +496,9 @@ void sched_trace_finalize_failed(xce_sched_trace_rec_t*, int err);
**Contracts**: Service [x]
```c
/* declared in include/sched/trace.h, implemented in sched_trace_capture.c */
/* declared in include/sched/trace.h; implemented in sched_trace_capture.c
* (selection, cluster-wide limits, check rows, attempt reasons) and
* sched_trace_outcome.c (dispatch, pend, reconstructed record) */
void sched_trace_capture_selection(xce_sched_trace_rec_t*, const xce_sched_core_t* core,
const xce_sched_job_t* job);
void sched_trace_capture_glimits(xce_sched_trace_rec_t*, const xce_sched_obj_t* obj);

5
.kiro/specs/scheduler-sim-decision-trace/tasks.md
View file

@ -82,7 +82,7 @@
- a full allocation with tracing never enabled creates no record.
- _Requirements: 11.1, 11.2, 11.3_
- [ ] 2. Allocator-side capture of scheduler state
- [x] 2. Allocator-side capture of scheduler state
- [x] 2.1 Capture job selection facts, cluster-wide limits and attempt failure reasons, test first
- Selection capture: policy name, the job's queue and queue priority, the job's priority and its queue time.
- Cluster-wide limit capture: each matched limit's name and scope instance, read from the job's matched cluster-wide limits.
@ -114,7 +114,7 @@
- The full scheduler test suite, including the 28 tests updated in 1.6, must still build and pass under ASAN with the completed capture source. Missing scheduler sources are added to those tests, or stubbed.
- _Requirements: 2.5, 6.1, 6.2, 6.3, 6.4, 9.4_
- [ ] 2.4 Split the trace sources under the 800-line file cap, with no behaviour change
- [x] 2.4 Split the trace sources under the 800-line file cap, with no behaviour change
- Added during implementation (2026-10-08, user decision). After 2.3, `sched_trace.c` is 1130 lines and `sched_trace_capture.c` is 1065, both over the scheduler's 800-line cap (src/scheduler/CLAUDE.md).
- Move outcome capture (dispatch, pend, reconstructed record) into a new `src/sched_trace_outcome.c`. Split `sched_trace.c` along a clear seam, e.g. entries and line budget versus counters, summaries and rollback, so every trace source is under 800 lines. Shared static helpers go behind an internal header (e.g. `src/sched_trace_internal.h`); the public `include/sched/trace.h` interface is unchanged.
- Register every new source in the scheduler's main source list, in the core isolation test's scan and no-stdio check, and in each white-box test that includes the trace sources (the 27 from 1.6, plus the trace model, capture and gate tests).
@ -359,3 +359,4 @@
- 2.2: rows go through `sched_trace_row()`, a model addition not named in the design, as CHECK_ROW (cluster-wide check, all rows) or HOST_ROW (host scope, unmet rows only). Host rows are held until the host verdict is stored and then stored one level deeper, each counting against the budget; they are dropped at summary level, where the verdict bits and limit tally are still set. Rows arriving outside an open check (`rec->check_open`) or a host scope are dropped, so 3.3/3.4 must open a host scope before host_iter's pre-filter calls `sched_check_host_satisfy` and must call `sched_trace_check_end` on every exit path of the cluster-wide check. 4.1 must render a NULL row name by its index, `limited_res == NULL` as a raw amount, and HOST_ROW at host depth + 1. Capture now calls librtti `vget()`, which every test already links.
- 2.2 OPEN for 6.2: if `limit_row()` runs out of memory, `sched_trace_check_rows` returns early, so that row's LIMIT bit and later rows' SHORTFALL bits are lost. The verdict derived from rows then becomes SELECT and the Req 9.3 counts are off. This needs fault injection to test, so fix it in 6.2: keep setting the verdict bits and continue.
- 2.3: the outcome capture functions are `sched_trace_capture_dispatch`, `sched_trace_capture_pend` and `sched_trace_rec_reconstructed`, declared in trace.h; 3.x and 5.3 call them. Limit charges are classified by the limit flag or a `limit=` prefix and resolved via glimits (shared) or the worker's hlimits (host, worker looked up by host name). An unresolved charge has `limited_res == NULL`, which is the raw-number fallback. Shared pend entries carry host "" and are treated as host-less. For 4.1: limit totals over omitted hosts have a scope without the host, and a NULL host name yields the scope "h=". Follow-up (low): capture's `key_find`/`vget` on resmgr/workers run without the RTTI read lock that `sched_get_worker` takes. That is safe while the sim runs in lock-step; the 2.4 split may wrap them.
- 2.4: the trace sources are now sched_trace.c (record, arena, entries, budget, requirement/plan/check), sched_trace_scan.c (positions, host verdicts/rows, tallies, rollback, commit, search end), sched_trace_capture.c, sched_trace_outcome.c (dispatch, pend, reconstructed) and sched_trace_internal.h (12 shared `sti_` helpers). Every target that compiles one trace .c file needs all four. Later tasks that extend the trace must keep each file under 800 lines and add new files to the core isolation scan.