test(scheduler-sim-decision-trace): bounds under large traces; update the scheduler guide

Task 6.4: a 1000-host, 64-task parallel job traced at full level stores
exactly the line bound with exact omitted counts and counters; 10,000
traced decisions keep the trace store within 64 MiB without evicting the
newest. The scheduler guide lists the REPL trace files and test rules
and carries measured file sizes. Full all-components ASAN build passes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-08 17:11:52 +00:00
parent 2e733994c1
commit 1b78fa9638
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
4 changed files with 518 additions and 3 deletions

View file

@ -301,7 +301,7 @@
- a non-interactive simulation never publishes settings.
- _Requirements: 1.6, 2.5, 7.1, 7.2, 7.3, 7.5, 10.1, 11.3_
- [ ] 6. Validation
- [x] 6. Validation
- [x] 6.1 End-to-end trace scenarios over the contention fixture
- Drive the real simulator REPL with tracing on for selected jobs and for all jobs, at both levels.
- Assert that every section of Requirements 2 to 6 appears with the expected content:
@ -337,7 +337,7 @@
- Done when the equivalence test and the full scheduler suite pass under ASAN.
- _Requirements: 10.1, 10.2, 10.3, 10.4, 10.5, 11.4_
- [ ] 6.4 Bounds under large traces and the final full build
- [x] 6.4 Bounds under large traces and the final full build
- Trace a 1000-host parallel job with up to 64 tasks, at full level with the default bound. The stored entries must stay within the bound, with exact counters.
- Run 10,000 traced decisions. Stored-trace memory must never exceed 64 MiB.
- Correct the stale file sizes in the scheduler component guide and list the new files.
@ -376,3 +376,4 @@
- 6.1: the e2e test (test_repl_trace_e2e.c, its _fixture.c and .h) drives the built xce_scheduler: step/full/all with the file copy, continue/breakpoint/summary/info trace, and selected jobs over a run-time extra fixture with compounds, an array and a selection mismatch. `XCE_TRACE_E2E_DUMP=1` prints the transcript. Findings: a compound dispatch shows one entry per member on a shared host, which is faithful (merge_job_alloc keys on instid); `break DECISION jobid=` matches only raw decimal ids, a pre-existing repl_break.c bug logged separately.
- 6.2: test_trace_error_paths.c injects seven faults into the real allocator thread. Hooked sources are #included for the macro renames; the rest are linked so the wraps see their callers. `XCE_TRACE_ERR_DUMP=1` prints the rendered traces. The 2.2 OPEN item is fixed: a limit_row OOM now marks the record incomplete, sets LIMIT via `sched_trace_host_limit_hit(rec, NULL, NULL)` and continues. The 4.2 OPEN item is fixed: an array listing OOM prints "[sim] info trace: array <id>: cannot list its elements: <cause>". The capture and store tests now #include their sources.
- 6.3: test_trace_equivalence.c (with _cmp.c and .h) runs four variants (off, all/summary, all/full+file, subset) in step and continue mode over the contention and 6.1 extra replays. It compares decoded decisions with full pend reasons, recording and snapshot bytes, verify output, and per-step worker/resource/limit dumps; `XCE_TRACE_EQUIV_DUMP=1` prints the transcripts. No difference was found. job->accounts is compared only indirectly, through snapshot job->allocs and the used totals.
- 6.4: test_trace_bounds.c (real allocator, 1000 workers, maxTasks=64) stores exactly 1000 entries and counts 187,930 omitted, with counters equal to the unbounded run. Over 10,000 traced puts the store peaks at 67,108,800 bytes and never evicts the newest. `XCE_TRACE_BOUNDS_DUMP=1` prints the numbers; the meson timeout is 300 s. The full all-components ASAN build passed: 148 C tests, xceadmin 373, Go ok. The large pend outcome showed ~63 duplicate NO_RESOURCE reasons per host, a pre-existing libxce/scheduler behaviour logged separately.