test(scheduler-allocator-runner): characterize the parjob allocator (task 2.2)
15 tests drive the real parjob plugin through job_alloc_fn: success, short shared, pending on hosts, mid-plan rollback, task counts 0/1, missing inputs, six OOM steps with their debits, COMPOUND. Values the runner changes on purpose sit in *_today tests. Spec: the task showed that parjob records host reasons only for a bucket search's first rejected candidate (the rejected host's request stays in alloc_req.host and the host iterator pre-filters later hosts against it). Decision: the runner clears alloc_req.host before every candidate, for both allocators, so every short host gets its reasons (Req 3.6/3.7); publishing reasons only when they change is a follow-up. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
91a8580948
commit
b4470b2b94
6 changed files with 874 additions and 56 deletions
|
|
@ -22,7 +22,7 @@
|
|||
- Done when the new target passes against the unchanged seqjob allocator
|
||||
- _Requirements: 4.5, 5.4, 5.8, 7.2_
|
||||
|
||||
- [ ] 2.2 Characterize the parallel-job allocator through its registered entry point
|
||||
- [x] 2.2 Characterize the parallel-job allocator through its registered entry point
|
||||
- New test target for the real parjob plugin with the same stubs, injection mechanism and assertions as 2.1
|
||||
- Fixtures: multi-bucket success; pending on a shared resource requesting exactly the short resources; pending on hosts; mid-plan rollback where a later bucket fails and a later plan succeeds; task counts 0 and 1 (pending, no pend-reason record); missing core, scheduler back-reference or primary request (invalid-argument, no pend-reason record); out of memory at the allocation records of host 0 and of a later host, at host allocation 0 and at a later host allocation, at the job allocation and at the shared records; the second primary of a COMPOUND job with non-overlapping resources
|
||||
- Done when the new target passes against the unchanged parjob allocator
|
||||
|
|
@ -41,12 +41,12 @@
|
|||
- [ ] 3.2 Build the plan and claim search with rollback, test first
|
||||
- Plans are tried in generator order and the first fully satisfied plan wins; each bucket claims the first unclaimed satisfying host in host-iteration order
|
||||
- Shared amounts are counted as uncommitted exactly once across an attempt that tries several plans
|
||||
- Before each bucket's host scan the per-host request is empty; before each candidate the slot count is the bucket's task count and the host count is 1
|
||||
- The per-host request is empty before every candidate host is pre-filtered (cleared at the start of each bucket and after every rejected candidate), so every selection-matching unclaimed short host gets its host/resource reasons; before each candidate the slot count is the bucket's task count and the host count is 1
|
||||
- A failing bucket releases every claim of the plan; a claim-record (snapshot) failure releases the candidate and earlier claims and returns the error; a plan with no buckets or a zero-slot bucket is skipped
|
||||
- Claims-list growth failure is injected with a call-counting replacement of the heap allocator applied only to the white-box-included runner source (reused by 3.3); snapshot failure through a new link-time wrap of the libxce vector-copy function (same precedent) that is armed to fail only its next call, because other libxce code also calls it
|
||||
- The claims list grows to the largest plan with no fixed limit; plan-generator initialisation failure returns pending; every return path after initialisation finishes the generator and frees the claims list
|
||||
- Done when runner tests covering order, single shared count, distinct hosts, rollback debits, empty-request-per-bucket, slot/host counts, growth, malformed plans and snapshot failure pass under ASAN
|
||||
- _Requirements: 2.3, 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 3.8, 5.5, 5.7, 7.2_
|
||||
- Done when runner tests covering order, single shared count, distinct hosts, rollback debits, empty-request-before-every-candidate with complete reasons, slot/host counts, growth, malformed plans and snapshot failure pass under ASAN
|
||||
- _Requirements: 2.3, 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 5.5, 5.7, 7.2_
|
||||
|
||||
- [ ] 3.3 Build result assembly and its out-of-memory paths, test first
|
||||
- One host allocation per claim in claim order from the claimed host's name and its request snapshot; one shared record per shared-request entry, none when empty; job identifier 0, empty reservation name, no duration
|
||||
|
|
@ -58,13 +58,14 @@
|
|||
|
||||
- [ ] 4. Move both allocators onto the framework routine
|
||||
- [ ] 4.1 Switch the parallel-job allocator
|
||||
- First add the parjob bug-1893 fixture (two shared resources, one short: exactly one host-less reason, naming the short one; today none is recorded) to the parjob characterization tests and see it fail against the old code
|
||||
- First rewrite the parjob characterization tests' `_today` host-reason assertions to the complete per-host reasons of 3.7 (every selection-matching unclaimed short host of each bucket search) and see them fail against the old code
|
||||
- Also add the parjob bug-1893 fixture (two shared resources, one short: exactly one host-less reason, naming the short one; today none is recorded) to the parjob characterization tests and see it fail against the old code
|
||||
- Give parjob its allocator contract with a task-count acceptance guard (more than one task), register the framework routine as its allocation entry point, and remove its private allocation routine and claim helpers
|
||||
- Build the runner into the parjob characterization target (2.2) so it keeps reaching the plugin through its entry point
|
||||
- Update the parjob combinatorial tests, the only permitted kind of change: build the runner into the white-box test, give their hand-built plugin structs the parjob allocator contract, and call the framework routine and the framework's claim search and rollback instead of the removed helpers
|
||||
- Done when every parjob characterization fixture (including the new one), the parjob combinatorial and end-to-end multi-slot tests, and the full scheduler suite pass under ASAN
|
||||
- _Depends: 2.2, 3.3_
|
||||
- _Requirements: 1.1, 1.2, 1.5, 1.6, 2.2, 5.2, 7.1_
|
||||
- _Requirements: 1.1, 1.2, 1.5, 1.6, 2.2, 3.6, 3.7, 5.2, 7.1_
|
||||
|
||||
- [ ] 4.2 Switch the sequential-job allocator
|
||||
- First add the seqjob intentional-change fixtures to the seqjob characterization tests and see them fail against the old code: a candidate host that the stale per-host request left by the job-level pass used to skip is now checked; complete per-host "no resource" reasons for every selection-matching short host with the main reason unchanged; bug-1893 (the 2.1 fixture that records no shared reason today now expects one host-less reason naming the short resource); release of the claimed host's debit when result building runs out of memory (this rewrites the 2.1 host-allocation out-of-memory fixture's per-host debit assertion to the new expected value)
|
||||
|
|
@ -96,4 +97,6 @@
|
|||
- Several white-box scheduler test targets run with `detect_leaks=0` because their fixtures are not torn down (`testsuite/meson.build:1756`); a leak summary from running such a binary directly is pre-existing, not a regression. New tests that assert "no leak" (3.3) need leak detection on for their own target.
|
||||
- bug-1893 (found in 2.1): `vget()` returns 0 on success, so today's shared-reason loop (`rc <= 0` skip) records NO shared reason at all. The runner must skip only on `rc < 0`. Characterization fixtures record "no shared reason today" in a separate `_today` test that 4.1/4.2 rewrite.
|
||||
- `xce_sched_core_fini()` does not free the string table `xce_sched_core_init()` creates; leak-checked test targets call `xstring_table_fini()` in teardown (see `test_alloc_equivalence_seqjob.c`).
|
||||
- Per-host reasons (decided 2026-09-30 after 2.2): the runner clears `alloc_req.host` before every candidate (start of bucket + after each rejected candidate), both allocators, so every selection-matching unclaimed short host gets reasons. Today parjob gives reasons only for a bucket search's first rejected candidate; the 2.2 file keeps those counts in `_today` tests that 4.1 rewrites.
|
||||
- `test_alloc_equivalence_seqjob.c` is at ~794/800 lines: 4.2's new fixtures go in a second file or after trimming.
|
||||
- 4.1 expectations for the parjob `_today` host-reason test under the per-candidate clear: two-bucket success 2 reasons (A slots, B slots); pending on hosts 6 (A mem, A slots, B mem, B slots, A mem, B mem); mid-plan rollback 5 slots reasons (A, B, C from plan [4]; B, C from plan [2,2] bucket 2). `eq_assert_reason_prefix` checks `>=`, so 4.1 must pin exact counts. The parjob file is at 799/800 lines.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue