diff --git a/.wolf/bugs/bug-1883.md b/.wolf/bugs/bug-1883.md new file mode 100644 index 00000000..48ed9d37 --- /dev/null +++ b/.wolf/bugs/bug-1883.md @@ -0,0 +1,96 @@ +--- +id: "bug-1883" +status: "open" +status_updated: "2026-09-29" +status_note: "newly logged" +component: "scheduler" +severity: "wrong-result" +timestamp: "2026-09-29T04:07:19Z" +last_seen: "2026-09-29T04:07:19Z" +occurrences: 1 +files: ["src/scheduler/src/plugins/seqjob_allocator.c", "src/scheduler/src/plugin.c", "src/scheduler/src/schedobj.c", "src/scheduler/src/host_iter.c"] +tags: ["scheduler", "allocation", "compound", "seqjob"] +related: ["bug-1768", "bug-1766"] +gitea_issue: 334 +gitea_synced_status: "open" +gitea_synced_hash: "12cdc1100496" +--- + +# bug-1883: xce_scheduler seqjob reuses a stale per-host want from an earlier COMPOUND/ALTER primary + +## Description +**Observed:** (by code reading; no failing run captured yet) +When a COMPOUND or ALTER job has a `maxTasks == 1` primary after another +primary in the same attempt, seqjob's host search for the later primary +starts from the per-host want (`ctx->alloc_req.host`) left behind by the +earlier primary, with `slots:1` appended rather than replacing it: + +``` +.host on entry to seqjob host loop (2nd+ primary, shared ctx) + = + slots:1 +``` + +- `plugin_prepare_allocation()` resets only `alloc_req.shared` in the + global pass (`src/scheduler/src/plugin.c:617-619`), never `.host`. +- `seqjob_prepare_allocation()` appends `slots=1` + (`src/scheduler/src/plugins/seqjob_allocator.c:118`), and + `xvec_idval_store` appends without dedup (`src/lib/xvec.c:247-270`). +- The host iterator pre-filters candidates against this stale vector + (`src/scheduler/src/host_iter.c:136,158`), silently dropping hosts that + would satisfy the later primary's real `{slots:1}` want. +- A global limit on `slots` reads the first match via `get_resval_by_id` + (`src/scheduler/src/resource.c:951-968`), so it sees the stale N, not 1. + +parjob does not have this problem because it clears `.host` per bucket +(`src/scheduler/src/plugins/parjob_allocator.c:478-479`). + +**Expected:** every primary's per-host want is computed from that primary's +own resource request only (the bug-1768 principle), so the later seqjob +primary is pre-filtered and limit-checked against `{slots:1}` plus its own +rusage. +**Impact:** wrong result. A later single-task primary of a COMPOUND/ALTER job +can pend (or pick a later host) although a host that fits it is free; pend +reasons can list hosts filtered on resources the primary never asked for. +Plain single-primary jobs are unaffected (fresh ctx). +**Found by:** review. Discovery/requirements research for spec +`scheduler-allocator-runner` (2026-09-28). That spec deliberately keeps this +behaviour (Requirement 3 AC 6) so the refactor stays behaviour-preserving; +fix here, separately, with its own test. + +## Environment +- OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64 +- libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2) +- Toolchain: gcc (GCC) 15.2.0, meson 1.9.1 +- Build: asan (xce.build) +- Code: scheduler-framework @ 51736fa4-dirty +- Deployment: n/a; found by code reading, not on a deployment. Affects any topology where the scheduler dispatches a COMPOUND/ALTER job with a `maxTasks == 1` primary after another primary. + +## Reproduction +1. `git checkout 51736fa4` (branch `scheduler-framework`). +2. Not yet reproduced by a test. Proposed white-box unit test (same + harness as `src/scheduler/testsuite/test_parjob_e2e_multislot.c`'s + compound tests): one shared `xce_sched_ctx_t`; primary A `maxTasks=4` + dispatched through parjob onto host-1 (4 slots) so `ctx->alloc_req.host` + ends as `slots:4`; then `sched_ctx_set_primary()` for primary B + `maxTasks=1`, and run seqjob against host-2 with exactly 1 free slot. +3. Expected RED: host-2 is pre-filtered out (stale `slots:4`) and B pends. + Expected GREEN after a fix: B is placed on host-2. +- Frequency: deterministic by code reading (not yet executed) +- Evidence: not kept (no run yet). Code-reading trace in the research + summary for spec `.kiro/specs/scheduler-allocator-runner/` (requirements + Requirement 3 AC 6). + +## Root cause +TODO: required for fixed/partial + +## Fix +TODO: required for fixed/partial + +## Verification +- Test: TODO: name + file; RED before the fix, GREEN after +- Suites: TODO: suite results and build profile +- Live: TODO: soak/deploy run, or n/a and why + +## References +- Related: bug-1768 (per-primary sizing), bug-1766 (COMPOUND publish-only-if-all-succeed) +- Spec: `.kiro/specs/scheduler-allocator-runner/` (keeps this behaviour; out of scope there)