docs(scheduler-allocator-runner): requirements, design and tasks; log bug-1892 and bug-1893
Requirements reviewed and revised (seqjob per-bucket reset, nslots/nhosts, bug-1893 fix in the runner), design and 11-task plan generated and reviewed. bug-1892: unsatisfy_() reads index-keyed uncommitted amounts by position. bug-1893: allocators' shared pend-reason loop reads past the unmet list. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
1fb899cd94
commit
e2266582b2
GPG key ID:
80533536B0EB651C
8 changed files with 900 additions and 45 deletions
86
.wolf/bugs/bug-1892.md
Normal file
86
.wolf/bugs/bug-1892.md
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
---
|
||||
id: "bug-1892"
|
||||
status: "open"
|
||||
status_updated: "2026-09-30"
|
||||
status_note: "newly logged"
|
||||
component: "scheduler"
|
||||
severity: "wrong-result"
|
||||
timestamp: "2026-09-30T03:58:52Z"
|
||||
last_seen: "2026-09-30T03:58:52Z"
|
||||
occurrences: 1
|
||||
files: ["src/scheduler/src/resource.c", "src/lib/xvec.c"]
|
||||
tags: ["scheduler", "allocation", "compound", "code-review"]
|
||||
related: ["bug-1893"]
|
||||
---
|
||||
|
||||
# bug-1892: xce_scheduler admission check reads per-resource uncommitted amounts by list position, not index
|
||||
|
||||
## Description
|
||||
**Observed:** found by code reading, so there is no failing run yet. `unsatisfy_()` (`src/scheduler/src/resource.c:130-169`) compares
|
||||
```c
|
||||
if (total[i] < inuse[i] + uncomitted[i] + wanted[i])
|
||||
```
|
||||
where `total`, `inuse` and `wanted` are *positional* arrays (entry `i` is the
|
||||
`i`-th resource of the request, gathered by `get_resvals(..., wanted->idxs, ...)`),
|
||||
but `uncomitted` is the context's *dense, resource-index-keyed* vector
|
||||
(`ctx->uncomitted.shared` / `ctx->uncomitted.hosts[w]`, 1024 floats,
|
||||
`schedobj.c:514/595`), filled by `xvec_idval(uncomit, &req, XVEC_ADD_FLOAT)`,
|
||||
which writes `d[idx] += val` (`lib/xvec.c`, `DEFINE_XVEC_IDVAL_OP`). Both
|
||||
`sched_check_shared_satisfy_()` (`resource.c:309`) and
|
||||
`sched_check_host_satisfy_()` (`resource.c:376`) pass `uncomitted->data`
|
||||
straight through, so entry `i` reads the uncommitted amount of resource index
|
||||
`i`, not of resource `wanted->idxs[i]`.
|
||||
|
||||
**Expected:** entry `i` uses `uncomitted[wanted->idxs[i]]` (gather it
|
||||
positionally like `total`/`inuse`, e.g. with the existing `get_resvals()`).
|
||||
|
||||
**Impact:** wrong admission result. Uncommitted amounts are nonzero only
|
||||
within one allocation context that already committed something, i.e. from the
|
||||
second primary of a COMPOUND (or ALTER) job on, since those primaries share one
|
||||
`ctx` (`scheduler.c:348-354`); the first primary and ordinary PRIMARY jobs
|
||||
always see all-zero uncommitted vectors, and parjob rolls back failed plans.
|
||||
For a later primary, the check subtracts an unrelated resource's uncommitted
|
||||
amount (or 0) instead of the earlier primaries' real claim on the same
|
||||
resource, so it can over-admit (earlier primary's claim ignored: the job is
|
||||
dispatched beyond host/shared capacity) or wrongly pend (another resource's
|
||||
amount charged). Pend reasons printed for the same check carry the wrong
|
||||
shortfall.
|
||||
|
||||
**Found by:** review of the `scheduler-allocator-runner` requirements
|
||||
(2026-09-29), while checking the shared-resource precheck the runner inherits.
|
||||
|
||||
## Environment
|
||||
- OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
|
||||
- libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2)
|
||||
- Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
|
||||
- Build: asan (xce.build)
|
||||
- Code: scheduler-framework @ a370d724-dirty
|
||||
- Deployment: n/a: found by code reading; affects any deployment scheduling COMPOUND/ALTER jobs
|
||||
|
||||
## Reproduction
|
||||
1. `git checkout a370d724`; read `src/scheduler/src/resource.c:130-169`
|
||||
(`unsatisfy_`) and its two callers at `:309` and `:376`, and
|
||||
`src/scheduler/src/schedobj.c:509-514,586-600` (dense 1024-entry uncommitted
|
||||
vectors) and `src/lib/xvec.c` `DEFINE_XVEC_IDVAL_OP` (`d[idx] += val`).
|
||||
2. Unit-test sketch (white-box `#include "resource.c"`, like
|
||||
`test_resource_accounting.c`): one host with `mem` total 10 at resource idx 5
|
||||
(idx 0 = `slots`, plenty); set the host's uncommitted vector `[5] = 8`
|
||||
(an earlier primary's claim); call `sched_check_host_satisfy()` with
|
||||
`wanted = {idx 5: 4}`. Expected 0 (8 + 4 > 10); today it returns 1, because
|
||||
it reads `uncomitted[0]` (= 0).
|
||||
- Frequency: deterministic (by code reading; no test exercises it yet)
|
||||
- Evidence: not kept (code reading only)
|
||||
|
||||
## Root cause
|
||||
TODO: required for fixed/partial
|
||||
|
||||
## Fix
|
||||
TODO: required for fixed/partial
|
||||
|
||||
## Verification
|
||||
- Test: TODO: name + file; RED before the fix, GREEN after
|
||||
- Suites: TODO: suite results and build profile
|
||||
- Live: TODO: soak/deploy run, or n/a and why
|
||||
|
||||
## References
|
||||
- Related: none
|
||||
80
.wolf/bugs/bug-1893.md
Normal file
80
.wolf/bugs/bug-1893.md
Normal file
|
|
@ -0,0 +1,80 @@
|
|||
---
|
||||
id: "bug-1893"
|
||||
status: "open"
|
||||
status_updated: "2026-09-30"
|
||||
status_note: "newly logged"
|
||||
component: "scheduler"
|
||||
severity: "wrong-result"
|
||||
timestamp: "2026-09-30T03:58:52Z"
|
||||
last_seen: "2026-09-30T03:58:52Z"
|
||||
occurrences: 1
|
||||
files: ["src/scheduler/src/plugins/seqjob_allocator.c", "src/scheduler/src/plugins/parjob_allocator.c"]
|
||||
tags: ["scheduler", "allocation", "preason", "code-review"]
|
||||
related: ["bug-1892"]
|
||||
---
|
||||
|
||||
# bug-1893: xce_scheduler allocators add spurious resource-0 shared pend reasons (loop reads past unmet list)
|
||||
|
||||
## Description
|
||||
**Observed:** found by code reading, so there is no failing run yet. When the shared-resource check
|
||||
fails, both allocators add the pend reasons with
|
||||
```c
|
||||
for (i = 0; i < ctx->alloc_req.shared.count; i++) {
|
||||
rc = vget(core->resmgr, ((int*)ctx->unmeet.idxs.data)[i], (void**)&r);
|
||||
if (rc <= 0) continue;
|
||||
add_job_preason(reason->reasons, PREASON_NO_RESOURCE, NULL, r->name);
|
||||
}
|
||||
```
|
||||
(`seqjob_allocator.c` `my_alloc`, `parjob_allocator.c` `my_alloc`). The loop
|
||||
bound is the number of *requested* shared resources, but `ctx->unmeet` holds
|
||||
only the *unmet* ones, packed at `[0, unmeet.count)` by `unsatisfy_()`
|
||||
(`resource.c:152-165`). Its arrays were `calloc`ed with capacity
|
||||
`shared.count` (`resource.c:280`), so entries `[unmeet.count, shared.count)`
|
||||
are 0.
|
||||
|
||||
**Expected:** iterate `i < ctx->unmeet.count` (as the host path does,
|
||||
`resource.c:566`), giving one `NO_RESOURCE` reason per unmet shared resource.
|
||||
|
||||
**Impact:** wrong pend reason, noise level. When a job requests N shared
|
||||
resources and only M < N are unmet, the loop adds N - M extra
|
||||
`PREASON_NO_RESOURCE` reasons naming whatever resource has index 0 (if one
|
||||
exists), which that job may never have requested. Main reason and allocation
|
||||
decision are unaffected. No out-of-bounds memory access (the tail is inside the
|
||||
allocation).
|
||||
|
||||
**Found by:** review of the `scheduler-allocator-runner` requirements
|
||||
(2026-09-29); the loop moves into the framework runner in that spec.
|
||||
|
||||
## Environment
|
||||
- OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
|
||||
- libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2)
|
||||
- Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
|
||||
- Build: asan (xce.build)
|
||||
- Code: scheduler-framework @ a370d724-dirty
|
||||
- Deployment: n/a: found by code reading; any deployment where a job requests 2+ shared resources and only some are short
|
||||
|
||||
## Reproduction
|
||||
1. `git checkout a370d724`; read the loop in
|
||||
`src/scheduler/src/plugins/seqjob_allocator.c` / `parjob_allocator.c`
|
||||
`my_alloc()` (after `sched_check_shared_satisfy`), and
|
||||
`src/scheduler/src/resource.c:130-169,266-316`.
|
||||
2. Unit-test sketch: resource idx 0 = `slots`; two shared resources A (idx 7,
|
||||
plenty) and B (idx 8, total 0); a seqjob primary requesting 1 of each.
|
||||
Expected: exactly one host-less reason (`NO_RESOURCE`, res "B"); today a
|
||||
second reason names `slots`.
|
||||
- Frequency: deterministic (by code reading; no test exercises it yet)
|
||||
- Evidence: not kept (code reading only)
|
||||
|
||||
## Root cause
|
||||
TODO: required for fixed/partial
|
||||
|
||||
## Fix
|
||||
TODO: required for fixed/partial
|
||||
|
||||
## Verification
|
||||
- Test: TODO: name + file; RED before the fix, GREEN after
|
||||
- Suites: TODO: suite results and build profile
|
||||
- Live: TODO: soak/deploy run, or n/a and why
|
||||
|
||||
## References
|
||||
- Related: none
|
||||
Loading…
Add table
Add a link
Reference in a new issue