87 lines
5.3 KiB
Markdown
87 lines
5.3 KiB
Markdown
---
|
|
id: "bug-1892"
|
|
status: "fixed"
|
|
status_updated: "2026-10-01"
|
|
status_note: ""
|
|
component: "scheduler"
|
|
severity: "wrong-result"
|
|
timestamp: "2026-09-30T03:58:52Z"
|
|
last_seen: "2026-09-30T03:58:52Z"
|
|
occurrences: 1
|
|
files: ["src/scheduler/src/resource.c", "src/lib/xvec.c"]
|
|
tags: ["scheduler", "allocation", "compound", "code-review"]
|
|
related: ["bug-1893"]
|
|
fix_commits: ["e822b888"]
|
|
---
|
|
|
|
# bug-1892: xce_scheduler admission check reads per-resource uncommitted amounts by list position, not index
|
|
|
|
## Description
|
|
**Observed:** found by code reading, so there is no failing run yet. `unsatisfy_()` (`src/scheduler/src/resource.c:130-169`) compares
|
|
```c
|
|
if (total[i] < inuse[i] + uncomitted[i] + wanted[i])
|
|
```
|
|
where `total`, `inuse` and `wanted` are *positional* arrays (entry `i` is the
|
|
`i`-th resource of the request, gathered by `get_resvals(..., wanted->idxs, ...)`),
|
|
but `uncomitted` is the context's *dense, resource-index-keyed* vector
|
|
(`ctx->uncomitted.shared` / `ctx->uncomitted.hosts[w]`, 1024 floats,
|
|
`schedobj.c:514/595`), filled by `xvec_idval(uncomit, &req, XVEC_ADD_FLOAT)`,
|
|
which writes `d[idx] += val` (`lib/xvec.c`, `DEFINE_XVEC_IDVAL_OP`). Both
|
|
`sched_check_shared_satisfy_()` (`resource.c:309`) and
|
|
`sched_check_host_satisfy_()` (`resource.c:376`) pass `uncomitted->data`
|
|
straight through, so entry `i` reads the uncommitted amount of resource index
|
|
`i`, not of resource `wanted->idxs[i]`.
|
|
|
|
**Expected:** entry `i` uses `uncomitted[wanted->idxs[i]]` (gather it
|
|
positionally like `total`/`inuse`, e.g. with the existing `get_resvals()`).
|
|
|
|
**Impact:** wrong admission result. Uncommitted amounts are nonzero only
|
|
within one allocation context that already committed something, i.e. from the
|
|
second primary of a COMPOUND (or ALTER) job on, since those primaries share one
|
|
`ctx` (`scheduler.c:348-354`); the first primary and ordinary PRIMARY jobs
|
|
always see all-zero uncommitted vectors, and parjob rolls back failed plans.
|
|
For a later primary, the check subtracts an unrelated resource's uncommitted
|
|
amount (or 0) instead of the earlier primaries' real claim on the same
|
|
resource, so it can over-admit (earlier primary's claim ignored: the job is
|
|
dispatched beyond host/shared capacity) or wrongly pend (another resource's
|
|
amount charged). Pend reasons printed for the same check carry the wrong
|
|
shortfall.
|
|
|
|
**Found by:** review of the `scheduler-allocator-runner` requirements
|
|
(2026-09-29), while checking the shared-resource precheck the runner inherits.
|
|
|
|
## Environment
|
|
- OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
|
|
- libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2)
|
|
- Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
|
|
- Build: asan (xce.build)
|
|
- Code: scheduler-framework @ a370d724-dirty
|
|
- Deployment: n/a: found by code reading; affects any deployment scheduling COMPOUND/ALTER jobs
|
|
|
|
## Reproduction
|
|
1. `git checkout a370d724`; read `src/scheduler/src/resource.c:130-169`
|
|
(`unsatisfy_`) and its two callers at `:309` and `:376`, and
|
|
`src/scheduler/src/schedobj.c:509-514,586-600` (dense 1024-entry uncommitted
|
|
vectors) and `src/lib/xvec.c` `DEFINE_XVEC_IDVAL_OP` (`d[idx] += val`).
|
|
2. Unit-test sketch (white-box `#include "resource.c"`, like
|
|
`test_resource_accounting.c`): one host with `mem` total 10 at resource idx 5
|
|
(idx 0 = `slots`, plenty); set the host's uncommitted vector `[5] = 8`
|
|
(an earlier primary's claim); call `sched_check_host_satisfy()` with
|
|
`wanted = {idx 5: 4}`. Expected 0 (8 + 4 > 10); today it returns 1, because
|
|
it reads `uncomitted[0]` (= 0).
|
|
- Frequency: deterministic (by code reading; no test exercises it yet)
|
|
- Evidence: not kept (code reading only)
|
|
|
|
## Root cause
|
|
`unsatisfy_()` (`src/scheduler/src/resource.c`) indexed the uncommitted vector positionally (`uncomitted[i]`) like `total`/`inuse`/`wanted`, but both callers, `sched_check_shared_satisfy_()` and `sched_check_host_satisfy_()`, pass the allocation context's dense vector keyed by resource index (`ctx->uncomitted.shared` / `.hosts[w]`, 1024 floats, filled with `xvec_idval()`). Entry `i` therefore read the uncommitted amount of resource index `i` instead of `wanted->idxs[i]`. It matters from the second primary of a COMPOUND/ALTER job on, the only place a context holds nonzero uncommitted amounts.
|
|
|
|
## Fix
|
|
`unsatisfy_()` takes the uncommitted `xvec_t*` and reads it through a new `uncomit_at_(uncomitted, wanted->idxs[i])`, which returns 0 for a NULL vector, a negative index, or an index past the vector's end. The last case is needed because `sim_loop.c`'s reconcile checks pass a short all-zero vector sized to the request; reading it by resource index would otherwise run past its end.
|
|
|
|
## Verification
|
|
- Test: `test_host_satisfy_counts_uncommitted_of_the_wanted_resource` RED ("Expected 0 Was 1": over-admit), `test_host_satisfy_ignores_uncommitted_of_another_resource` RED ("Expected 1 Was 0": wrong pend), `test_shared_satisfy_counts_uncommitted_of_the_wanted_resource` RED ("Expected 0 Was 1"), all GREEN after; `test_host_satisfy_short_uncommitted_vector_counts_as_zero` (guards the bounds check, passes before and after), all in testsuite/test_resource_accounting.c.
|
|
- Suites: scheduler ASAN suite 97/97 (build.alloc), 2026-10-01.
|
|
- Live: n/a for HA (no HA path touched). Sanity: the bug-1897 replay (`--sim`) still completes, exit 0, no ASan report.
|
|
|
|
## References
|
|
- Related: none
|