docs(scheduler-allocator-runner): requirements, design and tasks; log bug-1892 and bug-1893

Requirements reviewed and revised (seqjob per-bucket reset, nslots/nhosts,
bug-1893 fix in the runner), design and 11-task plan generated and reviewed.
bug-1892: unsatisfy_() reads index-keyed uncommitted amounts by position.
bug-1893: allocators' shared pend-reason loop reads past the unmet list.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-09-30 07:35:14 +00:00
parent 1fb899cd94
commit e2266582b2
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
8 changed files with 900 additions and 45 deletions

View file

@ -33,7 +33,11 @@ Driver: `issues/new_requirements.md` item 4.
- One framework function owns shared-resource prep, the bucket -> host-claim
loop with rollback, and the commit tail.
- Allocation decisions are bit-for-bit identical to today for every existing
test and sim-parity fixture.
test and sim-parity fixture. Intentional differences, all outside the
decision and main pend reason: pending seqjob jobs may gain per-host
`NO_RESOURCE` reasons (Constraints item 1); spurious shared reasons are
dropped (bug-1893, item 6); seqjob gets the `-EINVAL` guard and OOM
rollback (Req 5 AC 1/5).
- `new_requirements.md` item 4 closed.
## Approach
@ -60,7 +64,8 @@ goal that plugins no longer export `my_alloc()`).
- **In**: new allocator iface + framework runner; shared prep/try-plan/
finalize helpers; delete both `my_alloc()`s and parjob's claim helpers;
retype `xce_alloc_plan_iter_t.init` to `xce_sched_core_t*`; equivalence
tests; update `issues/new_requirements.md` item 4.
tests; the bug-1893 loop-bound fix (Constraints item 6); update
`issues/new_requirements.md` item 4.
- **Out**: see Out of Boundary.
## Boundary Candidates
@ -76,10 +81,16 @@ goal that plugins no longer export `my_alloc()`).
- Collapsing `sched_make_seqjob_allocation` / `sched_make_parjob_allocation`
dispatch in `scheduler.c` (optional, only if trivially falls out).
- The `commit` override hook.
- bug-1892 (`unsatisfy_()` reads the index-keyed uncommitted vector by list
position; wrong admission for later primaries of COMPOUND/ALTER jobs). It
lives in `resource.c`'s satisfy checks, which the runner calls unchanged;
fix it separately. Equivalence tests for COMPOUND/ALTER must not encode its
wrong outcomes as expected values.
- The `FIXME: no duration now` (NULL duration to `sched_make_allocs`):
carry it over unchanged, don't fix it here.
- Any change to plan ordering, host ordering, pend-reason content, or
preemption.
preemption, except the decided seqjob host-reason change in
Constraints item 1.
## Upstream / Downstream
- **Upstream**: `scheduler-allocation` (plan/host iterators, Req 4
@ -102,25 +113,72 @@ goal that plugins no longer export `my_alloc()`).
- Behavioral equivalence is mandatory. Preserve exactly: bug-1882 (half-built
alloc never left on `ctx->alloc`), bug-1767 (parjob `maxTasks <= 1`
returns 0 with no fabricated reason, and before the lazy reason is
created), bug-1768; `-ENOMEM` cleanup order of both current tails.
created), bug-1768; `-ENOMEM` cleanup order of both current tails, plus
the seqjob OOM rollback of item 2 (Req 5 AC 5).
- Divergences found in discovery that the design MUST resolve explicitly
(they are not in the 2026-09-23 proposal):
1. parjob does `xvec_idval_fini/init(&ctx->alloc_req.host)` per bucket
(`parjob_allocator.c:452`); seqjob never resets it. Decide whether the
shared loop always resets, and prove it is a no-op for seqjob.
2. seqjob builds its halloc directly from `ctx->alloc_req.host`; under the
shared per-claim snapshot it gains a new `-ENOMEM` path. It must go
through the same rollback (`sched_ctx_uncommit_host`).
3. seqjob does not set `ctx->nslots/nhosts` in its loop. Its values come
from `seqjob_prepare_allocation` (1,1). `sched_try_alloc_on_host_()`
(`resource.c`) does not read them directly; check which hook does, and
that `bucket_slots[0] == 1` for seqjob.
4. seqjob has no `maxTasks` guard today. A `seqjob_accepts`
(`maxTasks == 1`) changes behaviour for any direct caller with another
value (several testsuite files reference `seqjob_allocator`). Either
make `accepts` NULL for seqjob or audit those callers.
1. **Decided 2026-09-29: the shared loop resets `ctx->alloc_req.host` to
empty before every bucket's host scan, for both allocators.** Background:
the per-host request used by the authoritative check is already rebuilt
for every candidate (`sched_try_alloc_on_host_()` -> `plugin_prepare_
allocation()` with `ctx->worker` set resets and refills it,
`plugin.c:639`). The per-bucket reset only affects the host iterator's
pre-filter (`host_iter.c:143`), which reads `alloc_req.host` before that
rebuild and silently skips hosts that fail. parjob already resets
(`parjob_allocator.c:479`), so its pre-filter does no resource filtering.
seqjob does not, so its pre-filter reads stale content: the job-level
`slots:1` for the first candidate, then the last rejected candidate's
full request. Resetting for seqjob too drops the stale-state dependency
and makes seqjob's pre-filter a resource no-op, like parjob's.
**Accepted consequence (intentional change):** host choice is unchanged
(a host the stale pre-filter rejects also fails the authoritative check
while the per-host request is host-independent; where a limit makes it
host-specific, the stale filter could wrongly skip a host, which the
reset fixes). But a pending seqjob job now gets a `PREASON_NO_RESOURCE`
host/resource reason from the authoritative check
(`resource.c:575`) for every host that passes the select filter and
lacks resources, not only for hosts the stale pre-filter let through.
`main_reason` is unaffected (host/resource reasons go to
`res_based_reasons`, `lib/preason.c:147`), so sim parity
(`sim_verify_pend_main_reason_eq`) is unaffected; no testsuite file
asserts `PREASON_*` host reasons (checked 2026-09-29).
Also accepted (found in design review): seqjob now runs the per-host
prepare fan-out on those hosts too, so the limit plugin may instantiate
a host limit's virtual resource and set its value on the worker earlier
than before (decision-neutral, lazy creation moves earlier), at the cost
of one prepare + check per short host per pending seqjob attempt.
Rejected: keeping the reset parjob-only (preserves the stale-state
quirk in shared code); staging each bucket's real request before the
scan so the pre-filter filters correctly (changes parjob's pend
reasons too; a separate follow-up if wanted).
2. **Decided (requirements Req 5 AC 5):** seqjob builds its halloc directly
from `ctx->alloc_req.host`; under the shared per-claim snapshot it gains
a new `-ENOMEM` path, which goes through the same rollback
(`sched_ctx_uncommit_host`) as parjob's.
3. **Decided 2026-09-29: for seqjob, `ctx->nslots` and `ctx->nhosts` are
always 1.** Their only consumer is `sched_fill_limit_requests()` in
`plugin_prepare_allocation()` (`plugin.c:682` job-level pass, `:699`
per-host pass); `sched_try_alloc_on_host_()` does not read them. Today
seqjob sets 1/1 only in its job-level prepare pass
(`seqjob_allocator.c:92`) and relies on the values carrying into the
per-host passes. The shared loop sets `ctx->nslots =
plan->bucket_slots[b]` and `ctx->nhosts = 1` before every bucket's
host scan (as parjob does, `parjob_allocator.c:492`); seqjob's plan is
always one bucket of one slot (`seqjob_allocator.c:163`), so seqjob
sees exactly 1/1 in every pass. The seqjob plan iterator must keep
`nbuckets == 1` and `bucket_slots[0] == 1` (assert in tests).
4. **Decided (requirements Req 5 AC 3): seqjob's `accepts` is NULL.** seqjob
has no `maxTasks` guard today, and a `seqjob_accepts` (`maxTasks == 1`)
would change behaviour for direct callers with another value (48
testsuite files reference the allocators).
5. Proposal-sketch defects to avoid: `sched_ctx_primary == NULL` typo;
`claims[plan->nbuckets]` VLA (heap-allocate or bound by `maxTasks`).
6. **Decided 2026-09-29 (requirements review): the shared pend-reason loop
iterates the unmet list (`ctx->unmeet.count`), fixing bug-1893.** Both
current `my_alloc()`s loop to `ctx->alloc_req.shared.count` over
`ctx->unmeet`, whose tail is zero-filled, and add reasons naming
resource index 0. Moving the loop into the runner is the natural point
to fix it; main reason and decisions are unaffected.
- Claim `host_req` snapshots are owned by the framework: fini on every exit
path (success, bucket failure, OOM).
- Verify with: `test_seqjob_*`, `test_parjob_*`, `test_seqjob_rewrap_equivalence`,