docs(scheduler-allocation): approve requirements, add design and tasks

Extensive requirements review found and fixed a real correctness bug
(host_iter shared across plan buckets instead of re-scanned fresh per
bucket, which would silently drop solvable multi-host allocations),
a self-contradiction around parjob_prepare_allocation's scope, a
plan/host-iterator init calling-convention mismatch, and an inverted
kvtab .cmp polarity that would have made the unsatisfiable-resreq
cache permanently non-functional. brief.md brought back into sync
with requirements.md's corrected architecture.

design.md and tasks.md generated and validated (kiro-validate-design
GO after two fixes; independent task-graph sanity review PASS).
Two pre-existing, out-of-boundary gaps found along the way logged as
bug-1766/bug-1767.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
This commit is contained in:
Rongsong Shen 2026-09-04 12:59:39 +00:00
parent c4f2fec0d5
commit 48c789dfcb
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
7 changed files with 1044 additions and 75 deletions

View file

@ -28,7 +28,7 @@ The first gap (hardcoded picker) is addressed in the sibling **scheduler-core**
### Iterator-Driven Allocation Architecture
A lazy `xce_alloc_plan_iter_t` (candidate `(nhosts, slots_per_host)` plans, most-preferred first) composed with a lazy `xce_host_iter_t` (candidate hosts for a given plan, filtered by the existing select-expression evaluator), a `parjob_allocator` implementation that walks the plan iterator to do real combinatorial multi-slot allocation, and a session-scoped unsatisfiable-resreq fingerprint cache so a job whose resreq no plan can satisfy doesn't re-pay full allocation cost for every later job with the same fingerprint in the same scheduling session.
A lazy `xce_alloc_plan_iter_t` (candidate plans, each a partition of the job's task count into per-host slot buckets -- e.g. `[2,1]` for 3 tasks across 2 hosts, not a uniform `(nhosts, slots_per_host)` pair, which cannot express uneven splits -- most-preferred first) composed with a lazy `xce_host_iter_t` (candidate hosts for a given plan, filtered by the existing select-expression evaluator), a `parjob_allocator` implementation that walks the plan iterator to do real combinatorial multi-slot allocation, and a session-scoped unsatisfiable-resreq fingerprint cache so a job whose resreq no plan can satisfy doesn't re-pay full allocation cost for every later job with the same fingerprint in the same scheduling session.
### Three-Layer Iterator Composition
@ -44,14 +44,21 @@ The iterator composition operates at the **PRIMARY leaf**. COMPOUND and ALTER ar
```
xce_alloc_plan_iter_t -- init(resreq) -> next() -> plan -> next() -> ... -> NULL -> fini()
│ plan
│ plan (a partition into per-host buckets, e.g. [2,1])
▼
for each bucket in plan.bucket_slots[] -- NOT a 2-level pipeline: a plan
│ with N>1 buckets needs a DISTINCT
│ (fresh host_iter per bucket -- host per bucket, each found via
│ see "Corrected below") its own full host_iter scan
▼
xce_host_iter_t -- init(plan, workers) -> next() -> worker -> ... -> NULL -> fini()
│ worker
▼
try_alloc(worker, plan) -- check slots, shared res, commit or advance
try_alloc(worker, bucket_slots[i]) -- check slots, shared res, commit or advance
```
Three levels of iteration, not two: plan -> bucket/partition -> worker. A single-bucket plan (seqjob's trivial case) collapses the middle level to one iteration, which is why it's easy to under-specify this as a 2-level pipeline -- see "PRIMARY leaf allocation pseudocode" below for why the middle level must be a real loop with its own fresh `host_iter` per bucket, not folded into the plan/host levels.
### Key Properties
- **Lazy**: `plan_iter->next()` generates one plan at a time. `host_iter->next()` returns one host at a time. No full materialization.
@ -94,19 +101,58 @@ Approach B chosen over A because the iterator abstraction pays off immediately (
### PRIMARY leaf allocation pseudocode
**Corrected (this review) -- superseded from an earlier 2-level version of this pseudocode**, which tried one worker per plan and returned on the first success. That's only valid for a single-bucket plan (seqjob's trivial case); a plan with `nbuckets > 1` needs a distinct host per bucket, so the algorithm needs a third, explicit loop level. It also needs a *fresh* `host_iter` per bucket rather than one shared across the whole plan: `next()` is a forward-only cursor, so sharing it would permanently drop a host an earlier, larger bucket tried and rejected as too small -- even when that same host exactly satisfies a later, smaller bucket. Iterators are stack-allocated by the caller and initialized in place (matching this codebase's existing `sched_ctx_init(&ctx, ...)` convention), not returned by a heap-allocating factory. Full rationale (rollback scope, host exclusivity semantics, why descending bucket order matters for correctness not just speed): `requirements.md`'s "Composition Algorithm" section.
```c
int allocate_primary(sched, primary_resreq, ctx) {
plan_iter = plan_iter_init(primary_resreq, sched);
xce_alloc_plan_iter_t plan_iter_storage;
xce_alloc_plan_iter_t* plan_iter = &plan_iter_storage;
if (plan_iter->init(plan_iter, primary_resreq, sched) != 0) {
return ALLOC_FAIL;
}
while ((plan = plan_iter->next(plan_iter)) != NULL) {
host_iter = host_iter_init(plan, sched->workers);
while ((worker = host_iter->next(host_iter)) != NULL) {
if (try_alloc_on_host(worker, plan, ctx) > 0) {
// plan is a partition of maxTasks into per-host buckets, e.g. [2,1]
// for 3 tasks across 2 hosts -- NOT a uniform (nhosts, slots_per_host).
xce_sched_worker_t* claimed[primary_resreq->maxTasks]; // sized by maxTasks, the true upper bound on bucket count
int nclaimed = 0;
int plan_ok = 1;
// Three levels of iteration: plan (above) -> bucket/partition (this
// for) -> worker (host_iter, fresh per bucket, below).
for (int b = 0; b < plan->nbuckets; b++) {
xce_host_iter_t host_iter_storage;
xce_host_iter_t* host_iter = &host_iter_storage;
xce_sched_worker_t* w;
int bucket_ok = 0;
if (host_iter->init(host_iter, plan, sched->workers) == 0) {
while ((w = host_iter->next(host_iter)) != NULL) {
if (is_claimed(claimed, nclaimed, w)) {
continue;
}
if (try_alloc_on_host(w, plan->bucket_slots[b], ctx) > 0) {
claimed[nclaimed++] = w;
bucket_ok = 1;
break;
}
}
host_iter->fini(host_iter);
plan_iter->fini(plan_iter);
return ALLOC_SUCCESS;
}
if (!bucket_ok) {
plan_ok = 0;
break; // this bucket has no satisfying host -- abandon the plan
}
}
host_iter->fini(host_iter);
if (plan_ok) {
commit_plan_allocation(claimed, nclaimed, ctx);
plan_iter->fini(plan_iter);
return ALLOC_SUCCESS;
}
rollback_plan_allocation(claimed, nclaimed, ctx); // undo partial claims
}
plan_iter->fini(plan_iter);
return ALLOC_FAIL;