myxce/.kiro/specs/scheduler-allocator-runner/design.md

Design Document: scheduler-allocator-runner

Overview

Purpose: The scheduler framework takes ownership of the job-allocation algorithm. One framework routine, sched_allocator_run(), performs the shared-resource precheck, the plan → bucket → host claim search with rollback, and result assembly for every allocator plugin. An allocator plugin supplies only a candidate-plan iterator, an optional acceptance guard, and its existing prepare_allocation_hook.

Users: Scheduler plugin authors (a new allocator needs only a plan iterator) and scheduler maintainers (an allocation fix lands once).

Impact: Deletes my_alloc() from seqjob_allocator.c and parjob_allocator.c and parjob's claim helpers; adds alloc_runner.c; adds an allocator iface to the plugin ABI; retypes the plan iterator's init third parameter from xce_scheduler_t* to xce_sched_core_t*. Allocation decisions, host choice, main pend reason and result codes are unchanged for every input that reaches the allocators today. The intentional changes are the four listed in requirements.md → Introduction; the side effects of the per-candidate reset are under System Flows → Consequences.

Goals

• One allocation algorithm, used by seqjob and parjob in normal and simulation mode (1.1, 6.3).
• Behavioural equivalence proven by characterization tests written against the current code before it is removed (7.2).
• issues/new_requirements.md item 4 closed (7.4).

Non-Goals

• A per-plugin result-assembly (commit) hook.
• Retyping sched_make_allocs() or removing plugin->sched.
• Merging sched_make_seqjob_allocation() / sched_make_parjob_allocation() dispatch.
• Adding an allocation duration (the FIXME: no duration now stays).
• Fixing bug-1892 (unsatisfy_() reads uncommitted amounts by position).
• Publishing pend reasons only when they change (follow-up to the larger per-host reason sets of 3.7).

Boundary Commitments

This Spec Owns

• xce_sched_allocator_iface_t (shape, ownership, lifetime) and the allocator_iface field of xce_sched_plugin_t.
• sched_allocator_run() and its internal steps: input/acceptance guards, shared-resource precheck (incl. the bug-1893 loop-bound fix), claim search and rollback, per-candidate alloc_req.host reset and nslots/nhosts assignment, result assembly, cleanup of every attempt-owned resource.
• The xce_alloc_plan_iter_t.init parameter type.
• The seqjob/parjob plugin wiring (templates, iface instances, parjob_accepts) and the removal of their private allocation code.
• Equivalence and runner tests; the pend-reason factory audit's file list; issues/new_requirements.md item 4.

Out of Boundary

• Dispatch (scheduler.c sched_make_primary_alloc_() and the COMPOUND/ALTER callers): routing by maxTasks, the unsatisfiable cache, result merging, session stamping.
• resource.c satisfy checks (sched_check_shared_satisfy, sched_check_host_satisfy, unsatisfy_), sched_try_alloc_on_host_(), plugin_prepare_allocation(), the host iterator, and both plugins' prepare_allocation_hooks: called unchanged.
• The plan iterators' plan sequences (only the parameter type changes).
• sched_make_allocs(), make_*_alloc constructors in libxce, sim_alloc.c registration (comment updates only).
• Snapshot, WAL, recording formats: nothing persisted changes.

Allowed Dependencies

• alloc_runner.c → resource.c (sched_try_alloc_on_host_, sched_check_shared_satisfy, sched_ctx_uncommit_host), schedobj.c (sched_ctx_get_uncomit_shared), host_iter.c (sched_host_iter_*), scheduler.c (sched_make_allocs only), libxce (make_pendreason, add_job_preason, make_host_alloc, make_job_alloc, free_*, xvec_*).
• Plugins → alloc_runner.c only through sched_allocator_run and the iface type.
• alloc_runner.c must not reference any plugin file or plugin-specific symbol; plugins must not call runner internals.

Revalidation Triggers

• Any change to xce_sched_allocator_iface_t or xce_alloc_plan_iter_t signatures (all allocator plugins, test_parjob_combinatorial.c, test_alloc_plan_iter.c, test_alloc_runner.c's fake iface).
• Any change to the order of guard → precheck → search → assembly, or to when ctx->reason/ctx->alloc are set (dispatch, unsatisfiable cache, COMPOUND merge, scheduler-alter-pend-reasons).
• Fixing bug-1892 (changes COMPOUND admission; re-run COMPOUND characterization tests).
• A new allocator plugin (must supply a plan iterator whose buckets are ≤ its task count).

Architecture

Existing Architecture Analysis

sched_make_primary_alloc_() picks core->seqjob_allocator (maxTasks == 1) or core->parjob_allocator and calls plugin->job_alloc_fn(plugin, ctx). Each plugin's my_alloc() then repeats the whole algorithm. Plan generation already sits behind xce_alloc_plan_iter_t, host scanning behind xce_host_iter_t, per-host preparation behind plugin_prepare_allocation(). The picker side already uses a framework-owned iface (picker_iface) with a forward typedef in plugin.h.

Architecture Pattern & Boundary Map

Template method: the framework owns the algorithm; the plugin fills in plan generation.

graph TB
    Dispatch[scheduler dispatch] --> JobAllocFn[plugin job_alloc_fn]
    JobAllocFn --> Runner[sched_allocator_run]
    Runner --> Iface[allocator iface]
    Iface --> SeqPlans[seqjob plan iterator]
    Iface --> ParPlans[parjob plan iterator]
    Iface --> ParAccepts[parjob accepts]
    Runner --> SharedCheck[resource shared check]
    Runner --> HostIter[host iterator]
    Runner --> TryHost[try alloc on host]
    TryHost --> Prepare[plugin prepare allocation]
    Prepare --> SeqPrep[seqjob prepare hook]
    Prepare --> ParPrep[parjob prepare hook]
    Runner --> MakeAllocs[sched make allocs]

• Dependency direction: libxce → resource.c/schedobj.c/host_iter.c → alloc_runner.c → plugins (via iface data only) → dispatch. Plugins reference sched_allocator_run by address in their template; the runner never names a plugin. One sanctioned exception: the runner calls scheduler.c's sched_make_allocs() (a leftward call into the dispatch file), exactly as both my_alloc()s do today; removing it is the out-of-boundary sched_make_allocs() retype follow-up. No other scheduler.c symbol may be used.
• Existing patterns preserved: static iface storage referenced from the plugin struct (picker); white-box test #include of .c files; sched_make_allocs() stubbed in white-box tests.
• Steering compliance: new file under the 800-line cap; plugins stay under src/plugins/; no I/O, no locks, no blocking added (allocator thread only).

Technology Stack

Layer	Choice / Version	Role in Feature	Notes
Scheduler daemon	C (gnu17), Meson	Runner, iface, plugin wiring	No new dependency
Tests	Unity, white-box #include	Characterization + runner tests	Existing convention

File Structure Plan

Directory Structure

src/scheduler/
├── include/
│   ├── plugin.h                 # + forward typedef xce_sched_allocator_iface_t; + allocator_iface field
│   └── sched/alloc_iter.h       # plan iterator init retyped; + xce_sched_allocator_iface_t; + sched_allocator_run prototype
├── src/
│   ├── alloc_runner.c           # NEW: sched_allocator_run and its static steps
│   ├── sim_alloc.c              # comments only (my_alloc references)
│   └── plugins/
│       ├── seqjob_allocator.c   # - my_alloc; + seqjob_alloc_iface; template -> sched_allocator_run
│       └── parjob_allocator.c   # - my_alloc, claim helpers; + parjob_accepts, parjob_alloc_iface
├── meson.build                  # + src/alloc_runner.c in scheduler_srcs
└── testsuite/
    ├── test_alloc_runner.c      # NEW: runner tests with an in-test fake plan iterator
    ├── test_alloc_equivalence_seqjob.c   # NEW: seqjob characterization tests via job_alloc_fn
    ├── test_alloc_equivalence_parjob.c   # NEW: parjob characterization tests via job_alloc_fn
    └── meson.build              # + three targets

Modified Files

• src/scheduler/include/sched/alloc_iter.h — init's third parameter becomes xce_sched_core_t* core; defines xce_sched_allocator_iface_t and declares sched_allocator_run.
• src/scheduler/include/plugin.h — forward typedef and const xce_sched_allocator_iface_t* allocator_iface; in xce_sched_plugin_t, documented with the same ownership note as picker_iface.
• src/scheduler/src/plugins/seqjob_allocator.c — plan-iterator init signature; delete my_alloc(); add static const xce_sched_allocator_iface_t seqjob_alloc_iface (accepts = NULL); template .job_alloc_fn = sched_allocator_run, .allocator_iface = &seqjob_alloc_iface; update comments.
• src/scheduler/src/plugins/parjob_allocator.c — same, plus parjob_accepts() (maxTasks > 1); delete parjob_claim_t, parjob_is_claimed_, parjob_snapshot_host_req_, parjob_rollback_claims_, parjob_try_plan_.
• src/scheduler/src/sim_alloc.c — comment text only.
• src/scheduler/meson.build — add src/alloc_runner.c.
• src/scheduler/testsuite/test_seqjob_allocator.c, test_seqjob_rewrap_equivalence.c, test_parjob_combinatorial.c — add #include "../src/alloc_runner.c"; replace direct my_alloc()/parjob_try_plan_()/parjob_rollback_claims_() calls with sched_allocator_run()/sched_try_plan_()/sched_rollback_claims_(); plan-iterator init third argument becomes a core pointer.
• src/scheduler/testsuite/test_alloc_plan_iter.c — its stub_plan_iter_init third parameter becomes xce_sched_core_t* (plan-iterator retype).
• src/scheduler/testsuite/test_picker_pendreason_factory_audit.c — add src/alloc_runner.c to the scanned list.
• src/scheduler/testsuite/meson.build — three new targets.
• issues/new_requirements.md — item 4 marked resolved.

System Flows

flowchart TD
    Start[sched_allocator_run] --> Inputs{inputs present}
    Inputs -->|no| EInval[return EINVAL]
    Inputs -->|yes| Accept{accepts}
    Accept -->|rejected| Pend0[return 0]
    Accept -->|accepted or NULL| Reason[ensure pend reason record]
    Reason --> Shared{shared request met}
    Shared -->|error| Err[return error]
    Shared -->|no| SharedReasons[one reason per unmet entry then return 0]
    Shared -->|yes| Uncommit[count shared as uncommitted]
    Uncommit --> Init{plan init ok}
    Init -->|no| Pend1[return 0]
    Init -->|yes| Next{next plan}
    Next -->|none| Fini0[fini and return 0]
    Next -->|plan| Grow{claims buffer fits}
    Grow -->|grow fails| FiniOom[fini and return ENOMEM]
    Grow -->|malformed plan| Next
    Grow -->|ok| TryPlan[try plan]
    TryPlan -->|bucket unsatisfied| Next
    TryPlan -->|snapshot OOM| FiniErr[fini and return error]
    TryPlan -->|all buckets claimed| Finalize[build result]
    Finalize -->|OOM| FiniOom2[fini and return ENOMEM]
    Finalize -->|ok| Done[publish ctx alloc then fini and return 1]

Try plan (per bucket, in order): reset alloc_req.host to empty → new host iterator → for each candidate: skip if already claimed in this plan (and reset alloc_req.host again after every rejected candidate, before the next next()); set nslots = bucket_slots[b], nhosts = 1; sched_try_alloc_on_host_() > 0 → snapshot alloc_req.host into the claim (snapshot OOM: uncommit the candidate with the live request, roll back earlier claims, return the error) → next bucket. A bucket with no host rolls back every claim of the plan and returns 0.

Key decisions: the reset and the nslots/nhosts assignment are unconditional (brief items 1 and 3); every return after plan_init succeeded calls plan_fini and frees the claims buffer; claim snapshots are freed by whichever function owns them at that point (see AllocRunner → State Management).

Requirements Traceability

Requirement	Summary	Components	Interfaces	Flows
1.1	One routine for both plugins	AllocRunner, PluginWiring	sched_allocator_run	Start
1.2	Plugin supplies plan iterator, accepts, prepare hook only	AllocatorIface	xce_sched_allocator_iface_t	—
1.3	NULL accepts = accept all	AllocRunner	accepts	Accept
1.4	Rejected = pending, no reason, no checks	AllocRunner	accepts	Accept → Pend0
1.5	No private allocation routine in plugins	PluginWiring	—	—
1.6	Iface static, not freed, plugin lifetime	AllocatorIface, PluginWiring	allocator_iface	—
2.1	Reason record created after guards	AllocRunner (ensure_reason)	—	Reason
2.2	One reason per unmet entry (bug-1893)	AllocRunner (prepare_shared)	—	SharedReasons
2.3	Shared counted once, not released	AllocRunner (prepare_shared)	—	Uncommit
2.4	Prep failures return their error	AllocRunner	—	Err
3.1	Plans in generator order, stop at first satisfied	AllocRunner	plan_next	Next
3.2	Buckets in order, first unclaimed satisfying host	ClaimSearch	sched_host_iter_*	TryPlan
3.3	Roll back claims on unsatisfied bucket	ClaimSearch	sched_ctx_uncommit_host	TryPlan
3.4	Init failure / no plan = pending	AllocRunner	plan_init	Pend1, Fini0
3.5	nslots = bucket slots, nhosts = 1	ClaimSearch	—	TryPlan
3.6	Empty per-host request before each candidate	ClaimSearch	—	TryPlan
3.7	Per-host reasons complete, both allocators	ClaimSearch (via 3.6)	sched_try_alloc_on_host_	TryPlan
3.8	No fixed bucket limit	AllocRunner (claims buffer)	—	Grow
4.1	One host alloc per claim, claim order, snapshot request	ResultAssembly	sched_make_allocs, make_host_alloc	Finalize
4.2	Shared record per shared-request entry	ResultAssembly	sched_make_allocs	Finalize
4.3	jobid 0, empty reserve, no duration	ResultAssembly	make_job_alloc(0, …)	Finalize
4.4	Satisfied → set alloc, return 1	AllocRunner	—	Done
4.5	Results identical to current allocators	ResultAssembly, EquivalenceTests	—	—
5.1	Missing inputs → EINVAL, no reason	AllocRunner	—	EInval
5.2	parjob maxTasks <= 1 → pending	ParjobAccepts	parjob_accepts	Pend0
5.3	seqjob accepts all	PluginWiring (accepts = NULL)	—	Accept
5.4	OOM in result → ENOMEM, nothing left	ResultAssembly	—	FiniOom2
5.5	OOM releases unconverted claims + failed candidate	ClaimSearch, ResultAssembly	sched_rollback_claims_	FiniErr, FiniOom2
5.6	ctx->alloc set only on success	ResultAssembly	—	Done
5.7	Fini iterator and free snapshots on every exit	AllocRunner	plan_fini	all Fini nodes
5.8	Size from each primary's own request	AllocRunner (uses ctx->primary / ctx->alloc_req only)	—	—
6.1	Plan init takes core	PlanIterContract	init(iter, resreq, core)	Init
6.2	Identical plan sequences	PlanIterContract, EquivalenceTests	—	—
6.3	Sim uses the same routine, no scheduler object to plan iterators	PluginWiring, PlanIterContract	—	—
7.1	Existing tests pass, limited test edits	TestMigration	—	—
7.2	Equivalence tests	EquivalenceTests, RunnerTests	—	—
7.3	Factory audit covers runner	TestMigration	—	—
7.4	Item 4 resolved	Closure	—	—
7.5	Invariant audit clean	Closure	—	—

Components and Interfaces

Component	Domain/Layer	Intent	Req Coverage	Key Dependencies	Contracts
AllocatorIface	Plugin ABI (alloc_iter.h, plugin.h)	Declares what an allocator plugin supplies	1.2, 1.3, 1.6	—	Service
PlanIterContract	Plugin ABI (alloc_iter.h)	Plan iterator init takes the core	6.1, 6.2, 6.3	—	Service
AllocRunner	Framework (alloc_runner.c)	Guards, shared precheck, plan loop, cleanup	1.1, 1.3, 1.4, 2.x, 3.1, 3.4, 3.8, 4.4, 5.1, 5.7, 5.8	resource.c (P0), schedobj.c (P0)	Service, State
ClaimSearch	Framework (alloc_runner.c)	Bucket → host claim, snapshot, rollback	3.2, 3.3, 3.5, 3.6, 3.7, 5.5	host_iter.c (P0), resource.c (P0)	Service
ResultAssembly	Framework (alloc_runner.c)	Build and publish xce_job_alloc_t	4.1–4.5, 5.4–5.6	sched_make_allocs (P0), libxce (P0)	Service
PluginWiring	Plugins (seqjob_allocator.c, parjob_allocator.c)	Templates + iface instances; delete private code	1.1, 1.5, 1.6, 5.2, 5.3, 6.3	AllocRunner (P0)	—
ParjobAccepts	Plugin (parjob_allocator.c)	maxTasks > 1 guard	5.2	—	Service
EquivalenceTests / RunnerTests / TestMigration / Closure	Tests, docs	Evidence and closure	4.5, 6.2, 7.1–7.5	—	—

Plugin ABI

AllocatorIface and PlanIterContract

Field	Detail
Intent	The allocator plugin contract and the retyped plan-iterator initialiser
Requirements	1.2, 1.3, 1.6, 6.1, 6.2, 6.3

Contracts: Service [x]

Service Interface

/* include/sched/alloc_iter.h */
struct xce_alloc_plan_iter_s {
    int (*init)(xce_alloc_plan_iter_t* iter, xce_primary_resreq_t* resreq,
                xce_sched_core_t* core);            /* was xce_scheduler_t* */
    xce_alloc_plan_t* (*next)(xce_alloc_plan_iter_t* iter);
    void (*fini)(xce_alloc_plan_iter_t* iter);
    void* private;
};

struct xce_sched_allocator_iface_s {
    /* Optional. 1 = this plugin allocates `primary`, 0 = not (pending, no
     * reason, no checks). NULL = accept every request. */
    int (*accepts)(const xce_primary_resreq_t* primary);
    /* Required. Same contract as xce_alloc_plan_iter_t init/next/fini. */
    int (*plan_init)(xce_alloc_plan_iter_t* iter, xce_primary_resreq_t* resreq,
                     xce_sched_core_t* core);
    xce_alloc_plan_t* (*plan_next)(xce_alloc_plan_iter_t* iter);
    void (*plan_fini)(xce_alloc_plan_iter_t* iter);
};

/* job_alloc_fn for every allocator plugin; `plugin` is xce_sched_plugin_t*.
 * > 0 satisfied (ctx->alloc set), 0 pending, < 0 negative errno. */
extern int sched_allocator_run(void* plugin, xce_sched_ctx_t* ctx);

/* include/plugin.h */
typedef struct xce_sched_allocator_iface_s xce_sched_allocator_iface_t;
/* in xce_sched_plugin_t: static storage owned by the plugin file; the
 * registry never frees it (same ownership as picker_iface). NULL for
 * non-allocator plugins. */
const xce_sched_allocator_iface_t* allocator_iface;

• Preconditions: for an allocator plugin, allocator_iface and its three plan_* members are non-NULL; every plan's nbuckets >= 1 and each bucket_slots[i] >= 1.
• Postconditions: plan_init returns 0 and leaves iter ready for plan_next, or non-zero with nothing for the runner to release (the runner then skips plan_fini). It may read the core; both built-in iterators ignore it.
• Invariants: the iface is static const in the plugin file; plugin templates copy the pointer on registration (1.6).

Implementation Notes

• Integration: seqjob/parjob iterator functions and test_alloc_plan_iter.c's stub iterator change only their third parameter's type; test_parjob_combinatorial.c's poisoned-scheduler case becomes a poisoned-core case (6.2 evidence: plan sequences equal with NULL and poisoned core).
• Risks: a plugin that sets job_alloc_fn = sched_allocator_run without an iface gets -EINVAL (5.1), not a crash.

Framework runner (src/scheduler/src/alloc_runner.c)

AllocRunner

Field	Detail
Intent	The one allocation routine
Requirements	1.1, 1.3, 1.4, 2.1–2.4, 3.1, 3.4, 3.8, 4.4, 5.1, 5.7, 5.8

Responsibilities & Constraints

• Order: (1) input guard: plugin, ctx, plugin->core, plugin->sched, ctx->primary, plugin->allocator_iface and its plan_* members non-NULL, else -EINVAL before touching ctx->reason (5.1); (2) accepts if non-NULL, reject → 0 (1.3, 1.4); (3) ensure_reason: create make_pendreason(0) if ctx->reason == NULL, -ENOMEM on failure (2.1, 2.4); (4) prepare_shared (below); (5) plan_init(iter, ctx->primary, plugin->core), non-zero → 0 without plan_fini (3.4, as today); (6) plan loop; (7) cleanup.
• prepare_shared: sched_ctx_get_uncomit_shared; xvec_idval_fini(&ctx->unmeet); sched_check_shared_satisfy(core, uncomit, &ctx->alloc_req.shared, &ctx->unmeet); < 0 → return it; == 0 → for i < ctx->unmeet.count: vget(resmgr, unmeet.idxs[i]), skip only on rc < 0 (vget returns 0 on success; today's rc <= 0 skips every hit, bug-1893), add_job_preason(reason, PREASON_NO_RESOURCE, NULL, r->name); return 0 (2.2); > 0 → xvec_idval(uncomit, &ctx->alloc_req.shared, XVEC_ADD_FLOAT) once, error → return it (2.3, 2.4). Shared amounts are never subtracted by the runner (2.3).
• Claims buffer: sched_alloc_claim_t* claims (heap), capacity grown with realloc to plan->nbuckets before a plan is tried; growth failure → plan_fini, free, -ENOMEM (3.8). No claim is outstanding between plans.
• Uses only ctx->primary and ctx->alloc_req for sizing (5.8).
• Precondition ctx->alloc == NULL on entry (the dispatch and COMPOUND/ALTER callers move or free any previous result before the next attempt); the runner never frees or overwrites a non-NULL ctx->alloc except with its own result on success.
• A plan with nbuckets < 1 or any bucket_slots[i] < 1 violates the iterator precondition; the runner skips it as unsatisfied (built-in iterators never produce one, so this changes no current outcome).

Contracts: Service [x] / State [x]

Service Interface

int sched_allocator_run(void* plugin, xce_sched_ctx_t* ctx);

/* static, reachable from white-box tests */
static int sched_alloc_ensure_reason_(xce_sched_ctx_t* ctx);          /* 0 | -ENOMEM */
static int sched_alloc_prepare_shared_(xce_sched_core_t* core,
                                       xce_sched_ctx_t* ctx);          /* 1 met | 0 pend | <0 */

State Management

• Attempt-owned: plan iterator private state and the claims buffer, released by the runner on every return path after plan_init succeeded (5.7). Claim snapshots are owned by exactly one function at a time: sched_try_plan_() creates them and frees them itself on 0 / < 0; on 1 it hands all of them to sched_finalize_job_alloc_(), which frees every one on every path. The runner itself never frees a snapshot, so no path can free one twice.
• Thread cancellation: plan_next (both iterators) and sched_host_iter_next() are pthread_testcancel() points, and the allocator thread is cancelled only at shutdown (scheduler.c:905, sim.c:727). A cancel inside an attempt abandons the attempt-scoped heap (iterator state, host-iterator state, claim snapshots, and now the claims buffer) exactly as today; the runner adds no pthread_cleanup_push() handlers, consistent with the rest of the scheduler, which has none. "Every return" in 5.7 means every return path, not an asynchronous cancel.
• Context-owned (not released by the runner): ctx->reason, shared uncommitted amounts, per-host debits of claims already converted into host allocations (5.5 wording), ctx->alloc on success.

ClaimSearch

Field	Detail
Intent	Try one plan: claim a distinct host per bucket, or roll back
Requirements	3.2, 3.3, 3.5, 3.6, 3.7, 5.5

Service Interface

typedef struct {
    xce_sched_worker_t* w;
    xvec_idval_t        host_req;   /* deep copy of ctx->alloc_req.host at claim */
} sched_alloc_claim_t;

/* 1 all buckets claimed (claims[0..*nclaimed) owned by caller)
 * 0 a bucket had no host (all claims rolled back, snapshots freed)
 * <0 snapshot failure (candidate + earlier claims rolled back, snapshots freed) */
static int  sched_try_plan_(xce_sched_core_t* core, xce_sched_ctx_t* ctx,
                            xce_alloc_plan_t* plan, sched_alloc_claim_t* claims,
                            int* nclaimed);
static void sched_rollback_claims_(xce_sched_ctx_t* ctx,
                                   sched_alloc_claim_t* claims, int n);
static int  sched_snapshot_host_req_(xvec_idval_t* dst, const xvec_idval_t* src);

• Preconditions: claims has capacity >= plan->nbuckets.
• Postconditions: on 0 or < 0, every per-host debit made in this call is released (3.3, 5.5) and no snapshot remains.
• Behaviour is parjob's current parjob_try_plan_() with plan->ctx = ctx and the per-candidate nslots/nhosts assignment (3.5), applied to every plugin, and one change: alloc_req.host is cleared at the start of each bucket AND after every rejected candidate, i.e. before every host_iter->next() call (3.6). Clearing only once per bucket (parjob today) is not enough: a rejected candidate leaves its own per-host request in alloc_req.host, and the host iterator's pre-filter (host_iter.c) would then skip later short hosts silently. With the request always empty when next() pre-filters, the pre-filter does select-expression filtering only, every selection-matching unclaimed host reaches sched_try_alloc_on_host_(), and each short one gets its reasons (3.7).

ResultAssembly

Field	Detail
Intent	Turn claims into the published xce_job_alloc_t
Requirements	4.1–4.5, 5.4, 5.5, 5.6

Service Interface

/* 1: ctx->alloc set; -ENOMEM: ctx->alloc untouched, everything built freed.
 * Consumes claims: each snapshot is freed once its host alloc is built;
 * on failure, claims not yet converted are rolled back and freed. */
static int sched_finalize_job_alloc_(xce_scheduler_t* sched, xce_sched_ctx_t* ctx,
                                     sched_alloc_claim_t* claims, int nclaimed);

• Steps, matching parjob's current tail: calloc(nclaimed) host-alloc array (failure: roll back all claims); per claim sched_make_allocs(sched, "", nres, idxs, vals, NULL) (/* FIXME: no duration now */ kept) then make_host_alloc(w->name, nres, allocs), where nres == 0 allows a NULL allocs (failure: free allocs, roll back claims [i, n), free_HostAlloc_list(nclaimed, hallocs)); make_job_alloc(0, nclaimed, hallocs) into a local (failure: free_HostAlloc_list); shared records sched_make_allocs(sched, "", shared.count, …, NULL) when alloc_req.shared.count > 0, else allocs_shared = NULL, nres_shared = 0 (failure: free_JobAlloc_ptr(&local)); then ctx->alloc = local (5.6).
• sched is (xce_scheduler_t*)plugin->sched (the stand-in wrapper in sim), used only for sched_make_allocs() (out-of-boundary retype).
• Equivalence note (4.5): seqjob today builds its one-element host array with make_host_allocs(1, halloc); the runner's calloc + store produces the same array, freed by the same free_HostAlloc_list.

Plugins

PluginWiring and ParjobAccepts

Field	Detail
Intent	Register both plugins on the runner
Requirements	1.1, 1.5, 1.6, 5.2, 5.3, 6.3

/* parjob_allocator.c */
static int parjob_accepts(const xce_primary_resreq_t* primary)
{ return primary->maxTasks > 1; }                       /* bug-1767 */
static const xce_sched_allocator_iface_t parjob_alloc_iface = {
    .accepts = parjob_accepts, .plan_init = parjob_plan_iter_init,
    .plan_next = parjob_plan_iter_next, .plan_fini = parjob_plan_iter_fini };

/* seqjob_allocator.c */
static const xce_sched_allocator_iface_t seqjob_alloc_iface = {
    .accepts = NULL, .plan_init = seqjob_plan_iter_init,
    .plan_next = seqjob_plan_iter_next, .plan_fini = seqjob_plan_iter_fini };

/* both templates */
.job_alloc_fn = (plugin_job_allocate_fn_t)sched_allocator_run,
.allocator_iface = &<plugin>_alloc_iface,

• prepare_allocation_hook, init_fn, fini_fn, *_new_plugin(), register_*() unchanged; sim registration in sim_alloc.c therefore reaches the runner unchanged (6.3).

Consequences of the per-candidate reset (3.6, both allocators)

Because the host iterator's pre-filter no longer skips hosts on stale state, a pending attempt of either allocator now runs sched_try_alloc_on_host_(), and so the full plugin_prepare_allocation() fan-out, on every selection-matching unclaimed host that is short of resources. Today that happens only for the first rejected candidate of each bucket search (parjob) or for candidates that pass the stale filter (seqjob). Besides the complete per-host pend reasons (3.7), this means:

• Limit plugin side effects (limit.c match_job_host_limit_): for a host with a matching host-based limit it allocates the job's obj->hlimits[worker] list, may instantiate the limit's virtual resource in resmgr (instantiate_limit) and writes the limit value into worker->resvals. This is decision-neutral: any later check of that host by any job runs the same lazy instantiation before reading the value, so it only moves when the instance is created. It happens identically in normal and simulation mode (driver state-equivalence holds).
• Cost: one prepare fan-out plus satisfy check per short host per bucket search, where the stale pre-filter used to reject later hosts with a single satisfy check. Repeated attempts for the same unsatisfiable request are still short-circuited by the unsatisfiable cache before the allocator runs (scheduler.c:248).
• Pend-reason volume: a pending job can now carry one reason per short host (per bucket search for parjob). These reasons are published to the controller and persisted with the job. Publishing reasons only when they change is a recorded follow-up (Non-Goals), not part of this spec.

Data Models

No persisted or wire data changes. In-memory only: sched_alloc_claim_t (attempt-scoped) and the allocator_iface pointer (static). Snapshot/WAL/recording field-completeness is not affected (scheduler audit dimension 4: no change).

Error Handling

Error Strategy

The runner follows the allocator contract: > 0 satisfied, 0 pending, < 0 negative errno, early return on each error (project C convention), with cleanup of attempt-owned state on every path after plan_init.

Condition	Result	State left behind
Missing plugin/ctx/core/sched/primary/iface	-EINVAL	nothing; ctx->reason untouched
accepts rejects	0	nothing
make_pendreason fails	-ENOMEM	ctx->reason == NULL
Shared check error / uncommit add error	that error	reason record
Shared unmet	0	reason record with one NO_RESOURCE per unmet entry
plan_init fails	0	reason record, shared uncommitted
No plan satisfied	0	reason record (+ host reasons), shared uncommitted
Plan with nbuckets < 1 or a bucket of < 1 slots	plan skipped as unsatisfied	nothing from that plan
Claims buffer growth fails	-ENOMEM	as above; no host debit outstanding
Snapshot fails	its error (-ENOMEM)	candidate and earlier claims released
Host alloc build fails	-ENOMEM	unconverted claims released; converted ones' debits stay in ctx (as today)
Job alloc / shared records fail	-ENOMEM	ctx->alloc untouched; debits stay in ctx (as today)

Monitoring

Existing TRACE lines in resource.c are unchanged; the runner adds none (the allocator path runs per candidate and is already traced there).

Testing Strategy

TDD order follows the staged migration (research.md): characterization tests are written and pass against the current code first, then must pass unchanged after each switch.

Unit Tests

• test_alloc_runner.c (white-box #include of core.c, resource.c, schedobj.c, host_iter.c, alloc_runner.c; stubbed sched_make_allocs with OOM injection; stubbed plugin_prepare_allocation() that records ctx->nslots/ctx->nhosts/ctx->alloc_req.host.count on each call and stages a fixed per-host request, the same link-stub convention test_resource_accounting.c uses; an in-test fake iface whose plans are scripted):
  • guards: each missing input → -EINVAL, ctx->reason still NULL (5.1); rejecting accepts → 0, no reason, plan_init not called (1.4); NULL accepts reaches plan_init (1.3).
  • shared: two shared resources, one short → exactly one host-less NO_RESOURCE reason naming it (2.2, bug-1893); met → uncommitted added once across a 3-plan attempt (2.3).
  • search: plan order and first-satisfied stop (3.1); distinct hosts per bucket (3.2); failing bucket 2 rolls back bucket 1's debit (w uncommitted back to 0) (3.3); nslots/nhosts seen by a recording prepare hook equal bucket_slots[b]/1 (3.5); alloc_req.host empty before every candidate's pre-filter: with three short hosts in one bucket, all three get host/resource reasons (today only the first would) (3.6, 3.7); a plan with more buckets than a first small plan triggers buffer growth (3.8).
  • errors: snapshot OOM, host-alloc OOM at claim 0 and claim k, job-alloc OOM, shared-records OOM → -ENOMEM, ctx->alloc == NULL, released debits as in the Error table, ASAN clean (5.4, 5.5, 5.6, 5.7).
• test_parjob_combinatorial.c: plan sequences identical for NULL and poisoned core (6.2).

Equivalence (characterization) Tests

• test_alloc_equivalence_seqjob.c / test_alloc_equivalence_parjob.c (white-box #include of core.c, resource.c, schedobj.c, host_iter.c, the plugin file, and from its switch step alloc_runner.c; sched_make_allocs stubbed with OOM injection; plugin_prepare_allocation() stubbed by the existing convention of test_seqjob_allocator.c:84 — reset alloc_req.shared or .host like plugin.c, then call the plugin's own prepare hook — so limit fill is not exercised here; limits stay covered through the real plugin.c by test_sim_alloc_limit_fanout): drive the real plugin through plugin->job_alloc_fn on fixed fixtures (success; pending on shared; pending on hosts; parjob mid-plan rollback; parjob maxTasks 0 and 1; seqjob with maxTasks 0/2 via direct call; COMPOUND second primary with non-overlapping resources so bug-1892 is not exercised). Assert result code, host count/names, per-host resource entries/values, shared records, per-host and shared uncommitted debits, and pend reasons. Written and passing against today's code first (step 2). The fixtures for the intentional changes that the current code can run (seqjob per-host reasons, 3.7; bug-1893, 2.2; seqjob OOM rollback, 5.5) assert the new expected values: they are added at the start of the step that switches that plugin (RED against the not-yet-switched plugin) and turn GREEN within that step, so no step ends with a failing test (4.5, 7.2). seqjob's missing-input fixtures (5.1) are added after the switch only: today's seqjob dereferences the missing pointer, so they cannot fail as an assertion first; test_alloc_runner.c covers the same guard with the fake iface before the switch.

Regression (existing)

• The Req 7.1 list, unchanged in expected outcomes: test_seqjob_allocator, test_seqjob_rewrap_equivalence, test_parjob_combinatorial, test_parjob_e2e_multislot, test_maxtasks_wiring_*, test_scheduler_cache_dispatch*, test_sim_contention_*parity, test_sim_live_allocation, test_sim_alloc_limit_fanout, test_sched_election_sim_fixture, test_snapshot_load_allocator_regression; test_picker_pendreason_factory_audit with the new file listed (7.3). Full /xce-build scheduler (ASAN) suite green.

Invariant audit

• /xce-audit-invariants scheduler after the switch (7.5). Dimensions checked at design time: (1) driver state-equivalence — the runner is reached identically from driver.c and sim_loop.c via the same plugins; (2) core I/O isolation — alloc_runner.c does no I/O; (3) picker→allocator rendezvous — untouched; (4) snapshot parity — no persisted field changes.

Migration Strategy

1. Retype plan-iterator init (header, both iterators, test_parjob_combinatorial.c, test_alloc_plan_iter.c).
2. Characterization tests against current code (pass).
3. Add alloc_runner.c, iface, plugin.h field; switch parjob; delete parjob's private code; update white-box tests.
4. Switch seqjob; delete my_alloc(); update white-box tests.
5. Closure: comments in both plugins and sim_alloc.c, factory audit list, issues/new_requirements.md item 4, invariant audit.

Each step ends with the full scheduler suite green; the only RED tests within a step are that step's own intentional-change fixtures, and a failing step-2 characterization test at steps 3–4 blocks the step.