feat(scheduler-allocation): add lazy plan/host iterator contracts

Defines xce_alloc_plan_t (partition of a job's task count into
per-host slot buckets), xce_alloc_plan_iter_t, and xce_host_iter_t
(init/next/fini vtables, stack-allocate-then-init-in-place convention
matching sched_ctx_init) as reusable abstractions any allocator can
adopt. Proven usable by a trivial no-op reference implementation and
a fresh-per-bucket host_iter composition test, no allocator wired in
yet.

Task 1.1 (scheduler-allocation spec). Requirements 2.1, 3.1.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
This commit is contained in:
Rongsong Shen 2026-09-04 13:50:31 +00:00
parent d1a3e158ae
commit d5d2c1caf5
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
4 changed files with 482 additions and 1 deletions

View file

@ -1,7 +1,7 @@
# Implementation Plan
- [ ] 1. Foundation: allocation data types and extension points
- [ ] 1.1 Add the lazy plan-generation and host-candidate iterator contracts
- [x] 1.1 Add the lazy plan-generation and host-candidate iterator contracts
- Define the plan representation as a partition of a job's task count into per-host slot buckets, most-preferred ordering to be produced by whichever component generates plans
- Define the plan-iterator and host-iterator contracts (initialize, produce next candidate, release resources) as reusable abstractions any allocator can adopt
- Observable: the contracts compile and are referenced by at least a stub/no-op implementation with no allocator wired in yet
@ -103,3 +103,7 @@
- Observable: the soak reports zero new job loss, divergence, or liveness stall attributable to this feature
- _Requirements: 9.4_
- _Depends: 6.1_
## Implementation Notes
- (1.1) `xce_host_iter_t.next()` returns `xce_sched_worker_t*` (`sched/types.h`, carries `resvals`/`resInUse`/`host_eval_ctx`) — never `xce_worker_t*` (`xce/types.h`, the wire/config type `make_sched_worker()` converts FROM). requirements.md/design.md briefly had this wrong; fixed in commit `d1a3e158`. Task 2.1 (`host_iter.c`'s real implementation) and any later task touching worker candidates must use `xce_sched_worker_t*` from the start.

View file

@ -0,0 +1,140 @@
#if !defined(_SCHED_ALLOC_ITER_H_)
#define _SCHED_ALLOC_ITER_H_ 1
/*
* alloc_iter.h - lazy allocation-plan and host-candidate iterator
* contracts (scheduler-allocation spec, Req 2 AC 1-6 / Req 3 AC 1-6).
*
* This header defines two reusable abstractions any allocator plugin can
* adopt:
*
* xce_alloc_plan_iter_t - generates candidate allocation plans (a
* partition of a primary resreq's maxTasks into per-host slot buckets),
* one per next() call, most-preferred first.
*
* xce_host_iter_t - for a given plan, generates candidate hosts (one
* per next() call) matching the plan's select-expression + availability
* filter.
*
* No allocator is wired onto these interfaces yet -- see
* ../../testsuite/test_alloc_plan_iter.c for a trivial, no-op reference
* implementation of each proving the contract shape compiles and is
* usable.
*/
#include <rtti.h>
#include "resreq.h"
#include "scheduler.h"
#include "xce/types.h"
/*
* xce_alloc_plan_t - a candidate allocation plan: a partition of a
* primary resreq's maxTasks into per-host slot buckets, e.g. for
* maxTasks=3: [3], [2,1], [1,1,1] -- NOT a uniform (nhosts,
* slots_per_host) pair, which cannot express uneven splits.
*
* Produced by xce_alloc_plan_iter_t.next(); the returned pointer is valid
* only until the following next() or fini() call on that same iterator
* (it is not independently heap-owned by the caller).
*/
typedef struct {
int nbuckets; /* number of hosts this plan needs (==
* len(bucket_slots)) */
int* bucket_slots; /* partition of maxTasks; bucket_slots[i] = slots
* needed on the i-th distinct host claimed for
* this plan */
/* host_filter: derived from primary_resreq->select, plus slot
* availability -- consumed by xce_host_iter_t (Req 3), not exposed as
* a separate field here. This interface does not mandate a
* representation for it; each concrete plan-iterator implementation
* carries whatever it needs to reconstruct the filter. */
} xce_alloc_plan_t;
typedef struct xce_alloc_plan_iter_s xce_alloc_plan_iter_t;
/*
* xce_alloc_plan_iter_t - lazy, cancel-safe candidate-plan generator (Req
* 2). One instance is owned by a single allocator's job_alloc_fn for the
* lifetime of one job's allocation attempt.
*
* Allocation convention: stack-allocated by the caller, then initialized
* IN PLACE -- init() takes the iterator's own address as its first
* argument, matching this codebase's existing sched_ctx_init(&ctx, ...)
* convention (schedobj.c). This is NOT a heap-allocating factory
* function:
*
* xce_alloc_plan_iter_t iter_storage;
* xce_alloc_plan_iter_t* iter = &iter_storage;
* xce_alloc_plan_t* plan;
*
* if (iter->init(iter, primary_resreq, sched) != 0) { ... }
* while ((plan = iter->next(iter)) != NULL) { ... }
* iter->fini(iter);
*/
struct xce_alloc_plan_iter_s {
/* Initialize the iterator for a given primary resreq and scheduler
* state. Returns 0 on success, nonzero on error. */
int (*init)(xce_alloc_plan_iter_t* iter, xce_primary_resreq_t* resreq,
xce_scheduler_t* sched);
/* Return the next candidate plan, or NULL when exhausted. Successive
* calls return plans ordered most-preferred to least-preferred, per
* the owning allocator's own preference policy; the NULL
* end-of-iteration signal is stable across repeated calls once
* exhausted. The returned plan pointer is valid until the next
* next() or fini() call. Calls pthread_testcancel() (or an
* equivalent real cancellation point) at least once per invocation. */
xce_alloc_plan_t* (*next)(xce_alloc_plan_iter_t* iter);
/* Release all resources held by the iterator. The interface defines
* no other required cleanup step. */
void (*fini)(xce_alloc_plan_iter_t* iter);
/* Private state: current position, strategy-specific data. */
void* private;
};
typedef struct xce_host_iter_s xce_host_iter_t;
/*
* xce_host_iter_t - lazy, cancel-safe candidate-host filter for one plan
* bucket (Req 3). Shares the same stack-allocate-then-init-in-place
* convention as xce_alloc_plan_iter_t above.
*
* Scoped PER PLAN BUCKET, not per plan: a multi-bucket plan requires a
* fresh xce_host_iter_t instance (a new init() call) for each bucket --
* never one instance shared across a plan's buckets. next() is a
* forward-only cursor with no rewind; sharing one instance across buckets
* would permanently drop a host an earlier, larger bucket tried and
* rejected as too small, even when that same host exactly satisfies a
* later, smaller bucket. Host exclusivity ACROSS buckets of the same plan
* (is_claimed()) is the caller's responsibility, not this interface's --
* this iterator only ever answers "does this select expression +
* availability filter match," with no memory of prior buckets.
*/
struct xce_host_iter_s {
/* Initialize the iterator for a given plan and worker set. Returns 0
* on success, nonzero on error. */
int (*init)(xce_host_iter_t* iter, xce_alloc_plan_t* plan, var workers);
/* Return the next candidate worker matching the plan's host filter,
* or NULL when exhausted for this instance's bucket search --
* exhaustion is scoped to this one instance, not to the plan as a
* whole (see the per-bucket-freshness note above: a plan with more
* buckets remaining requires a fresh host_iter instance). The
* returned worker pointer is valid until the next next() or fini()
* call. Calls pthread_testcancel() (or an equivalent real
* cancellation point) at least once per invocation. */
xce_sched_worker_t* (*next)(xce_host_iter_t* iter);
/* Release all resources held by the host iterator, independent of
* the owning plan iterator's lifecycle. */
void (*fini)(xce_host_iter_t* iter);
/* Private state: current position, filtering state. */
void* private;
};
#endif

View file

@ -1474,6 +1474,24 @@ sched_builtin_fcfs_cursor_uaf_test = executable(
dependencies: [scheduler_test_deps],
install: false)
# Contract test for the lazy allocation-plan and host-candidate iterator
# interfaces (scheduler-allocation spec, task 1.1, Req 2 AC 1-6 / Req 3 AC
# 1-6). Only #includes the new ../include/sched/alloc_iter.h -- no
# allocator is wired onto these interfaces yet (parjob_allocator/
# seqjob_allocator rewrap is later tasks' own work). Proves the contract
# shape compiles and is usable via a trivial, no-op reference
# implementation of each interface, stack-allocated and initialized in
# place per the documented convention, driven through a full
# init()/next()/fini() cycle -- including the exact plan -> per-bucket
# fresh-host_iter composition shape requirements.md's Composition
# Algorithm describes for allocate_primary() (Req 3 AC 4 / Req 4 AC 2).
sched_alloc_plan_iter_test = executable(
'sched_alloc_plan_iter_test',
['test_alloc_plan_iter.c'],
include_directories:[incdir],
dependencies: [scheduler_test_deps],
install: false)
# Register as meson tests so `meson test --suite scheduler` runs them all.
# detect_leaks=0 matches the suite convention (fixtures are not torn down).
test('scheduler test', scheduler_test, suite: 'scheduler',
@ -1656,3 +1674,6 @@ test('scheduler limit plugin test', sched_limit_plugin_test, suite: 'scheduler',
test('scheduler builtin fcfs cursor uaf test',
sched_builtin_fcfs_cursor_uaf_test, suite: 'scheduler',
env: ['ASAN_OPTIONS=detect_leaks=0'])
test('scheduler alloc plan iter test',
sched_alloc_plan_iter_test, suite: 'scheduler',
env: ['ASAN_OPTIONS=detect_leaks=0'])

View file

@ -0,0 +1,316 @@
/*
* test_alloc_plan_iter.c - contract test for the lazy allocation-plan and
* host-candidate iterator interfaces (scheduler-allocation spec, task
* 1.1, Req 2 AC 1 / Req 3 AC 1).
*
* This task only adds a header (sched/alloc_iter.h) declaring
* xce_alloc_plan_t/xce_alloc_plan_iter_t/xce_host_iter_t -- no allocator
* is wired onto them yet (parjob_allocator/seqjob_allocator rewrap is
* later tasks' work). So the RED/GREEN evidence for this task is:
*
* RED: before sched/alloc_iter.h existed, this file failed to compile
* (unknown types xce_alloc_plan_t/xce_alloc_plan_iter_t/
* xce_host_iter_t).
* GREEN: with the header in place, a trivial, no-op reference
* implementation of each interface (below) can be constructed,
* stack-allocated and initialized in place per the documented
* convention, and driven through a full init()/next()/fini()
* cycle -- including the exact plan -> per-bucket host_iter
* composition shape requirements.md's Composition Algorithm
* describes -- proving the contract shape actually compiles and
* is usable, not just that a struct literal type-checks.
*
* Both stub implementations ignore the real xce_primary_resreq_t/
* xce_scheduler_t/plan/workers arguments (ok, per the header's own
* "each concrete plan-iterator implementation carries whatever it needs"
* note) and always yield exactly one candidate before signaling stable
* exhaustion.
*/
#include <errno.h>
#include <pthread.h>
#include <stdlib.h>
#include <string.h>
#include <unity/unity.h>
#include "sched/alloc_iter.h"
/* ---- trivial no-op xce_alloc_plan_iter_t reference implementation ---- */
typedef struct {
int served; /* 0 = not yet served, 1 = exhausted */
int bucket_slots[1];
xce_alloc_plan_t plan;
} stub_plan_priv_t;
static int stub_plan_iter_init(xce_alloc_plan_iter_t* iter,
xce_primary_resreq_t* resreq,
xce_scheduler_t* sched)
{
stub_plan_priv_t* priv;
/* Stub: ignores the real resreq/sched state -- a real implementation
* (parjob's combinatorial one, seqjob's trivial one) would derive its
* plan space from resreq->maxTasks here. */
(void)resreq;
(void)sched;
if (iter == NULL) {
return -EINVAL;
}
priv = (stub_plan_priv_t*)calloc(1, sizeof(*priv));
if (priv == NULL) {
return -ENOMEM;
}
priv->served = 0;
priv->bucket_slots[0] = 1;
priv->plan.nbuckets = 1;
priv->plan.bucket_slots = priv->bucket_slots;
iter->private = priv;
return 0;
}
static xce_alloc_plan_t* stub_plan_iter_next(xce_alloc_plan_iter_t* iter)
{
stub_plan_priv_t* priv;
/* Req 2 AC 6: a real cancellation point at least once per invocation
* -- combinatorial generation has none of its own. */
pthread_testcancel();
if (iter == NULL) {
return NULL;
}
priv = (stub_plan_priv_t*)iter->private;
if (priv == NULL || priv->served) {
return NULL; /* Req 2 AC 4: stable exhaustion signal */
}
priv->served = 1;
return &priv->plan;
}
static void stub_plan_iter_fini(xce_alloc_plan_iter_t* iter)
{
if (iter == NULL) {
return;
}
free(iter->private);
iter->private = NULL;
}
/* ---- trivial no-op xce_host_iter_t reference implementation ---- */
typedef struct {
int served; /* 0 = not yet served, 1 = exhausted */
xce_sched_worker_t worker;
} stub_host_priv_t;
static int stub_host_iter_init(xce_host_iter_t* iter, xce_alloc_plan_t* plan,
var workers)
{
stub_host_priv_t* priv;
/* Stub: ignores the real plan/workers state -- the one shared real
* implementation (host_iter.c, Req 3) would filter `workers` by
* plan's select expression + slot availability here. */
(void)plan;
(void)workers;
if (iter == NULL) {
return -EINVAL;
}
priv = (stub_host_priv_t*)calloc(1, sizeof(*priv));
if (priv == NULL) {
return -ENOMEM;
}
priv->served = 0;
priv->worker.name = "stub-host";
iter->private = priv;
return 0;
}
static xce_sched_worker_t* stub_host_iter_next(xce_host_iter_t* iter)
{
stub_host_priv_t* priv;
/* Req 3 AC 6: a real cancellation point at least once per invocation
* -- scanning/filtering has none of its own. */
pthread_testcancel();
if (iter == NULL) {
return NULL;
}
priv = (stub_host_priv_t*)iter->private;
if (priv == NULL || priv->served) {
return NULL; /* Req 3 AC 3: exhausted for this instance */
}
priv->served = 1;
return &priv->worker;
}
static void stub_host_iter_fini(xce_host_iter_t* iter)
{
if (iter == NULL) {
return;
}
free(iter->private);
iter->private = NULL;
}
void setUp(void) { return; }
void tearDown(void) { return; }
/*
* Req 2 AC 1/2/4/5: the plan iterator is stack-allocated by the caller
* and initialized in place (not a heap-allocating factory -- see the
* header's own convention note), yields exactly one plan, then a stable
* NULL on every subsequent next() call, and fini() releases it cleanly.
*/
static void test_plan_iter_stub_produces_one_plan_then_stable_null(void)
{
xce_alloc_plan_iter_t iter_storage;
xce_alloc_plan_iter_t* iter = &iter_storage;
xce_alloc_plan_t* plan;
memset(iter, 0, sizeof(*iter));
iter->init = stub_plan_iter_init;
iter->next = stub_plan_iter_next;
iter->fini = stub_plan_iter_fini;
TEST_ASSERT_EQUAL_INT(0, iter->init(iter, NULL, NULL));
plan = iter->next(iter);
TEST_ASSERT_NOT_NULL(plan);
TEST_ASSERT_EQUAL_INT(1, plan->nbuckets);
TEST_ASSERT_NOT_NULL(plan->bucket_slots);
TEST_ASSERT_EQUAL_INT(1, plan->bucket_slots[0]);
/* Req 2 AC 4: end-of-iteration signal, stable across repeated calls. */
TEST_ASSERT_NULL(iter->next(iter));
TEST_ASSERT_NULL(iter->next(iter));
iter->fini(iter);
TEST_ASSERT_NULL(iter->private);
}
/*
* Req 3 AC 1/2/3/5: the host iterator is likewise stack-allocated and
* initialized in place from a plan, yields exactly one candidate host,
* then a stable NULL, and fini() is independent of the owning plan
* iterator's own lifecycle -- proved by finiing the host iterator first
* and then confirming plan_iter's own exhaustion/fini() sequence is
* completely unaffected.
*/
static void test_host_iter_stub_produces_one_worker_then_stable_null(void)
{
xce_alloc_plan_iter_t plan_iter_storage;
xce_alloc_plan_iter_t* plan_iter = &plan_iter_storage;
xce_alloc_plan_t* plan;
xce_host_iter_t host_iter_storage;
xce_host_iter_t* host_iter = &host_iter_storage;
xce_sched_worker_t* w;
memset(plan_iter, 0, sizeof(*plan_iter));
plan_iter->init = stub_plan_iter_init;
plan_iter->next = stub_plan_iter_next;
plan_iter->fini = stub_plan_iter_fini;
TEST_ASSERT_EQUAL_INT(0, plan_iter->init(plan_iter, NULL, NULL));
plan = plan_iter->next(plan_iter);
TEST_ASSERT_NOT_NULL(plan);
memset(host_iter, 0, sizeof(*host_iter));
host_iter->init = stub_host_iter_init;
host_iter->next = stub_host_iter_next;
host_iter->fini = stub_host_iter_fini;
/* `plan` is still valid here -- neither next() nor fini() has been
* called again on plan_iter since it was returned above, per the
* header's own "valid until the next next()/fini() call" contract. */
TEST_ASSERT_EQUAL_INT(0, host_iter->init(host_iter, plan, NULL));
w = host_iter->next(host_iter);
TEST_ASSERT_NOT_NULL(w);
TEST_ASSERT_EQUAL_STRING("stub-host", w->name);
/* Req 3 AC 3: exhausted for this instance's bucket search, stable. */
TEST_ASSERT_NULL(host_iter->next(host_iter));
TEST_ASSERT_NULL(host_iter->next(host_iter));
/* Req 3 AC 5: fini() on host_iter is independent of plan_iter's own
* lifecycle -- finiing it here must not disturb plan_iter, which is
* still fini()'d separately, afterward, below. */
host_iter->fini(host_iter);
TEST_ASSERT_NULL(host_iter->private);
/* plan_iter is unaffected by host_iter->fini() -- still stably
* exhausted (it already served its one plan above). Only now is its
* own storage (and the `plan` pointer into it) released. */
TEST_ASSERT_NULL(plan_iter->next(plan_iter));
plan_iter->fini(plan_iter);
TEST_ASSERT_NULL(plan_iter->private);
}
/*
* Req 3 AC 4 / Req 4 AC 2 composition shape: a fresh xce_host_iter_t
* instance is constructed from the SAME plan once per bucket, each a
* fully independent instance -- the exact plan -> bucket -> host_iter
* nesting requirements.md's Composition Algorithm pseudocode describes
* for allocate_primary(), exercised here end to end against the stub
* implementations (a 2-bucket plan, each bucket getting its own fresh
* host_iter instance that is init()'d and fini()'d independently).
*/
static void test_fresh_host_iter_per_bucket_composes_with_one_plan(void)
{
int bucket_slots[2] = { 2, 1 };
xce_alloc_plan_t two_bucket_plan
= { .nbuckets = 2, .bucket_slots = bucket_slots };
int b;
for (b = 0; b < two_bucket_plan.nbuckets; b++) {
xce_host_iter_t host_iter_storage;
xce_host_iter_t* host_iter = &host_iter_storage;
xce_sched_worker_t* w;
memset(host_iter, 0, sizeof(*host_iter));
host_iter->init = stub_host_iter_init;
host_iter->next = stub_host_iter_next;
host_iter->fini = stub_host_iter_fini;
/* A brand-new init() per bucket -- never one instance reused
* across the loop (Req 3's "fresh per bucket" requirement). */
TEST_ASSERT_EQUAL_INT(
0, host_iter->init(host_iter, &two_bucket_plan, NULL));
w = host_iter->next(host_iter);
TEST_ASSERT_NOT_NULL(w);
host_iter->fini(host_iter);
TEST_ASSERT_NULL(host_iter->private);
}
}
int main(void)
{
UNITY_BEGIN();
RUN_TEST(test_plan_iter_stub_produces_one_plan_then_stable_null);
RUN_TEST(test_host_iter_stub_produces_one_worker_then_stable_null);
RUN_TEST(test_fresh_host_iter_per_bucket_composes_with_one_plan);
return UNITY_END();
}