lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(scheduler-sim-live-allocation): close host-scoped limit fan-out seam gap (task 4.3)

Browse source 
Fully closes bug-1829's own tracked follow-up from task 4.1: retypes
seqjob_allocator.c/parjob_allocator.c's init_fn AND runtime hooks
(prepare_allocation_hook, job_alloc_fn) to read the plugin's own
honest plugin->core field (task 1.2) instead of deriving an
xce_scheduler_t* from plugin->sched -- mirroring task 2.3's limit.c
migration exactly. The gap was broader than initially scoped: not
just init_fn, but every runtime hook independently derived sched from
plugin->sched on each allocation attempt.

Adds seqjob_allocator_new_plugin()/parjob_allocator_new_plugin()
factories (plugin.h, mirroring limit_new_plugin()), letting
sim_alloc.c's bootstrap functions register both plugins DIRECTLY
against &(sim->core) via the existing plugin_register_core() -- the
same call sim_bootstrap_picker()/sim_bootstrap_limit_plugin() already
use. Task 4.1's throwaway xce_scheduler_t wrapper is gone from the
plugin-registry/fan-out path entirely; plugin_prepare_allocation()'s
fan-out during a live seqjob/parjob dispatch now genuinely reaches
sim->core.plugins, the same registry the limit plugin is registered
in.

One narrow, necessary exception remains, independently verified
confined: my_alloc() still needs a small xce_scheduler_t wrapper
(only .core.resmgr aliased to sim->core) to satisfy sched_make_allocs()
(scheduler.c, out of this task's boundary, unmodified) -- set on
plugin->sched only AFTER plugin_register_core() has run, never read by
the plugin registry or any prepare_allocation_hook fan-out.

TDD: new test_sim_alloc_limit_fanout.c drives a real live dispatch
(sched_job_picker()/sched_allocator() thread pair) with a host-scoped
limit registered directly against sim->core, proving a real usage
instance is created -- isolated from both reconciliation and
global-limit matching. RED: dispatch succeeded but the usage-instance
iterator was empty. GREEN: 82/82, zero regressions.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RLSNJhNoULvVcUhXEFafQC
This commit is contained in:
Rongsong Shen 2026-09-19 07:38:34 +00:00
parent 915782fc66
commit f6a2862110
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
12 changed files with 778 additions and 193 deletions
Download patch file Download diff file Expand all files Collapse all files

4
.kiro/specs/scheduler-sim-live-allocation/design.md
View file

@ -142,9 +142,9 @@ graph TB
### Known Seam Gap: Host-Scoped Limit Fan-Out During Live Seqjob/Parjob Dispatch
**Confirmed during task 4.1's review, mechanism-traced end to end — not resolved by this task, explicitly owned by tasks 4.2/5.2.** `register_seqjob_allocator()`/`register_parjob_allocator()`'s own `init_fn` (a pre-existing, real, out-of-boundary dependency this feature's tasks 1.2/2.3 never touched, since those covered `job_election_hook`/`prepare_allocation_hook`/`commit_hook`, not `init_fn`) still requires a real `xce_scheduler_t*`, which simulation mode does not have. `sim_bootstrap_seqjob_allocator()`/`sim_bootstrap_parjob_allocator()` satisfy this with a throwaway `xce_scheduler_t` wrapper, calling `plugin_register()` (not `plugin_register_core()`) against it — which means the seqjob/parjob plugin instances' own `xce_sched_plugin_t.core` field (task 1.2) ends up honestly pointing at the **wrapper's** core, not `sim->core`. `seqjob_allocator.c`/`parjob_allocator.c`'s `my_alloc()` then derives its own `sched`/`core` reference from `plugin->sched`/`plugin->core` (its OWN plugin instance, not the `core` parameter threaded down from `sched_make_allocation()`), so `sched_try_alloc_on_host_()` → `plugin_prepare_allocation()`'s fan-out during a REAL sim-mode seqjob/parjob dispatch attempt iterates the **wrapper's** plugin registry, not `sim->core.plugins` — meaning any plugin registered only against `sim->core` directly (the limit plugin, via `sim_bootstrap_limit_plugin()`) is never reached for host-scoped matching during a live sim-mode dispatch.
**Confirmed during task 4.1's review, mechanism-traced end to end — not resolved by that task, closed by task 4.3 (see task 4.3's own commit/buglog-1829 entry for the actual fix: `plugin->core`-based retyping of `seqjob_allocator.c`/`parjob_allocator.c`'s `init_fn` and runtime hooks, plus direct `plugin_register_core()` registration against `sim->core`, eliminating the wrapper-registry entirely from the fan-out path).** `register_seqjob_allocator()`/`register_parjob_allocator()`'s own `init_fn` (a pre-existing, real, out-of-boundary dependency this feature's tasks 1.2/2.3 never touched, since those covered `job_election_hook`/`prepare_allocation_hook`/`commit_hook`, not `init_fn`) still requires a real `xce_scheduler_t*`, which simulation mode does not have. `sim_bootstrap_seqjob_allocator()`/`sim_bootstrap_parjob_allocator()` satisfy this with a throwaway `xce_scheduler_t` wrapper, calling `plugin_register()` (not `plugin_register_core()`) against it — which means the seqjob/parjob plugin instances' own `xce_sched_plugin_t.core` field (task 1.2) ends up honestly pointing at the **wrapper's** core, not `sim->core`. `seqjob_allocator.c`/`parjob_allocator.c`'s `my_alloc()` then derives its own `sched`/`core` reference from `plugin->sched`/`plugin->core` (its OWN plugin instance, not the `core` parameter threaded down from `sched_make_allocation()`), so `sched_try_alloc_on_host_()` → `plugin_prepare_allocation()`'s fan-out during a REAL sim-mode seqjob/parjob dispatch attempt iterates the **wrapper's** plugin registry, not `sim->core.plugins` — meaning any plugin registered only against `sim->core` directly (the limit plugin, via `sim_bootstrap_limit_plugin()`) is never reached for host-scoped matching during a live sim-mode dispatch.
This is a genuine functional gap (not a harmless unused-registry artifact — the wrapper's registry is on the real, executed code path), confined specifically to host-scoped limit matching during a live seqjob/parjob allocation attempt in simulation mode. It does not affect: global-limit matching (reached via `plugin_job_election()`, called directly against `sim->core` before allocation, per the "Plugin Hook Dispatchers"/"Limit Plugin Core-Safe Matching" components above); normal mode (unaffected, its own `xce_scheduler_t` is the real one throughout); or the jbuf rendezvous, decision-commit, or accounting-state correctness of the dispatch itself (the allocation DECISION and its resource charge are correct; only a specific plugin hook's visibility during that decision's host search is narrowed). Tasks 4.2 and 5.2 own closing or re-scoping this gap as part of their own host-scoped-limit-under-live-dispatch work.
This is a genuine functional gap (not a harmless unused-registry artifact — the wrapper's registry is on the real, executed code path), confined specifically to host-scoped limit matching during a live seqjob/parjob allocation attempt in simulation mode. It does not affect: global-limit matching (reached via `plugin_job_election()`, called directly against `sim->core` before allocation, per the "Plugin Hook Dispatchers"/"Limit Plugin Core-Safe Matching" components above); normal mode (unaffected, its own `xce_scheduler_t` is the real one throughout); or the jbuf rendezvous, decision-commit, or accounting-state correctness of the dispatch itself (the allocation DECISION and its resource charge are correct; only a specific plugin hook's visibility during that decision's host search is narrowed). **Closed by task 4.3** (see above) — task 5.2 verifies the resulting end-to-end behavior as part of its own acceptance test, rather than closing the gap itself.
## System Flows

2
.kiro/specs/scheduler-sim-live-allocation/tasks.md
View file

@ -85,7 +85,7 @@
- _Requirements: 3.4, 3.5_
- _Boundary: Sim-Mode Reconciliation Narrowing_
- [ ] 4.3 Close the host-scoped limit fan-out seam gap for live simulation-mode dispatch
- [x] 4.3 Close the host-scoped limit fan-out seam gap for live simulation-mode dispatch
- Found during task 4.1's review, mechanism-traced and documented in design.md's "Known Seam Gap: Host-Scoped Limit Fan-Out During Live Seqjob/Parjob Dispatch": `sim_bootstrap_seqjob_allocator()`/`sim_bootstrap_parjob_allocator()`'s throwaway-wrapper registration means the seqjob/parjob allocator plugins' prepare-allocation-hook fan-out iterates the wrapper's own separate plugin registry, never reaching any plugin (the limit plugin included) registered only against simulation mode's real core — so a host-scoped limit's usage instance can never be created via a live seqjob/parjob dispatch attempt today
- Fix this with an honest, narrow seam (mirroring the `unsatisfiable_cache` callback pattern from 1.1, or an equally narrow alternative) satisfying seqjob/parjob's own pre-existing `init_fn` requirement for a real `xce_scheduler_t*` without substituting a separate plugin registry for the fan-out itself — do not repeat the throwaway-wrapper-with-a-separate-registry shape that created this gap
- TDD: a test proving a host-scoped limit registered directly against simulation mode's real core is matched and its usage instance created during a genuine seqjob or parjob dispatch attempt, independent of recorded-decision reconciliation and independent of global-limit matching

23
.wolf/buglog.json
View file

@ -22515,14 +22515,17 @@
"timestamp": "2026-09-19T04:00:00Z",
"error_message": "AddressSanitizer: SEGV (NULL pointer read) in sched_make_seqjob_allocation (src/scheduler/src/scheduler.c:59, `plugin->job_alloc_fn` where `plugin = core->seqjob_allocator` is NULL), called from sched_make_primary_alloc_ -> sched_make_primary_allocation -> sched_make_allocation -> sched_allocator (sched_election.c) -> the real pthread spawned by sched_sim_driver_run() (sim_loop.c). Discovered while writing the mandated TDD test for scheduler-sim-live-allocation task 4.1 (its own Observable: 'a simulation-mode session ... produces a genuine dispatch or pend decision for a job based on its actual resource availability, instead of unconditionally pending it') -- the crash is 100% reproducible for ANY job reaching primary resource allocation in simulation mode, regardless of resreq shape or dispatch/pend outcome (sched_make_parjob_allocation has the identical NULL-core->parjob_allocator crash for maxTasks>1 jobs).",
"root_cause": "core->seqjob_allocator/core->parjob_allocator (xce_sched_core_t fields, core.h) are never populated for the simulation driver, anywhere. Normal mode populates them via sched_init()'s sched_register_plugins() -> register_seqjob_allocator()/register_parjob_allocator() (plugins/seqjob_allocator.c, plugins/parjob_allocator.c), whose init_fn does `((xce_scheduler_t*)plugin->sched)->core.seqjob_allocator = plugin` -- REQUIRING a genuine, live xce_scheduler_t*. sched_sim_driver_start() (sim.c) registers no plugin at all (sim.h's own documented 'Known scope boundary'), and xce_sched_sim_driver_t deliberately has no xce_scheduler_t*/xce_scheduler_t field of its own (Req 5 AC 17, simulation isolation). This is NOT limited to the init_fn: every one of these two plugins' RUNTIME hooks (seqjob_prepare_allocation()/my_alloc(), parjob_prepare_allocation()/my_alloc(), and several internal static helpers taking xce_scheduler_t* sched as a parameter) ALSO read `plugin->sched->core.*` (resmgr for the 'slots'/rusage lookups, workers for host_iter, and sched_make_allocs()/sched_try_alloc_on_host_() in scheduler.c/resource.c which ALSO take xce_scheduler_t*/need `sched->core.resmgr`) on every single allocation attempt, not just at registration. This gap was already discovered and explicitly deferred by scheduler-sim-live-allocation task 3.1 (see test_sched_election_sim_fixture.c's own header comment: 'a pre-existing, tracked, OUT-OF-SCOPE gap ... NOT one of task 3.1's five permitted changes -- fixing it would be scope creep into a different, not-yet-assigned task') -- task 3.1's own fixture never reached this crash only because IT registers seqjob_allocator directly against a real, throwaway xce_scheduler_t wrapper whose OWN .core IS the core under test (single-core fixture), a shape simulation mode itself can never have (its core lives inside xce_sched_sim_driver_t, never xce_scheduler_t). Task 4.1 is the first task whose OWN production change (spawning the real sched_job_picker()/sched_allocator() thread pair against simulation mode's own core, replacing the old always-PEND sim_picker_thread_fn() placeholder that never called sched_make_allocation() at all) makes this code path reachable in simulation mode for the first time ever.",
"fix": "PARTIALLY FIXED (2026-09-19, scheduler-sim-live-allocation task 4.1, within that task's own file boundary -- a full fix is tracked as NOT YET DONE, see below). New file src/scheduler/src/sim_alloc.c (added to scheduler_srcs and sched_sim_isolation_lib, meson.build) provides sim_bootstrap_seqjob_allocator()/sim_bootstrap_parjob_allocator(), called from sched_sim_driver_run() (sim_loop.c) right after the existing sim_bootstrap_picker()/sim_bootstrap_limit_plugin() calls (same ordering constraint, same 'extern declared where consumed' idiom). Each builds a long-lived (heap-allocated, never freed -- same accepted-leak convention this file family already uses for its own never-joined threads), throwaway xce_scheduler_t wrapper, points its .core.resmgr/.core.workers fields (both `var`, core.h -- plain assignable rtti handles, not embedded values) at simulation mode's OWN resmgr/workers objects BEFORE calling the real, unmodified register_seqjob_allocator()/register_parjob_allocator() against it, then copies the resulting plugin pointer onto sim->core.seqjob_allocator/parjob_allocator. Because resmgr/workers are shared BY REFERENCE, any resource/worker registered later against the real sim->core (via the genuine event-ingest path) is immediately visible through the wrapper's aliased fields too, so seqjob_prepare_allocation()/my_alloc() see simulation mode's REAL, live resource/worker state -- not an empty, disconnected copy. Verified live: test_sim_live_allocation.c (new dedicated fixture, real scheduler_srcs link, real WORKERS_PUSH/RESOURCES_PUSH/JOBS_PUSH replay events, real sched_sim_driver_run() call, static-storage xce_sched_sim_driver_t) drives a job with a satisfiable resource want through the full real chain and observes a genuine `[sim] DISPATCH jobid=...` -- previously impossible (would have crashed) before this fix, PEND-only after reverting to sim_picker_thread_fn() at HEAD (confirmed RED). Full scheduler suite: 81/81, zero regressions (25 pre-existing white-box sim tests that #include sim_loop.c directly needed matching link-satisfying no-op/PEND-only stubs for the two new bootstrap functions and for the sched_job_picker()/sched_allocator() symbols sim_loop.c now calls directly instead of its own removed sim_picker_thread_fn() -- none of those 25 files test genuine resource-based allocation, so behavior-preserving stubs keep their existing assertions valid unchanged). NOT YET FIXED (tracked follow-up, explicitly out of task 4.1's file boundary): each wrapper's own core.plugins registry is separate from sim->core.plugins, so a plugin registered only against sim->core (e.g. the limit plugin via sim_bootstrap_limit_plugin()) is never reached by plugin_prepare_allocation()'s fan-out when it runs inside seqjob/parjob's own call chain rooted at the wrapper's core -- limit-plugin host-scoped matching under a REAL simulation-mode dispatch is not yet wired end-to-end (task 4.2/5.2's own territory, which specifically covers limit-usage parity under live sim-mode dispatch -- re-verify this exact scenario when implementing those). The FULL, root-cause fix -- retyping seqjob_allocator.c's/parjob_allocator.c's runtime hooks and internal helpers from `plugin->sched`/`xce_scheduler_t* sched` parameters to the already-available `plugin->core`/`xce_sched_core_t*` (task 1.2's own established pattern, mirroring task 2.3's limit.c migration) -- was NOT attempted: those hooks also call sched_make_allocs() (scheduler.c) and sched_try_alloc_on_host_() (resource.c) with the same xce_scheduler_t*, and scheduler.c/resource.c were explicitly out of task 4.1's file boundary (already migrated/committed by earlier tasks in this spec). A future task should do this properly rather than layering more wrapper-aliasing tricks.",
"fix": "FULLY FIXED (2026-09-19, scheduler-sim-live-allocation task 4.3, closing this bug's own tracked follow-up). Root cause: seqjob_allocator_init()/parjob_allocator_init() (plugins/seqjob_allocator.c, plugins/parjob_allocator.c) and every runtime hook these plugins register (seqjob_prepare_allocation()/my_alloc(), parjob_prepare_allocation()/my_alloc()/parjob_try_plan_()) now read the plugin's own honestly-typed `core` field (task 1.2's xce_sched_plugin_t.core) instead of deriving an xce_scheduler_t* from plugin->sched -- mirroring task 2.3's plugins/limit.c migration exactly. Two new factory functions, seqjob_allocator_new_plugin()/parjob_allocator_new_plugin() (plugin.h, mirroring limit_new_plugin()/builtin_fcfs_new_plugin()), let sim_alloc.c's sim_bootstrap_seqjob_allocator()/sim_bootstrap_parjob_allocator() register the allocator plugin DIRECTLY against &(sim->core) via the existing plugin_register_core() -- the same call sim_bootstrap_picker()/sim_bootstrap_limit_plugin() already use -- so plugin->core honestly points at sim->core and every later plugin_prepare_allocation() fan-out (my_alloc() -> sched_try_alloc_on_host_() -> plugin_prepare_allocation(), resource.c/plugin.c) reaches the SAME core->plugins registry the limit plugin (and picker) are registered in. Task 4.1's throwaway xce_scheduler_t wrapper is gone from the plugin-registry/fan-out path entirely -- no separate registry stands in for it anymore. One narrow, necessary exception remains: my_alloc() still keeps a small, long-lived, heap-allocated xce_scheduler_t wrapper (never freed, same accepted-leak convention) whose ONLY purpose is satisfying sched_make_allocs()'s (scheduler.c, out of this task's file boundary, unmodified) own real xce_scheduler_t* requirement for the FINAL allocation record of an already-decided winning host -- that function's body only ever reads sched->core.resmgr/lock/nextid, so the wrapper's .core.resmgr is ALIASED (a `var`, not copied) to sim->core's own; it is set on plugin->sched ONLY AFTER plugin_register_core() has already run (init_fn no longer reads plugin->sched at all) and is never used for the plugin registry or any prepare_allocation_hook fan-out, so it cannot reintroduce the separate-registry gap this fix closes. Verified live: new test_sim_alloc_limit_fanout.c (real scheduler_srcs link, bare xce_sched_sim_driver_t fixture, real sched_job_picker()/sched_allocator() thread pair) registers a host-scoped limit directly against sim->core (sim_bootstrap_limit_plugin()) alongside the seqjob allocator plugin (sim_bootstrap_seqjob_allocator()), drives a genuine live dispatch to a real DISPATCH decision (JOB_STAT_RUN), and confirms the limit's usage instance is created (limit_get_usage_instances()) -- RED before this fix (job dispatches correctly but zero usage instances; the fan-out never reached the limit plugin), GREEN after. Full scheduler suite: 82/82, zero regressions -- three white-box test files that construct an xce_sched_plugin_t directly without going through plugin_register()/plugin_register_core() (test_seqjob_allocator.c, test_seqjob_rewrap_equivalence.c, test_parjob_combinatorial.c) needed their fixture plugin instances' own `.core` field set alongside the pre-existing `.sched` field, plus parjob_try_plan_()'s one direct white-box call site (test_parjob_combinatorial.c) updated from &sched to &(sched.core) for its now-honest xce_sched_core_t* signature -- the same established ripple-fix pattern this feature's own tasks.md Implementation Notes already document for prior signature migrations.",
"files": [
"src/scheduler/src/sim_alloc.c (new file)",
"src/scheduler/src/sim_loop.c (sched_sim_driver_run() call sites)",
"src/scheduler/meson.build (scheduler_srcs, sched_sim_isolation_lib)",
"src/scheduler/testsuite/test_sim_live_allocation.c (new file, RED/GREEN proof)",
"src/scheduler/src/plugins/seqjob_allocator.c (root cause, not modified)",
"src/scheduler/src/plugins/parjob_allocator.c (root cause, not modified)"
"src/scheduler/include/plugin.h (seqjob_allocator_new_plugin()/parjob_allocator_new_plugin() factory declarations, task 4.3)",
"src/scheduler/src/plugins/seqjob_allocator.c (task 4.3: init_fn + prepare_allocation_hook + job_alloc_fn retyped to read plugin->core; new seqjob_allocator_new_plugin() factory -- genuinely modified, no longer just the root-cause file)",
"src/scheduler/src/plugins/parjob_allocator.c (task 4.3: same retyping + new parjob_allocator_new_plugin() factory)",
"src/scheduler/src/sim_alloc.c (task 4.1: new file, initial bootstrap via throwaway wrapper; task 4.3: rewritten to register directly against sim->core via plugin_register_core(), no wrapper in the registration/fan-out path)",
"src/scheduler/src/sim_loop.c (task 4.1: sched_sim_driver_run() call sites -- unmodified by task 4.3)",
"src/scheduler/meson.build (task 4.1: scheduler_srcs, sched_sim_isolation_lib -- unmodified by task 4.3)",
"src/scheduler/testsuite/test_sim_live_allocation.c (task 4.1: new file, live-dispatch RED/GREEN proof)",
"src/scheduler/testsuite/test_sim_alloc_limit_fanout.c (task 4.3: new file, host-scoped-limit-fan-out RED/GREEN proof)",
"src/scheduler/testsuite/test_seqjob_allocator.c, test_seqjob_rewrap_equivalence.c, test_parjob_combinatorial.c (task 4.3: white-box fixture ripple -- plugin instances built directly in these tests needed their own .core field set alongside .sched)"
],
"tags": [
"scheduler",
@ -22536,13 +22539,15 @@
"scheduler-sim-live-allocation",
"task-4.1",
"partially-fixed",
"follow-up-tracked"
"follow-up-tracked",
"task-4.3",
"fully-fixed"
],
"related_bugs": [
"bug-1828"
],
"occurrences": 1,
"last_seen": "2026-09-19T04:00:00Z"
"last_seen": "2026-09-19T08:00:00Z"
}
]
}

26
src/scheduler/include/plugin.h
View file

@ -188,6 +188,32 @@ extern var limit_get_usage_instances(xce_sched_core_t* core);
extern int register_seqjob_allocator(void*);
extern int register_parjob_allocator(void*);
/*
* seqjob_allocator_new_plugin/parjob_allocator_new_plugin -
* scheduler-sim-live-allocation spec, task 4.3: factory half of
* register_seqjob_allocator()'s/register_parjob_allocator()'s own
* template-build step, extracted so a caller with no xce_scheduler_t*
* (the simulation driver, via plugin_register_core() -- see sim_alloc.c)
* can obtain a freshly calloc'd, fully populated xce_sched_plugin_t for
* the seqjob/parjob allocator plugin -- same hooks/init_fn/fini_fn
* template register_seqjob_allocator()/register_parjob_allocator()
* themselves use, just not yet registered anywhere. Mirrors
* limit_new_plugin()'s (plugins/limit.c) identical extraction exactly
* (scheduler-sim-diagnostics spec, task 6; bug-1825), closing bug-1829's
* tracked follow-up: seqjob_allocator_init()/parjob_allocator_init() (and
* the prepare_allocation_hook/job_alloc_fn hooks these plugins register)
* now read the honestly-typed plugin->core field (task 1.2) instead of
* plugin->sched, so a plugin built here and registered via
* plugin_register_core(&(sim->core), plugin) directly reaches the SAME
* core->plugins registry every other sim-mode plugin (the picker, the
* limit plugin) is registered in -- no separate wrapper registry, unlike
* task 4.1's now-superseded fix. Returns NULL on allocation failure.
* Caller owns the returned plugin (frees it only on a subsequent
* registration failure -- once registered, the plugin registry owns it).
*/
extern xce_sched_plugin_t* seqjob_allocator_new_plugin(void);
extern xce_sched_plugin_t* parjob_allocator_new_plugin(void);
/* ---------------------------------------------------------------------
* Job picker plugin interface (xce_job_picker_iface_t)
*

97
src/scheduler/src/plugins/parjob_allocator.c
View file

@ -16,11 +16,16 @@
#include "xlist.h"
#include "xvec.h"
/*
* scheduler-sim-live-allocation spec, task 4.3 (bug-1829's tracked
* follow-up): reads the honestly-typed plugin->core field (task 1.2)
* instead of plugin->sched -- see seqjob_allocator.c's identical
* seqjob_allocator_init() for the full rationale (plugin->core is set by
* BOTH plugin_register()/plugin_register_core() BEFORE init_fn runs).
*/
static int parjob_allocator_init(xce_sched_plugin_t* plugin)
{
xce_scheduler_t* sched = (xce_scheduler_t*)plugin->sched;
sched->core.parjob_allocator = plugin;
plugin->core->parjob_allocator = plugin;
return 1;
}
@ -78,13 +83,13 @@ static int skip_bad_worker_(xce_sched_worker_t* w)
static int parjob_prepare_allocation(void* arg, xce_sched_ctx_t* ctx)
{
xce_sched_plugin_t* plugin = (xce_sched_plugin_t*)arg;
xce_scheduler_t* sched = (xce_scheduler_t*)plugin->sched;
xce_sched_core_t* core = plugin->core;
xce_sched_obj_t* obj = NULL;
xce_resource_t* r = NULL;
xce_primary_resreq_t* primary = NULL;
int rc = 0;
if ((plugin == NULL) || (ctx == NULL) || (sched == NULL)) {
if ((plugin == NULL) || (ctx == NULL) || (core == NULL)) {
return -EINVAL;
}
@ -113,19 +118,19 @@ static int parjob_prepare_allocation(void* arg, xce_sched_ctx_t* ctx)
ctx->nhosts = 1;
ctx->nslots = primary->maxTasks;
calc_wanted_global_resvals(&(ctx->alloc_req.shared), sched->core.resmgr,
calc_wanted_global_resvals(&(ctx->alloc_req.shared), core->resmgr,
&(primary->rusages), PER_TASK, ctx->nhosts,
ctx->nslots);
} else {
calc_wanted_host_resvals(&(ctx->alloc_req.host), sched->core.resmgr,
calc_wanted_host_resvals(&(ctx->alloc_req.host), core->resmgr,
&(primary->rusages), PER_TASK, ctx->nhosts,
ctx->nslots);
}
/* add slots into requested resources */
lock(sched->core.resmgr, RTTI_LOCK_READ);
rc = key_find(sched->core.resmgr, (void*)"slots", (void**)&r);
unlock(sched->core.resmgr);
lock(core->resmgr, RTTI_LOCK_READ);
rc = key_find(core->resmgr, (void*)"slots", (void**)&r);
unlock(core->resmgr);
if (rc < 0) {
return rc;
@ -433,7 +438,7 @@ static void parjob_rollback_claims_(xce_sched_ctx_t* ctx,
* Already-claimed buckets (including the one that just failed to
* snapshot) are rolled back before returning.
*/
static int parjob_try_plan_(xce_scheduler_t* sched, xce_sched_ctx_t* ctx,
static int parjob_try_plan_(xce_sched_core_t* core, xce_sched_ctx_t* ctx,
xce_alloc_plan_t* plan, parjob_claim_t* claimed,
int* nclaimed_out)
{
@ -478,7 +483,7 @@ static int parjob_try_plan_(xce_scheduler_t* sched, xce_sched_ctx_t* ctx,
host_iter->next = sched_host_iter_next;
host_iter->fini = sched_host_iter_fini;
if (host_iter->init(host_iter, plan, sched->core.workers) == 0) {
if (host_iter->init(host_iter, plan, core->workers) == 0) {
while ((cand = host_iter->next(host_iter)) != NULL) {
if (parjob_is_claimed_(claimed, nclaimed, cand)) {
continue;
@ -487,14 +492,16 @@ static int parjob_try_plan_(xce_scheduler_t* sched, xce_sched_ctx_t* ctx,
ctx->nslots = plan->bucket_slots[b];
ctx->nhosts = 1;
/* scheduler-sim-live-allocation spec, task 2.1 (Core-Safe
* Allocation Chain): sched_try_alloc_on_host_() is now
* core-typed -- minimal &(sched->core) wrap; `sched`
* itself keeps its existing xce_scheduler_t* type here
* (derived from plugin->sched via my_alloc(), the
* plugin's own separate, untouched driver-reference
* field), unchanged by this task. */
if (sched_try_alloc_on_host_(cand, &(sched->core), ctx) > 0) {
/* scheduler-sim-live-allocation spec, task 4.3:
* sched_try_alloc_on_host_() takes core directly (already
* core-typed since task 2.1) -- this is the exact call
* whose own internal plugin_prepare_allocation() fan-out
* (resource.c) must reach the SAME core->plugins registry
* every other plugin (e.g. the limit plugin) is registered
* against, so it reads the caller's honest `core`
* parameter (my_alloc()'s own plugin->core), not a
* throwaway wrapper derived from plugin->sched. */
if (sched_try_alloc_on_host_(cand, core, ctx) > 0) {
int rc = parjob_snapshot_host_req_(
&(claimed[nclaimed].host_req), &(ctx->alloc_req.host));
@ -538,6 +545,10 @@ static int parjob_try_plan_(xce_scheduler_t* sched, xce_sched_ctx_t* ctx,
static int my_alloc(xce_sched_plugin_t* plugin, xce_sched_ctx_t* ctx)
{
xce_sched_core_t* core = plugin->core;
/* Kept alongside `core` above ONLY for the sched_make_allocs() calls
* below (scheduler.c, out of this task's boundary) -- see
* seqjob_allocator.c's identical my_alloc() for the full rationale. */
xce_scheduler_t* sched = (xce_scheduler_t*)plugin->sched;
xce_primary_resreq_t* primary = ctx->primary;
xce_pendreason_t* reason = ctx->reason;
@ -547,7 +558,7 @@ static int my_alloc(xce_sched_plugin_t* plugin, xce_sched_ctx_t* ctx)
xce_alloc_plan_t* plan;
int rc;
if ((sched == NULL) || (primary == NULL)) {
if ((core == NULL) || (sched == NULL) || (primary == NULL)) {
return -EINVAL;
}
@ -594,11 +605,11 @@ static int my_alloc(xce_sched_plugin_t* plugin, xce_sched_ctx_t* ctx)
return rc;
}
/* scheduler-sim-live-allocation task 1.3: sched_check_shared_satisfy()
* is now core-typed -- minimal &(sched->core) wrap; `sched` itself
* keeps its existing xce_scheduler_t* type here (derived from
* plugin->sched), unchanged by this task. */
rc = sched_check_shared_satisfy(&(sched->core), uncomit,
/* scheduler-sim-live-allocation task 4.3: sched_check_shared_satisfy()
* takes core directly (already core-typed since task 1.3) -- read
* from the plugin's own honest `core` field, not derived from
* plugin->sched. */
rc = sched_check_shared_satisfy(core, uncomit,
&(ctx->alloc_req.shared), &(ctx->unmeet));
if (rc < 0) {
return rc;
@ -610,7 +621,7 @@ static int my_alloc(xce_sched_plugin_t* plugin, xce_sched_ctx_t* ctx)
for (i = 0; i < ctx->alloc_req.shared.count; i++) {
xce_resource_t* r = NULL;
rc = vget(sched->core.resmgr, ((int*)ctx->unmeet.idxs.data)[i],
rc = vget(core->resmgr, ((int*)ctx->unmeet.idxs.data)[i],
(void**)&r);
if (rc <= 0) {
continue;
@ -649,7 +660,7 @@ static int my_alloc(xce_sched_plugin_t* plugin, xce_sched_ctx_t* ctx)
int nclaimed = 0;
int plan_rc;
plan_rc = parjob_try_plan_(sched, ctx, plan, claimed, &nclaimed);
plan_rc = parjob_try_plan_(core, ctx, plan, claimed, &nclaimed);
if (plan_rc < 0) {
/* genuine infrastructure error -- parjob_try_plan_() already
* rolled back whatever it partially claimed before returning
@ -726,9 +737,16 @@ static int my_alloc(xce_sched_plugin_t* plugin, xce_sched_ctx_t* ctx)
return 0;
}
int register_parjob_allocator(void* sched)
/*
* parjob_allocator_new_plugin - scheduler-sim-live-allocation spec, task
* 4.3: factory half of register_parjob_allocator()'s own template-build
* step -- see seqjob_allocator.c's identical seqjob_allocator_new_plugin()
* and plugin.h's own doc comment on this declaration for the full
* rationale.
*/
xce_sched_plugin_t* parjob_allocator_new_plugin(void)
{
static xce_sched_plugin_t parjob_allocator_plugin = {
static xce_sched_plugin_t parjob_allocator_plugin_template = {
.name = "xce_sched_parjob_allocator",
.hooks
= { .config_hook = NULL,
@ -744,14 +762,25 @@ int register_parjob_allocator(void* sched)
.data = NULL
};
xce_sched_plugin_t* plugin = NULL;
xce_sched_plugin_t* plugin
= (xce_sched_plugin_t*)calloc(1, sizeof(*plugin));
if (plugin == NULL) {
return NULL;
}
memcpy(plugin, &parjob_allocator_plugin_template, sizeof(*plugin));
return plugin;
}
int register_parjob_allocator(void* sched)
{
xce_sched_plugin_t* plugin = parjob_allocator_new_plugin();
plugin = (xce_sched_plugin_t*)calloc(1, sizeof(*plugin));
if (plugin == NULL) {
return -ENOMEM;
}
memcpy(plugin, &parjob_allocator_plugin, sizeof(*plugin));
return plugin_register(sched, plugin);
}

104
src/scheduler/src/plugins/seqjob_allocator.c
View file

@ -16,11 +16,21 @@
#include "xcealloc.h"
#include "xvec.h"
/*
* scheduler-sim-live-allocation spec, task 4.3 (bug-1829's tracked
* follow-up): reads the honestly-typed plugin->core field (task 1.2)
* instead of plugin->sched -- plugin->core is set by BOTH plugin_register()
* (`plugin->core = &(sched->core);`) and plugin_register_core()
* (`plugin->core = core;`) BEFORE init_fn ever runs (plugin.c), so this
* write always lands on the same core the caller registered against --
* &(sched->core) in normal mode (identical to today's behavior), or a
* simulation driver's own &(sim->core) directly (sim_alloc.c), with no
* throwaway xce_scheduler_t wrapper standing in for it. Mirrors task 2.3's
* limit.c migration exactly.
*/
static int seqjob_allocator_init(xce_sched_plugin_t* plugin)
{
xce_scheduler_t* sched = (xce_scheduler_t*)plugin->sched;
sched->core.seqjob_allocator = plugin;
plugin->core->seqjob_allocator = plugin;
return 1;
}
@ -35,14 +45,14 @@ static void seqjob_allocator_fini(xce_sched_plugin_t* plugin)
static int seqjob_prepare_allocation(void* arg, xce_sched_ctx_t* ctx)
{
xce_sched_plugin_t* plugin = (xce_sched_plugin_t*)arg;
xce_scheduler_t* sched = (xce_scheduler_t*)plugin->sched;
xce_sched_core_t* core = plugin->core;
xce_sched_worker_t* worker = NULL;
xce_sched_obj_t* obj = NULL;
xce_resource_t* r = NULL;
xce_primary_resreq_t* primary = NULL;
int rc = 0;
if ((plugin == NULL) || (ctx == NULL) || (sched == NULL)) {
if ((plugin == NULL) || (ctx == NULL) || (core == NULL)) {
return -EINVAL;
}
@ -82,17 +92,17 @@ static int seqjob_prepare_allocation(void* arg, xce_sched_ctx_t* ctx)
if (ctx->worker == NULL) {
ctx->nhosts = 1;
ctx->nslots = 1;
calc_wanted_global_resvals(&(ctx->alloc_req.shared), sched->core.resmgr,
calc_wanted_global_resvals(&(ctx->alloc_req.shared), core->resmgr,
&(primary->rusages), PER_TASK, 1, 1);
} else {
calc_wanted_host_resvals(&(ctx->alloc_req.host), sched->core.resmgr,
calc_wanted_host_resvals(&(ctx->alloc_req.host), core->resmgr,
&(primary->rusages), PER_TASK, 1, 1);
}
/* add slots into requested resources */
lock(sched->core.resmgr, RTTI_LOCK_READ);
rc = key_find(sched->core.resmgr, (void*)"slots", (void**)&r);
unlock(sched->core.resmgr);
lock(core->resmgr, RTTI_LOCK_READ);
rc = key_find(core->resmgr, (void*)"slots", (void**)&r);
unlock(core->resmgr);
if (rc < 0) {
return rc;
@ -203,6 +213,16 @@ static void seqjob_plan_iter_fini(xce_alloc_plan_iter_t* iter)
static int my_alloc(xce_sched_plugin_t* plugin, xce_sched_ctx_t* ctx)
{
xce_sched_core_t* core = plugin->core;
/* Kept alongside `core` above ONLY for sched_make_allocs() below
* (scheduler.c, out of this task's boundary) -- that function's own
* signature still needs a genuine xce_scheduler_t*, but its body only
* ever reads sched->core.resmgr/lock/nextid (scheduler.c's own
* sched_make_allocs()/sched_get_id() bodies), so a caller with no real
* xce_scheduler_t* of its own (simulation mode) can safely hand it a
* narrow, resmgr-aliased stand-in (sim_alloc.c) -- never used for
* anything plugin-registry/fan-out related, which is the SEPARATE
* concern `core` above (task 4.3) exists to fix honestly. */
xce_scheduler_t* sched = (xce_scheduler_t*)plugin->sched;
int rc;
xce_pendreason_t* reason = ctx->reason;
@ -225,11 +245,11 @@ static int my_alloc(xce_sched_plugin_t* plugin, xce_sched_ctx_t* ctx)
return rc;
}
/* scheduler-sim-live-allocation task 1.3: sched_check_shared_satisfy()
* is now core-typed -- minimal &(sched->core) wrap; `sched` itself
* keeps its existing xce_scheduler_t* type here (derived from
* plugin->sched), unchanged by this task. */
rc = sched_check_shared_satisfy(&(sched->core), uncomit,
/* scheduler-sim-live-allocation task 4.3: sched_check_shared_satisfy()
* takes core directly (already core-typed since task 1.3) -- read
* from the plugin's own honest `core` field, not derived from
* plugin->sched. */
rc = sched_check_shared_satisfy(core, uncomit,
&(ctx->alloc_req.shared), &(ctx->unmeet));
if (rc < 0) {
return rc;
@ -241,7 +261,7 @@ static int my_alloc(xce_sched_plugin_t* plugin, xce_sched_ctx_t* ctx)
for (i = 0; i < ctx->alloc_req.shared.count; i++) {
xce_resource_t* r = NULL;
rc = vget(sched->core.resmgr, ((int*)ctx->unmeet.idxs.data)[i],
rc = vget(core->resmgr, ((int*)ctx->unmeet.idxs.data)[i],
(void**)&r);
if (rc <= 0) {
continue;
@ -300,17 +320,20 @@ static int my_alloc(xce_sched_plugin_t* plugin, xce_sched_ctx_t* ctx)
host_iter->next = sched_host_iter_next;
host_iter->fini = sched_host_iter_fini;
if (host_iter->init(host_iter, plan, sched->core.workers) == 0) {
if (host_iter->init(host_iter, plan, core->workers) == 0) {
xce_sched_worker_t* cand;
while ((cand = host_iter->next(host_iter)) != NULL) {
/* scheduler-sim-live-allocation spec, task 2.1
* (Core-Safe Allocation Chain): sched_try_alloc_on_
* host_() is now core-typed -- minimal &(sched->core)
* wrap; `sched` itself keeps its existing
* xce_scheduler_t* type here (derived from
* plugin->sched), unchanged by this task. */
if (sched_try_alloc_on_host_(cand, &(sched->core), ctx)
/* scheduler-sim-live-allocation spec, task 4.3:
* sched_try_alloc_on_host_() takes core directly
* (already core-typed since task 2.1) -- this is the
* exact call whose own internal plugin_prepare_
* allocation() fan-out (resource.c) must reach the
* SAME core->plugins registry every other plugin
* (e.g. the limit plugin) is registered against, so
* it reads the plugin's honest `core` field, not a
* throwaway wrapper derived from plugin->sched. */
if (sched_try_alloc_on_host_(cand, core, ctx)
> 0) {
w = cand;
break;
@ -367,9 +390,19 @@ static int my_alloc(xce_sched_plugin_t* plugin, xce_sched_ctx_t* ctx)
return 0;
}
int register_seqjob_allocator(void* sched)
/*
* seqjob_allocator_new_plugin - scheduler-sim-live-allocation spec, task
* 4.3: factory half of register_seqjob_allocator()'s own template-build
* step, extracted so a caller with no xce_scheduler_t* (the simulation
* driver, via plugin_register_core() -- see sim_alloc.c) can obtain a
* freshly built plugin instance to register directly against a real
* xce_sched_core_t*. See plugin.h's own doc comment on this declaration
* for the full rationale (mirrors limit_new_plugin()/builtin_fcfs_new_
* plugin()'s identical, already-established precedent).
*/
xce_sched_plugin_t* seqjob_allocator_new_plugin(void)
{
static xce_sched_plugin_t seqjob_allocator_plugin = {
static xce_sched_plugin_t seqjob_allocator_plugin_template = {
.name = "xce_sched_seqjob_allocator",
.hooks
= { .config_hook = NULL,
@ -385,14 +418,25 @@ int register_seqjob_allocator(void* sched)
.data = NULL
};
xce_sched_plugin_t* plugin = NULL;
xce_sched_plugin_t* plugin
= (xce_sched_plugin_t*)calloc(1, sizeof(*plugin));
if (plugin == NULL) {
return NULL;
}
memcpy(plugin, &seqjob_allocator_plugin_template, sizeof(*plugin));
return plugin;
}
int register_seqjob_allocator(void* sched)
{
xce_sched_plugin_t* plugin = seqjob_allocator_new_plugin();
plugin = (xce_sched_plugin_t*)calloc(1, sizeof(*plugin));
if (plugin == NULL) {
return -ENOMEM;
}
memcpy(plugin, &seqjob_allocator_plugin, sizeof(*plugin));
return plugin_register(sched, plugin);
}

250
src/scheduler/src/sim_alloc.c
View file

@ -7,92 +7,80 @@
/*
* sim_alloc.c - scheduler-sim-live-allocation spec, task 4.1 (Req 1.1, 1.2,
* boundary: Unified Picker/Allocator Threads). Split out of sim_loop.c (this
* project's 800-line-per-file convention -- see sim_loop.c's own file header
* comment) rather than folded into it, the same "new sibling file when the
* owning file has zero line-budget headroom" precedent sim_picker.c/
* sim_limit.c already establish.
* boundary: Unified Picker/Allocator Threads), rewritten by task 4.3 (Req
* 2.2, boundary: Unified Picker/Allocator Threads, Limit Plugin Core-Safe
* Matching) to close bug-1829's own tracked follow-up. Split out of
* sim_loop.c (this project's 800-line-per-file convention -- see
* sim_loop.c's own file header comment) rather than folded into it, the
* same "new sibling file when the owning file has zero line-budget
* headroom" precedent sim_picker.c/sim_limit.c already establish.
*
* ── The gap this closes ──────────────────────────────────────────────────
* Task 4.1 wires sched_sim_driver_run() to spawn the REAL sched_job_picker()/
* sched_allocator() thread pair (sched_election.c) against simulation
* mode's own core state -- for the first time ever, a simulation-mode
* session can reach sched_allocator()'s real sched_make_allocation() call,
* which dispatches to sched_make_seqjob_allocation()/sched_make_parjob_
* allocation() (scheduler.c) based on the job's maxTasks. Both of THOSE
* unconditionally dereference `core->seqjob_allocator`/`core->parjob_
* allocator` -- fields that, for simulation mode, were NEVER populated
* anywhere: sched_sim_driver_start() (sim.c) registers no plugin at all
* (sim.h's own documented "Known scope boundary"), and normal mode's own
* registration path, register_seqjob_allocator()/register_parjob_
* allocator() (plugins/seqjob_allocator.c, plugins/parjob_allocator.c),
* requires a genuine `xce_scheduler_t*` -- their own init_fn writes
* `((xce_scheduler_t*)plugin->sched)->core.seqjob_allocator = plugin`, and
* every one of their runtime hooks (seqjob_prepare_allocation()/my_alloc(),
* parjob_prepare_allocation()/my_alloc()) ALSO reads `plugin->sched`
* throughout, not merely at registration time -- an `xce_sched_sim_driver_t`
* deliberately has no such field (Req 5 AC 17). This is a real, tracked,
* pre-existing gap: test_sched_election_sim_fixture.c's own header comment
* (task 3.1) already flagged it ("a pre-existing, tracked, OUT-OF-SCOPE
* gap... NOT one of task 3.1's five permitted changes"), routing around it
* in that test with a throwaway xce_scheduler_t wrapper because that test
* never needed the wrapper and the core-under-test to be the SAME core.
* Task 4.1 is the first task that actually reaches this code path in
* PRODUCTION (the old, now-removed sim_picker_thread_fn() never called
* sched_make_allocation() at all) -- without this fix, ANY job reaching
* real allocation in simulation mode crashes (NULL deref on
* `core->seqjob_allocator`/`parjob_allocator`), for every resreq shape,
* regardless of dispatch/pend outcome. Closing it is a direct
* precondition for this task's own Observable ("produces a genuine
* dispatch or pend decision... based on actual resource availability"),
* exactly the "blocking, in-boundary, minimally-scoped pre-existing bug
* discovered via TDD" this codebase's own precedent (bug-1828, task 2.3)
* treats as acceptable to fix within the task that discovers it -- logged
* as bug-1829 (.wolf/buglog.json) with the real impact stated in full.
* ── The gap task 4.1 closed ───────────────────────────────────────────────
* sched_sim_driver_run() spawns the REAL sched_job_picker()/sched_allocator()
* thread pair (sched_election.c) against simulation mode's own core state --
* sched_allocator()'s real sched_make_allocation() call unconditionally
* dereferences `core->seqjob_allocator`/`core->parjob_allocator`, fields
* that, for simulation mode, were never populated anywhere (sim.h's own
* documented "Known scope boundary"; bug-1829). Task 4.1 fixed this so a
* genuine dispatch/pend decision could be reached at all.
*
* A FULL fix -- retyping seqjob_allocator.c's/parjob_allocator.c's runtime
* hooks from `plugin->sched` to the already-available `plugin->core` field
* (task 1.2), mirroring task 2.3's limit.c migration -- is NOT attempted
* here: those hooks also call sched_make_allocs()/sched_try_alloc_on_
* host_() (scheduler.c/resource.c) with the SAME xce_scheduler_t*, and
* scheduler.c is explicitly out of this task's file boundary (already
* migrated/committed). That full migration is real, tracked follow-up work
* (logged in the same buglog entry), not this task's job.
* ── The gap task 4.3 closes (bug-1829's own tracked follow-up) ───────────
* Task 4.1's own fix registered the seqjob/parjob allocator plugin against
* a throwaway `xce_scheduler_t` wrapper via the real, unmodified
* register_seqjob_allocator()/register_parjob_allocator() -- required only
* because those two functions' own init_fn (seqjob_allocator_init()/
* parjob_allocator_init(), plugins/seqjob_allocator.c/parjob_allocator.c)
* needed a genuine xce_scheduler_t*. Because that registration path
* (plugin_register()) also sets the plugin instance's own `core` field
* (task 1.2) to `&(wrapper.core)`, every later prepare_allocation_hook
* fan-out reached during a real dispatch attempt (my_alloc() ->
* sched_try_alloc_on_host_() -> plugin_prepare_allocation(), resource.c/
* plugin.c) iterated the WRAPPER's own separate core->plugins registry --
* which held only the allocator plugin itself -- never reaching a plugin
* registered only against the REAL simulation core (sim->core), the limit
* plugin (sim_bootstrap_limit_plugin()) included. A host-scoped limit's
* usage instance could therefore never be created via a live seqjob/parjob
* dispatch attempt in simulation mode.
*
* ── The fix ─────────────────────────────────────────────────────────────
* A long-lived (heap-allocated, never freed -- same "accepted leak, no
* sim-driver teardown exists" convention this file family already
* establishes for the picker/allocator threads themselves), throwaway
* `xce_scheduler_t` wrapper per allocator plugin, whose `.core.resmgr`/
* `.core.workers` fields are pointed at simulation mode's OWN resmgr/
* workers `var` handles (core.h) -- not copied data, the SAME underlying
* rtti-managed container objects -- before the real, unmodified
* register_seqjob_allocator()/register_parjob_allocator() runs against the
* wrapper. `var` (core.h) is a plain assignable reference, not an embedded
* value, so any resource/worker registered later against sim->core (via
* the real event-ingest path, after this bootstrap runs) is immediately
* visible through the wrapper's aliased fields too -- the two plugins'
* runtime hooks (which read resmgr/workers off `plugin->sched->core`)
* therefore see simulation mode's REAL, live resource/worker state, not an
* empty, disconnected copy. Every other call these plugins make
* (sched_check_shared_satisfy()/sched_try_alloc_on_host_(), both core-safe
* since tasks 1.3/2.1) is invoked with `&(wrapper->core)` throughout their
* own internal call chain (never mixed with `sim->core` directly), so the
* plugin's own internal plugin-hook fan-out (plugin_prepare_allocation(),
* task 1.2) stays self-consistent against the wrapper's own plugin
* registry -- which holds exactly the one allocator plugin this bootstrap
* registered onto it, nothing else.
* ── The fix (task 4.3) ────────────────────────────────────────────────────
* seqjob_allocator_init()/parjob_allocator_init() (and every runtime hook
* these plugins register -- seqjob_prepare_allocation()/my_alloc(),
* parjob_prepare_allocation()/my_alloc()/parjob_try_plan_()) now read the
* plugin's own honestly-typed `core` field (task 1.2) instead of deriving
* an `xce_scheduler_t*` from `plugin->sched` -- mirroring task 2.3's
* plugins/limit.c migration exactly. `core` is set by BOTH registration
* entry points BEFORE init_fn ever runs (plugin.c), so
* seqjob_allocator_new_plugin()/parjob_allocator_new_plugin() (new factory
* functions, mirroring limit_new_plugin()/builtin_fcfs_new_plugin()) can be
* registered DIRECTLY against `&(sim->core)` via the existing
* plugin_register_core() -- the exact same call sim_bootstrap_picker()/
* sim_bootstrap_limit_plugin() already use -- reaching the SAME
* core->plugins registry every other sim-mode plugin is registered in. No
* separate wrapper registry stands in for the fan-out anymore.
*
* Known limitation (out of this task's Observable, tracked in the same
* buglog entry): because each wrapper's OWN `core.plugins` registry is
* separate from `sim->core.plugins`, a plugin registered only against
* `sim->core` (e.g. sim_bootstrap_limit_plugin()'s limit plugin) is NOT
* reached by plugin_prepare_allocation()'s fan-out when it runs inside
* these allocators' own call chain -- limit-plugin host-scoped matching
* under a REAL simulation-mode dispatch is therefore not yet wired
* end-to-end. That is squarely task 4.2/5.2's own territory (limit-usage
* parity under live sim-mode dispatch), not task 4.1's (which only needs a
* genuine dispatch/pend decision to exist at all).
* A narrow exception remains, by necessity, not oversight:
* seqjob_allocator.c's/parjob_allocator.c's own my_alloc() still calls
* sched_make_allocs() (scheduler.c, out of this task's file boundary) to
* build the final xce_alloc_t records for the winning host -- a pre-
* existing, real dependency this task did not introduce and cannot retype
* (scheduler.c is off-limits). sched_make_allocs()'s own body only ever
* reads `sched->core.resmgr` (to resolve a resource idx to its name) and
* calls sched_get_id(sched) (`sched->core.lock`/`.nextid`, a fresh
* allocation-id counter) -- both my_alloc()'s own honest `core` parameter
* AND this narrow legacy need are satisfied by keeping a small,
* long-lived (heap-allocated, never freed -- same accepted-leak convention
* this file family already uses for its own never-joined threads)
* throwaway `xce_scheduler_t` wrapper, whose `.core.resmgr` alone is
* ALIASED (a `var`, core.h -- a plain assignable reference, not an embedded
* value) to simulation mode's own real resmgr, so sched_make_allocs()
* resolves resource names correctly. This wrapper is set on `plugin->sched`
* ONLY AFTER plugin_register_core() has already run (init_fn no longer
* reads plugin->sched at all) -- it is never used for the plugin registry
* or any prepare_allocation_hook fan-out, so it cannot reintroduce the
* separate-registry gap this task closes. (The wrapper's own `.nextid`
* counter is therefore separate from sim->core's -- an already-accepted,
* pre-existing quirk of this exact call, unchanged from task 4.1's own
* wrapper-based fix and outside this task's own Observable.)
*/
/*
@ -110,8 +98,9 @@
*/
void sim_bootstrap_seqjob_allocator(xce_sched_sim_driver_t* sim)
{
xce_scheduler_t* wrapper;
int rc;
xce_scheduler_t* wrapper;
xce_sched_plugin_t* plugin;
int rc;
if (sim == NULL) {
return;
@ -120,43 +109,64 @@ void sim_bootstrap_seqjob_allocator(xce_sched_sim_driver_t* sim)
wrapper = (xce_scheduler_t*)calloc(1, sizeof(*wrapper));
if (wrapper == NULL) {
ERROR("%s out of memory bootstrapping the sim-mode seqjob "
"allocator plugin",
"allocator plugin's sched_make_allocs() stand-in",
__func__);
return;
}
rc = xce_sched_core_init(&(wrapper->core));
if (rc < 0) {
ERROR("%s failed to init the seqjob allocator bootstrap wrapper "
"core (rc=%d)",
ERROR("%s failed to init the seqjob allocator sched_make_allocs() "
"stand-in core (rc=%d)",
__func__, rc);
free(wrapper);
return;
}
/* Share sim's own resource/worker registries -- see this file's own
* header comment for the full rationale. `sim->core.resmgr`/`workers`
* are `var` handles (core.h), plain assignable references, not
* embedded values -- this aliases the SAME underlying rtti-managed
* container objects, not a disconnected copy. */
wrapper->core.resmgr = sim->core.resmgr;
wrapper->core.workers = sim->core.workers;
/* See this file's own header comment: aliases the SAME underlying
* rtti-managed resmgr object, not a disconnected copy -- needed only
* for my_alloc()'s own remaining sched_make_allocs() calls. */
wrapper->core.resmgr = sim->core.resmgr;
rc = register_seqjob_allocator(wrapper);
if (rc < 0) {
ERROR("%s failed to register the sim-mode seqjob allocator plugin "
"(rc=%d)",
__func__, rc);
return; /* wrapper leaked -- accepted, see header comment */
plugin = seqjob_allocator_new_plugin();
if (plugin == NULL) {
ERROR("%s out of memory building the sim-mode seqjob allocator "
"plugin instance",
__func__);
free(wrapper);
return;
}
sim->core.seqjob_allocator = wrapper->core.seqjob_allocator;
/* THE fix (task 4.3): register directly against sim->core -- the SAME
* registry sim_bootstrap_picker()/sim_bootstrap_limit_plugin() already
* use. plugin_register_core() sets plugin->core = &(sim->core) BEFORE
* calling init_fn, so seqjob_allocator_init() (now core-safe) writes
* core->seqjob_allocator onto the REAL sim->core, and every later
* plugin_prepare_allocation() fan-out this plugin's own hooks trigger
* (also now core-safe) reaches sim->core.plugins -- the same registry
* the limit plugin is registered in. */
rc = plugin_register_core(&(sim->core), plugin);
if (rc < 0) {
ERROR("%s failed to register the sim-mode seqjob allocator plugin "
"against the simulation core (rc=%d)",
__func__, rc);
free(plugin);
free(wrapper);
return;
}
/* plugin->sched: deliberately set AFTER registration -- init_fn no
* longer reads it (see seqjob_allocator_init()). Narrow, remaining use:
* my_alloc()'s own sched_make_allocs() calls only (never the plugin
* registry/fan-out path fixed above). */
plugin->sched = wrapper;
}
void sim_bootstrap_parjob_allocator(xce_sched_sim_driver_t* sim)
{
xce_scheduler_t* wrapper;
int rc;
xce_scheduler_t* wrapper;
xce_sched_plugin_t* plugin;
int rc;
if (sim == NULL) {
return;
@ -165,30 +175,40 @@ void sim_bootstrap_parjob_allocator(xce_sched_sim_driver_t* sim)
wrapper = (xce_scheduler_t*)calloc(1, sizeof(*wrapper));
if (wrapper == NULL) {
ERROR("%s out of memory bootstrapping the sim-mode parjob "
"allocator plugin",
"allocator plugin's sched_make_allocs() stand-in",
__func__);
return;
}
rc = xce_sched_core_init(&(wrapper->core));
if (rc < 0) {
ERROR("%s failed to init the parjob allocator bootstrap wrapper "
"core (rc=%d)",
ERROR("%s failed to init the parjob allocator sched_make_allocs() "
"stand-in core (rc=%d)",
__func__, rc);
free(wrapper);
return;
}
wrapper->core.resmgr = sim->core.resmgr;
wrapper->core.workers = sim->core.workers;
wrapper->core.resmgr = sim->core.resmgr;
rc = register_parjob_allocator(wrapper);
if (rc < 0) {
ERROR("%s failed to register the sim-mode parjob allocator plugin "
"(rc=%d)",
__func__, rc);
return; /* wrapper leaked -- accepted, see header comment */
plugin = parjob_allocator_new_plugin();
if (plugin == NULL) {
ERROR("%s out of memory building the sim-mode parjob allocator "
"plugin instance",
__func__);
free(wrapper);
return;
}
sim->core.parjob_allocator = wrapper->core.parjob_allocator;
rc = plugin_register_core(&(sim->core), plugin);
if (rc < 0) {
ERROR("%s failed to register the sim-mode parjob allocator plugin "
"against the simulation core (rc=%d)",
__func__, rc);
free(plugin);
free(wrapper);
return;
}
plugin->sched = wrapper;
}

28
src/scheduler/testsuite/meson.build
View file

@ -2107,3 +2107,31 @@ sched_sim_live_allocation_test = executable(
test('scheduler sim live allocation test',
sched_sim_live_allocation_test, suite: 'scheduler',
env: ['ASAN_OPTIONS=detect_leaks=0'])
# scheduler-sim-live-allocation spec, task 4.3 (Req 2.2, boundary: Unified
# Picker/Allocator Threads, Limit Plugin Core-Safe Matching): proves a
# host-scoped limit registered directly against simulation mode's real core
# (sim_bootstrap_limit_plugin()) is matched -- and its usage instance
# created -- during a genuine seqjob/parjob dispatch attempt driven through
# the real sched_job_picker()/sched_allocator() thread pair, closing the
# gap task 4.1 left (bug-1829: the seqjob/parjob allocator plugin's own
# plugin->core pointed at a throwaway wrapper's separate registry, never
# sim.core.plugins). Same real-scheduler_srcs link technique as
# sched_election_sim_fixture_test/sched_sim_live_allocation_test above.
sched_sim_alloc_limit_fanout_test_srcs = ['test_sim_alloc_limit_fanout.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
sched_sim_alloc_limit_fanout_test_srcs += ['../' + f]
endif
endforeach
sched_sim_alloc_limit_fanout_test = executable(
'sched_sim_alloc_limit_fanout_test',
sched_sim_alloc_limit_fanout_test_srcs,
include_directories:[incdir],
dependencies: [scheduler_test_deps],
install: false)
test('scheduler sim alloc limit fanout test',
sched_sim_alloc_limit_fanout_test, suite: 'scheduler',
env: ['ASAN_OPTIONS=detect_leaks=0'])

23
src/scheduler/testsuite/test_parjob_combinatorial.c
View file

@ -145,6 +145,12 @@ int plugin_prepare_allocation(xce_sched_core_t* sched, xce_sched_ctx_t* ctx)
memset(&fake_plugin, 0, sizeof(fake_plugin));
fake_plugin.sched = sched;
/* scheduler-sim-live-allocation spec, task 4.3: parjob_prepare_
* allocation() now reads plugin->core (honestly typed, task 1.2)
* instead of plugin->sched -- `sched` here is already an
* xce_sched_core_t* (this stub's own signature), so no cast is
* needed. */
fake_plugin.core = sched;
return parjob_prepare_allocation(&fake_plugin, ctx);
}
@ -311,6 +317,11 @@ static int drive_my_alloc(xce_scheduler_t* sched, xce_sched_obj_t* obj,
memset(&plugin, 0, sizeof(plugin));
plugin.sched = sched;
/* scheduler-sim-live-allocation spec, task 4.3: my_alloc() now reads
* plugin->core (honestly typed, task 1.2) for every core-safe call,
* alongside the still-needed plugin->sched (sched_make_allocs(),
* scheduler.c, unchanged). */
plugin.core = &(sched->core);
return my_alloc(&plugin, ctx);
}
@ -818,7 +829,11 @@ static void test_descending_bucket_order_succeeds_where_ascending_fails(void)
plan.nbuckets = 2;
plan.bucket_slots = bucket_slots_desc;
rc = parjob_try_plan_(&sched, &ctx, &plan, claimed, &nclaimed);
/* scheduler-sim-live-allocation spec, task 4.3: parjob_try_plan_() is
* now honestly typed xce_sched_core_t* (mirrors task 2.1's identical
* ctx->sched ripple immediately above) -- pass &(sched.core), not
* &sched. */
rc = parjob_try_plan_(&(sched.core), &ctx, &plan, claimed, &nclaimed);
TEST_ASSERT_EQUAL_INT(1, rc);
TEST_ASSERT_EQUAL_INT(2, nclaimed);
TEST_ASSERT_TRUE(claimed[0].w == host_first);
@ -841,7 +856,11 @@ static void test_descending_bucket_order_succeeds_where_ascending_fails(void)
plan.nbuckets = 2;
plan.bucket_slots = bucket_slots_asc;
rc = parjob_try_plan_(&sched, &ctx, &plan, claimed, &nclaimed);
/* scheduler-sim-live-allocation spec, task 4.3: parjob_try_plan_() is
* now honestly typed xce_sched_core_t* (mirrors task 2.1's identical
* ctx->sched ripple immediately above) -- pass &(sched.core), not
* &sched. */
rc = parjob_try_plan_(&(sched.core), &ctx, &plan, claimed, &nclaimed);
TEST_ASSERT_EQUAL_INT(0, rc);
TEST_ASSERT_EQUAL_INT(0, nclaimed);

14
src/scheduler/testsuite/test_seqjob_allocator.c
View file

@ -95,6 +95,12 @@ int plugin_prepare_allocation(xce_sched_core_t* sched, xce_sched_ctx_t* ctx)
memset(&fake_plugin, 0, sizeof(fake_plugin));
fake_plugin.sched = sched;
/* scheduler-sim-live-allocation spec, task 4.3: seqjob_prepare_
* allocation() now reads plugin->core (honestly typed, task 1.2)
* instead of plugin->sched -- `sched` here is already an
* xce_sched_core_t* (this stub's own signature), so no cast is
* needed. */
fake_plugin.core = sched;
return seqjob_prepare_allocation(&fake_plugin, ctx);
}
@ -280,6 +286,11 @@ test_my_alloc_allocates_single_matching_host_through_shared_iterators(void)
memset(&plugin, 0, sizeof(plugin));
plugin.sched = &sched;
/* scheduler-sim-live-allocation spec, task 4.3: my_alloc() now reads
* plugin->core (honestly typed, task 1.2) for every core-safe call,
* alongside the still-needed plugin->sched (sched_make_allocs(),
* scheduler.c, unchanged). */
plugin.core = &(sched.core);
rc = my_alloc(&plugin, &ctx);
@ -341,6 +352,9 @@ test_seqjob_prepare_allocation_guards_against_maxtasks_gt_1_fanout(void)
memset(&plugin, 0, sizeof(plugin));
plugin.sched = &sched;
/* scheduler-sim-live-allocation spec, task 4.3: seqjob_prepare_
* allocation() now reads plugin->core (honestly typed, task 1.2). */
plugin.core = &(sched.core);
ctx.worker = NULL;
rc = seqjob_prepare_allocation(&plugin, &ctx);

11
src/scheduler/testsuite/test_seqjob_rewrap_equivalence.c
View file

@ -125,6 +125,12 @@ int plugin_prepare_allocation(xce_sched_core_t* sched, xce_sched_ctx_t* ctx)
memset(&fake_plugin, 0, sizeof(fake_plugin));
fake_plugin.sched = sched;
/* scheduler-sim-live-allocation spec, task 4.3: seqjob_prepare_
* allocation() now reads plugin->core (honestly typed, task 1.2)
* instead of plugin->sched -- `sched` here is already an
* xce_sched_core_t* (this stub's own signature), so no cast is
* needed. */
fake_plugin.core = sched;
return seqjob_prepare_allocation(&fake_plugin, ctx);
}
@ -314,6 +320,11 @@ static int drive_my_alloc(xce_scheduler_t* sched, xce_sched_obj_t* obj,
memset(&plugin, 0, sizeof(plugin));
plugin.sched = sched;
/* scheduler-sim-live-allocation spec, task 4.3: my_alloc() now reads
* plugin->core (honestly typed, task 1.2) for every core-safe call,
* alongside the still-needed plugin->sched (sched_make_allocs(),
* scheduler.c, unchanged). */
plugin.core = &(sched->core);
return my_alloc(&plugin, ctx);
}

389
src/scheduler/testsuite/test_sim_alloc_limit_fanout.c Normal file
View file

@ -0,0 +1,389 @@
/*
* test_sim_alloc_limit_fanout.c - scheduler-sim-live-allocation spec, task
* 4.3 (Req 2.2, boundary: Unified Picker/Allocator Threads, Limit Plugin
* Core-Safe Matching).
*
* ── The gap this closes (design.md's "Known Seam Gap: Host-Scoped Limit
* Fan-Out During Live Seqjob/Parjob Dispatch") ─────────────────────────
* Task 4.1's sim_bootstrap_seqjob_allocator()/sim_bootstrap_parjob_allocator()
* (sim_alloc.c) registered the seqjob/parjob allocator plugin against a
* throwaway xce_scheduler_t wrapper's OWN separate core.plugins registry --
* required only because register_seqjob_allocator()/register_parjob_
* allocator()'s own init_fn (seqjob_allocator_init()/parjob_allocator_init(),
* plugins/seqjob_allocator.c/parjob_allocator.c) still read plugin->sched,
* needing a real xce_scheduler_t*. seqjob_allocator.c's/parjob_allocator.c's
* own my_alloc() then resolves its plugin-hook fan-out (plugin_prepare_
* allocation()) via THAT SAME plugin instance's own plugin->core field, not
* the core parameter threaded down from sched_make_allocation() -- so with
* plugin->core pointing at the wrapper's own core, the fan-out iterates the
* wrapper's registry (holding only the allocator plugin itself), never
* reaching a plugin registered on the REAL simulation core (sim->core) --
* the limit plugin (sim_bootstrap_limit_plugin()) included. A host-scoped
* limit's usage instance could therefore never be created via a live
* seqjob/parjob dispatch attempt.
*
* This test proves the fix: a host-scoped limit registered directly against
* simulation mode's real core (sim_bootstrap_limit_plugin(), unchanged) is
* matched -- and its usage instance created -- during a GENUINE seqjob
* dispatch attempt, driven through the real, unmodified sched_job_picker()/
* sched_allocator() thread pair (sched_election.c) spawned against nothing
* but &(sim.core), the exact shape sched_sim_driver_run() itself uses.
* Independent of recorded-decision reconciliation: this fixture never
* builds a .replay stream or calls sim_reconcile_unaccounted_jobs() at all.
* Independent of global-limit matching: the one limit configured here is
* host-scoped (a percent-based limit on a real, non-GLOBAL/non-SHARED
* resource), never a global limit.
*
* ── Link style: real scheduler_srcs, not a white-box #include ────────────
* Same technique test_sched_election_sim_fixture.c/test_sim_live_
* allocation.c already establish (see those files' own header comments):
* links the REAL, unmodified `scheduler_srcs` object list (minus
* src/main.c) so my_alloc()'s real fan-out through the real plugin.c/
* resource.c/plugins/limit.c chain is genuinely exercised, not stubbed.
*
* ── Fixture shape ──────────────────────────────────────────────────────
* A bare xce_sched_sim_driver_t, built and bootstrapped by hand (mirroring
* test_sched_election_sim_fixture.c's own "bare core fixture" technique)
* rather than driven through a real .replay file: xce_sched_core_init(),
* then the SAME four bootstrap calls sched_sim_driver_run() (sim_loop.c)
* itself makes, in the same order, all against this one real sim.core --
* sim_bootstrap_picker(), sim_bootstrap_limit_plugin() (seeded with one
* host-scoped limit config), sim_bootstrap_seqjob_allocator(),
* sim_bootstrap_parjob_allocator(). A real worker (added directly via
* sched_add_worker(), not a wire WORKERS_PUSH ingest -- this fixture needs
* no replay/capnp machinery) gives the job's auto-injected "slots" want and
* this test's own limit resource genuine capacity, so the fixture proves
* the STRONGEST possible outcome: a real DISPATCH, not merely a PEND that
* happened to touch the fan-out.
*
* `job_wrapper` (a genuine, real, but otherwise-unused xce_scheduler_t)
* exists SOLELY to satisfy make_sched_job()'s own out-of-boundary
* precondition (schedobj.c still takes `void* arg` cast to xce_scheduler_t*
* -- a pre-existing gap task 3.1's own fixture already routes around the
* same way). Its core.resmgr/workers are ALIASED (not copied) to sim.core's
* own -- the same "var is a plain assignable reference" technique task
* 4.1's own sim_alloc.c fix established -- so parse_resreq() resolves
* resource names against the exact same resmgr this test's real dispatch
* attempt uses. job_wrapper is never passed to anything else.
*
* Same "fixtures here are never torn down, exactly one RUN_TEST(), process
* exits immediately after" convention every other real-scheduler_srcs
* bare-core-fixture test in this directory follows (test_sched_election_
* sim_fixture.c's own file header comment) -- the two spawned, never-
* joined picker/allocator threads read a stack-local `sim` for the rest of
* the process's short remaining life, which is safe for the same reason
* documented there.
*/
#include <errno.h>
#include <pthread.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <unity/unity.h>
#include "controller.capnp.h"
#include "decision.h"
#include "plugin.h"
#include "preason.h"
#include "resobj.h"
#include "sched/types.h"
#include "scheduler.h"
#include "sim.h"
#include "xcelimit.h"
#include "xstring.h"
/* Consumed the same "extern declared where consumed" way sim_loop.c itself
* declares these four (sim_loop.c's own file header comment) -- none of
* them has a shared header declaration. */
extern void sim_bootstrap_picker(xce_sched_sim_driver_t* sim);
extern void sim_bootstrap_limit_plugin(xce_sched_sim_driver_t* sim);
extern void sim_bootstrap_seqjob_allocator(xce_sched_sim_driver_t* sim);
extern void sim_bootstrap_parjob_allocator(xce_sched_sim_driver_t* sim);
#define TEST_JOBID ((uint64_t)430301ULL)
#define QUEUE_NAME "normal"
#define TEST_RESREQ ((char*)"select[type==any] rusage[mem=1]")
#define WORKER_NAME "sim_alloc_limit_worker"
#define WORKER_SLOTS 4.0
#define WORKER_HOSTRES 1000.0
#define LIMIT_RESNAME "sim_alloc_host_res"
#define LIMIT_NAME "sim_alloc_host_limit"
void setUp(void) { return; }
void tearDown(void) { return; }
/* ---------------------------------------------------------------------- */
/* Fixture helpers */
/* ---------------------------------------------------------------------- */
static xce_resource_t* register_plain_resource(var resmgr, const char* name)
{
xce_resource_t res;
memset(&res, 0, sizeof(res));
res.name = strdup(name);
return makeResource(resmgr, &res);
}
/* add_test_worker_resval - same field-population step core_build_worker()/
* make_sched_worker() (core_events.c/schedobj.c) use to fill a worker's
* resvals, applied by hand since neither of those two helpers is reachable
* without a real wire xce_worker_t payload (out of this task's boundary to
* add one). */
static void add_test_worker_resval(xce_sched_worker_t* worker,
xce_sched_core_t* core,
const char* resname, double value)
{
xce_resource_t* r = NULL;
xce_val_t* val = NULL;
int rc;
rc = key_find(core->resmgr, (void*)resname, (void**)&r);
TEST_ASSERT_TRUE(rc >= 0);
TEST_ASSERT_NOT_NULL(r);
val = (xce_val_t*)calloc(1, sizeof(*val));
TEST_ASSERT_NOT_NULL(val);
val->which = Value_u_fvalue;
val->u.fvalue = value;
zarray_put(&(worker->resvals), r->idx, val);
}
static xce_sched_worker_t* make_test_worker(xce_sched_core_t* core,
const char* name)
{
xce_sched_worker_t* worker = (xce_sched_worker_t*)calloc(1,
sizeof(*worker));
TEST_ASSERT_NOT_NULL(worker);
worker->name = strdup(name);
worker->status = 0;
worker->idx = -1;
zarray_init(&(worker->resvals), 128, (zarray_item_killer_t)kill_sched_resval);
zarray_init(&(worker->resInUse), 128, (zarray_item_killer_t)kill_sched_resval);
add_test_worker_resval(worker, core, "slots", WORKER_SLOTS);
add_test_worker_resval(worker, core, LIMIT_RESNAME, WORKER_HOSTRES);
return worker;
}
/* ---------------------------------------------------------------------- */
/* The test */
/* ---------------------------------------------------------------------- */
static void
test_host_scoped_limit_usage_instance_created_via_live_sim_dispatch(void)
{
xce_sched_sim_driver_t sim;
xce_scheduler_t job_wrapper; /* setup-only, see file header */
xce_sched_plugin_t* picker = NULL;
xce_sched_plugin_t* limit_plugin = NULL;
xce_sched_queue_t* queue;
xce_sched_worker_t* worker;
xce_job_t j;
job_submit_data_t submit;
xce_sched_job_t* job;
xce_limit_val_t limit_val = { .resname = (char*)LIMIT_RESNAME,
.percent = 1,
.val = 50.0 };
xce_limit_config_t limit_cfg = { .name = (char*)LIMIT_NAME,
.val = &limit_val };
xce_limit_config_t* limits[1] = { &limit_cfg };
xce_conf4sched_t config;
pthread_t picker_tid;
pthread_t allocator_tid;
xce_decision_t* decision = NULL;
xce_decision_t* end_marker = (xce_decision_t*)0x1;
var instances_handle;
iterator_t it;
void* item = NULL;
int rc;
memset(&sim, 0, sizeof(sim));
rc = xce_sched_core_init(&(sim.core));
TEST_ASSERT_EQUAL_INT(0, rc);
/* Real production resources: "slots" (the auto-injected per-task want
* every primary resreq gets) and LIMIT_RESNAME (this test's own
* host-scoped limit's underlying resource -- instantiate_limit()
* (xcelimit.c) requires it to genuinely exist in core->resmgr, same
* requirement test_repl_limit_table.c's own host-scoped fixture
* documents). Neither carries XCE_RESF_GLOBAL/XCE_RESF_SHARED, so
* is_host_based_limit() resolves LIMIT_NAME as host-based purely from
* its own percent!=0 shape. */
TEST_ASSERT_NOT_NULL(register_plain_resource(sim.core.resmgr, "slots"));
TEST_ASSERT_NOT_NULL(
register_plain_resource(sim.core.resmgr, LIMIT_RESNAME));
/* Real production bootstrap sequence -- the SAME four calls
* sched_sim_driver_run() (sim_loop.c) makes, in the SAME order, all
* against this one real sim.core. */
sim_bootstrap_picker(&sim);
rc = key_find(sim.core.plugins, "builtin_fcfs", (void**)&picker);
TEST_ASSERT_TRUE(rc >= 0);
TEST_ASSERT_NOT_NULL(picker);
memset(&config, 0, sizeof(config));
config.nlimits = 1;
config.limits = limits;
sim.core.config = &config;
sim_bootstrap_limit_plugin(&sim);
rc = key_find(sim.core.plugins, "xce_sched_limit", (void**)&limit_plugin);
TEST_ASSERT_TRUE(rc >= 0);
TEST_ASSERT_NOT_NULL(limit_plugin);
TEST_ASSERT_EQUAL_PTR_MESSAGE(
&(sim.core), limit_plugin->core,
"sanity: the limit plugin must be registered directly against "
"sim.core -- the SAME registry this test's seqjob/parjob allocator "
"plugin must ALSO reach for its own prepare_allocation_hook "
"fan-out to ever find it");
/* THE fix under test (task 4.3): sim_bootstrap_seqjob_allocator()/
* sim_bootstrap_parjob_allocator() must register their plugin
* instance's own `core` field against this SAME sim.core -- not a
* throwaway wrapper's own separate registry (task 4.1's own
* documented gap, design.md's "Known Seam Gap: Host-Scoped Limit
* Fan-Out During Live Seqjob/Parjob Dispatch"). */
sim_bootstrap_seqjob_allocator(&sim);
sim_bootstrap_parjob_allocator(&sim);
TEST_ASSERT_NOT_NULL_MESSAGE(
sim.core.seqjob_allocator,
"sim_bootstrap_seqjob_allocator() must populate core->seqjob_"
"allocator (task 4.1, unrelated to this task's own fix)");
/* Real queue registration -- same sequence test_sched_election_sim_
* fixture.c's own fixture establishes. */
queue = (xce_sched_queue_t*)calloc(1, sizeof(*queue));
TEST_ASSERT_NOT_NULL(queue);
queue->name = makeString(QUEUE_NAME);
rc = sched_add_queue(&(sim.core), queue);
TEST_ASSERT_TRUE(rc >= 0);
picker->picker_iface->on_queue_update(picker, queue);
/* A real worker with genuine "slots" AND LIMIT_RESNAME capacity, added
* directly via the real sched_add_worker() (core.c). */
worker = make_test_worker(&(sim.core), WORKER_NAME);
rc = sched_add_worker(&(sim.core), worker);
TEST_ASSERT_TRUE(rc >= 0);
/* Real job construction via make_sched_job() -- see file header for
* why job_wrapper (a real xce_scheduler_t, its core.resmgr/workers
* ALIASED to sim.core's own) is used only for this one call. */
memset(&job_wrapper, 0, sizeof(job_wrapper));
rc = xce_sched_core_init(&(job_wrapper.core));
TEST_ASSERT_EQUAL_INT(0, rc);
job_wrapper.core.resmgr = sim.core.resmgr;
job_wrapper.core.workers = sim.core.workers;
/* Also alias .queues (not just .resmgr/.workers): make_sched_job()'s
* own internal sched_get_queue(&(job_wrapper.core), ...) call is
* get-only (never auto-creates, core.c) -- without this alias it
* would look up "normal" in job_wrapper.core's own separate, empty
* queue registry and return NULL, leaving job->queue NULL, which
* silently fails builtin_fcfs_on_job_push()'s own (job->queue == NULL)
* guard (returns -EINVAL, job never becomes a picker candidate) --
* caught via gdb during this test's own RED-phase debugging. */
job_wrapper.core.queues = sim.core.queues;
memset(&submit, 0, sizeof(submit));
submit.queue = (char*)QUEUE_NAME;
submit.user = (char*)"sim_alloc_limit_user";
submit.project_name = (char*)"default";
submit.resreq = TEST_RESREQ;
submit.mintasks = 1;
submit.maxtasks = 1;
memset(&j, 0, sizeof(j));
j.jobid = TEST_JOBID;
j.state = JOB_STAT_PEND;
j.submit = &submit;
job = make_sched_job(&job_wrapper, &j);
TEST_ASSERT_NOT_NULL(job);
TEST_ASSERT_EQUAL_INT(XCE_RESREQ_PRIMARY, job->schedobj->tag);
/* make_sched_job()'s own parse_resreq() call does not thread
* submit.mintasks/maxtasks into the parsed primary's own minTasks/
* maxTasks (task 3.1's own documented gap) -- set directly so this
* fixture deterministically exercises the seqjob path. */
job->schedobj->u.primary->resreq->minTasks = 1;
job->schedobj->u.primary->resreq->maxTasks = 1;
rc = sched_add_job(&(sim.core), job);
TEST_ASSERT_TRUE(rc >= 0);
picker->picker_iface->on_job_push(picker, job);
/* ── The actual thing under test ─────────────────────────────────
* the REAL, unmodified sched_job_picker()/sched_allocator() thread
* pair (sched_election.c), spawned against nothing but &(sim.core) --
* exactly the shape sched_sim_driver_run() itself uses. */
rc = pthread_create(&picker_tid, NULL, sched_job_picker, &(sim.core));
TEST_ASSERT_EQUAL_INT(0, rc);
rc = pthread_create(&allocator_tid, NULL, sched_allocator, &(sim.core));
TEST_ASSERT_EQUAL_INT(0, rc);
rc = xce_core_go(&(sim.core));
TEST_ASSERT_EQUAL_INT(0, rc);
cbuf_get(&(sim.core.decisions), &decision);
TEST_ASSERT_NOT_NULL_MESSAGE(
decision, "the real thread pair must carry this job all the way to "
"a committed decision");
cbuf_get(&(sim.core.decisions), &end_marker);
TEST_ASSERT_NULL_MESSAGE(end_marker,
"second decisions-cbuf entry must be the "
"end-of-session NULL marker");
rc = xce_core_end_session(&(sim.core));
TEST_ASSERT_EQUAL_INT(0, rc);
TEST_ASSERT_EQUAL_UINT_MESSAGE(
JOB_STAT_RUN, job->state,
"the worker's real slots/host-resource capacity must be genuinely "
"sufficient for a real DISPATCH -- the strongest possible proof "
"that the per-host prepare_allocation_hook fan-out this test cares "
"about actually ran, not merely that some decision was reached");
/* The Observable this task's brief requires: a host-scoped limit's
* usage instance is visible via the existing usage-instances accessor,
* created by this LIVE dispatch attempt alone -- reconciliation and
* global-limit matching were never invoked anywhere in this fixture. */
instances_handle = limit_get_usage_instances(&(sim.core));
TEST_ASSERT_NOT_NULL(instances_handle);
rc = iter_make(&it, instances_handle);
TEST_ASSERT_TRUE(rc >= 0);
rc = iter_next(&it, &item);
TEST_ASSERT_TRUE(rc >= 0);
TEST_ASSERT_NOT_EQUAL(RTTI_ITER_NO_DATA, rc);
TEST_ASSERT_NOT_NULL_MESSAGE(
item,
"a host-scoped limit registered directly against sim.core must be "
"matched -- and its usage instance created -- during a genuine "
"live seqjob/parjob dispatch attempt; before task 4.3's fix the "
"seqjob/parjob allocator plugin's own plugin->core pointed at a "
"throwaway wrapper's separate registry, so this fan-out never "
"reached the limit plugin registered on sim.core");
iter_destroy(&it);
printf(
"test_host_scoped_limit_usage_instance_created_via_live_sim_"
"dispatch: PASS\n");
}
int main(void)
{
UNITY_BEGIN();
RUN_TEST(
test_host_scoped_limit_usage_instance_created_via_live_sim_dispatch);
return UNITY_END();
}