lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(scheduler): route sim reconciliation's on_dispatch() through the allocator thread (bug-1837)

Browse source 
sim_reconcile_unaccounted_jobs() (sim_loop.c) was calling
picker->picker_iface->on_dispatch() directly from the sim driver thread,
violating plugin.h's documented "allocator thread only" contract for
Post-allocation hooks. Dormant in practice only because builtin_fcfs.c
happens to lock internally around every hook -- a defensive measure the
contract text doesn't actually require of picker plugins.

Since scheduler-sim-live-allocation, sim mode spawns the same real
sched_job_picker()/sched_allocator() thread pair normal mode uses, so a
real allocator thread now exists to route through. Added a
reconcile_req/reconcile_ack cbuf pair on xce_sched_core_t and a reserved
XCE_SCHED_RECONCILE_WAKE sentinel pushed through jbuf to wake the
allocator thread without touching jbuf's pinned capacity-1/item-type
invariant (Req 5 AC 7). sched_allocator() services the request and calls
on_dispatch() itself; sim_reconcile_invoke_on_dispatch() blocks on the ack.
Falls back to a direct call when no allocator thread is live for a given
core (standalone unit tests), which is safe by construction.

TDD: new test spawns a stand-in allocator-relay thread and asserts
on_dispatch() fires there, not on the caller's thread -- confirmed RED
against the old direct-call code, GREEN after. Four other test files with
hand-rolled sched_allocator() stubs needed the same relay branch (found by
an actual hang at meson's timeout). Full scheduler suite: 85/85, zero
regressions.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-09-21 12:58:57 +00:00
parent 94115b0024
commit 4f94638a1d
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
15 changed files with 541 additions and 67 deletions
Download patch file Download diff file Expand all files Collapse all files

30
src/scheduler/include/core.h
View file

@ -165,6 +165,16 @@ typedef void (*xce_unsatisfiable_populate_cb_t)(xce_sched_core_t* core,
xce_sched_obj_t* obj,
int reason_category);
/* Reserved jbuf sentinel for the Recorded-Decision Reconciliation
* allocator-thread relay (bug-1837) -- see xce_sched_core_s's reconcile_req/
* reconcile_ack fields' own doc comment below for the full protocol. Never a
* real jobid (job ids are assigned by the controller's monotonic id
* generator, starting near 1, nowhere near UINT64_MAX) and distinct from
* jbuf's own pre-existing 0 "end of session" sentinel -- same reserved-value
* idiom this codebase already uses elsewhere for an analogous "not a real
* value" marker (see SCHED_SIM_START_TS_LATEST, sim.h). */
#define XCE_SCHED_RECONCILE_WAKE ((uint64_t)-1)
struct xce_sched_core_s {
mylock_t lock;
int stop;
@ -198,6 +208,26 @@ struct xce_sched_core_s {
pthread_t allocator;
cbuf_t wbuf; /* action to trigger scheduling */
cbuf_t jbuf; /* job picked for allocation */
/* Recorded-Decision Reconciliation allocator-thread relay (bug-1837):
* plugin.h documents on_dispatch()/on_pend() as allocator-thread-only
* hooks, but sim_loop.c's sim_reconcile_unaccounted_jobs() (the driver
* thread) needs to invoke on_dispatch() for a job it reconstructs from
* replay history -- there is no fresh allocation to run through the
* normal jbuf/next_job() pipeline for it (the job is already RUN per
* history, not a live candidate), so it cannot flow through jbuf's
* existing real-jobid protocol. These two dedicated cbufs let the driver
* hand the call off to the real sched_allocator() thread (sched_
* election.c) instead of calling it directly: reconcile_req carries the
* jobid to dispatch, reconcile_ack echoes on_dispatch()'s return code
* once applied. Deliberately SEPARATE from jbuf, not a widened jbuf item
* type -- jbuf's item size/capacity are a pinned determinism invariant
* (test_core_jbuf_capacity_pin.c, Req 5 AC 7) that must never change.
* See XCE_SCHED_RECONCILE_WAKE below and sched_allocator()'s matching
* branch (sched_election.c) for the full handoff protocol. Normal mode
* never populates reconcile_req (only sim_loop.c's reconciliation does),
* so this is a true no-op for normal mode's allocator thread. */
cbuf_t reconcile_req; /* jobid for on_dispatch, allocator-thread relay */
cbuf_t reconcile_ack; /* on_dispatch()'s rc, echoed back once applied */
cbuf_t decisions; /* scheduler decisions */
/* Req 5 AC 6 (scheduler-core spec, task 4.4): driver-registered
* decision callback + opaque driver context, set via

19
src/scheduler/include/sim.h
View file

@ -93,6 +93,25 @@ typedef struct {
bool interactive;
uint64_t snapshot_ts;
xce_sched_repl_state_t* repl_state;
/* bug-1837 fix: set true by sched_sim_driver_run() (sim_loop.c),
* immediately after it successfully spawns the real sched_allocator()
* thread (sched_election.c) -- both the write here and every read in
* sim_reconcile_unaccounted_jobs() happen on the SAME thread (the one
* running sched_sim_driver_run()'s own round loop), in program order, so
* this is plain single-threaded state, not a cross-thread flag; no
* atomics needed. Tells sim_reconcile_invoke_on_dispatch() (sim_loop.c)
* whether a real allocator thread exists to relay on_dispatch() through
* (core->reconcile_req/reconcile_ack, core.h) or whether this xce_sched_
* sim_driver_t was only ever driven directly (no thread pair spawned at
* all -- e.g. test_sim_reconcile_unaccounted.c's own "call sim_
* reconcile_unaccounted_jobs() standalone" Testing Strategy), in which
* case calling on_dispatch() directly is safe by construction: there is
* only ever one thread touching picker state in that case, so plugin.h's
* allocator-thread-only contract has nothing to violate. Stays false
* forever for a bare xce_sched_sim_driver_t built via fixture_init()-
* style direct core init, matching sched_sim_driver_start()'s own
* memset(sim, 0, sizeof(*sim)) default. */
bool allocator_thread_live;
} xce_sched_sim_driver_t;
/*