Elevates the normal-mode/simulation-mode driver equivalence guarantee from a one-line architectural description in CLAUDE.md to an enforced steering constraint (scheduler-driver-equivalence.md), backed by bug-1611/bug-1613 as concrete evidence of violation. Scopes the fix as its own spec (scheduler-sim-decision-parity, requirements + design) rather than reopening the closed scheduler-core spec or folding it into scheduler-sim-repl-verbosity's explicitly narrower boundary. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
5 KiB
5 KiB
XCE Scheduler
Overview
C-based job scheduling engine with a plugin architecture. Determines job-to-node assignments based on resource requirements and scheduling policies. Since scheduler-core (2026-08), the engine is split into a mode-agnostic core state machine driven by two interchangeable drivers — a controller-connected normal-mode driver and a standalone .replay-file simulation driver — so scheduling decisions can be recorded live and deterministically replayed offline for debugging/tuning.
Build
/xce-build scheduler # debug + ASAN + tests; add `all` instead of `scheduler` when a downstream
# task also needs xceadmin/controller binaries coexisting in the same build dir
Architecture
- Core (
core.c/core_events.c/core_session.c/core_clock.c) — passive, event-drivenxce_sched_core_tstate machine. Zero controller I/O (nonetio/xceapi/client/delay/idgen);core_clock.cis the one sanctioned exception (clock_gettime, virtualized two-base+speed clock formula,xce_core_now()).test_core_isolation.cmechanically enforces this forcore.c/core.h/core_events.c/core_session.c(source-text scan) pluscore_clock.ctoo on thenm(1)undefined-symbol layer (all four compiled together into the isolatedsched_core_isolation_libarchive, meson.build) —core_clock.cis deliberately excluded only from the wall-clock text scan, since it owns the one sanctionedclock_gettime()call. Picker→allocator handoff is a strictjbufcbuf capacity-1 rendezvous — this is the project's determinism invariant, not a tuning knob; never raise it without re-deriving Req 5 AC 7 in.kiro/specs/scheduler-core/requirements.md. - Picker plugin interface (
plugin.h'sxce_job_picker_iface_t,plugins/builtin_fcfs.c) — job-selection strategies are pluggable;scheduler.c'ssched_job_picker()branches on whether a plugin still uses the legacyjob_elect_fn-only shape (plugins/fcfs.c, kept as-is for a compat-wrapper exercise, not the live default) vs the native hook interface (builtin_fcfs, the registered default). New scheduling policies → add as a picker plugin undersrc/plugins/, do not modify the core loop. - Drivers —
driver.c(normal mode: capnp↔core event translation, decision publishing, deferred admin-request handling) andsim.c/sim_loop.c/sim_record.c/sim_verify.c(simulation mode:.replayfile loading, discrete virtual clock,--sim-verify/--sim-alt-events/--sim-output). Both drive the identical core session/decision-callback protocol — one core, two consumers. This equivalence is a DESIGN INVARIANT, not just an architectural description — see.kiro/steering/scheduler-driver-equivalence.mdfor the precise (state-level, not just decision-level) contract, known violations (bug-1611, bug-1613), and the testing discipline required of any change in this area. - Replay/recording —
recording.c/recording_events.c/recording_session.c(live capture) andsim.c/sim_loop.c(simulation-side decode) do the actual capnp encode/decode of the.replaybinary stream (src/schema/controller.capnp);snapshot.c/snapshot_wire.c/snapshot_load.conly populate/consume the plain in-memorysched_snapshot_data_tstruct thatrecording.c'swrite_snapshot_entry()then serializes — no capnp or file I/O lives in the snapshot files themselves. Controlled viaOPCode_schedAdmin(src/controller/cluster.c) andxceadmin scheduler record start|stop|status/scheduler dump <FILE>. - Interactive REPL (
repl.c/repl_break.c/repl_info.c/repl_params.c,--interactive) — single-step/breakpoint/state-inspection/snapshot-capture control over the simulation driver; drives the exact samesched_sim_driver_run()as batch--sim, gated only bysim->interactive, so REPL and batch runs are decision-equivalent by construction (test_repl_equivalence.c).
Links against libxce from src/lib/. Full design/requirements: .kiro/specs/scheduler-core/.
Conventions
- C standard: gnu17
- Format with
clang-format - New scheduling policies → add as a plugin under
src/plugins/, do not modify the core engine - Shared types defined in
src/include/xce/(libxce) - Files stay under the repo's 800-line cap — several core/sim/repl/snapshot/recording modules are deliberately split across multiple
*_events.c/*_session.c/*_wire.c-style files for this reason; keep new logic in the file matching its concern rather than growing one file past the cap. Not yet true project-wide:scheduler.c(1382 lines) andplugins/builtin_fcfs.c(1026 lines) are pre-existing exceptions over the cap with no split or exemption note — do not use them as precedent for new files, and prefer splitting them opportunistically rather than growing them further.
See Also
- Shared library:
src/lib/CLAUDE.md - Root guidelines:
CLAUDE.md(repo root) - Architecture/requirements:
.kiro/specs/scheduler-core/design.md,.kiro/specs/scheduler-core/requirements.md