fix(scheduler-sim-diagnostics): bootstrap limit plugin in sim mode (bug-1825)

info limit was correctly implemented and tested but inert against a real
replay: register_limit()'s only call site was normal-mode sched_register_
plugins(); sim mode's own plugin bootstrap (sim_bootstrap_picker()) never
registered the limit plugin. The snapshot format already carries the
original session's limit config (sim->core.config, a borrowed reference set
by xce_sched_core_load_snapshot()) -- nothing needed to change in the
recording/replay format.

New sim_bootstrap_limit_plugin() (sim_limit.c, mirroring sim_picker.c's
structure) registers the plugin via plugin_register_core() and seeds it via
a new limit_seed_config(plugin, xce_sched_core_t* core, config) entry point,
wired into sim_loop.c alongside the existing sim_bootstrap_picker() call.
limit_new_plugin() extracted from register_limit() as a factory, mirroring
builtin_fcfs_new_plugin()'s established pattern; register_limit() itself is
behavior-preserving.

First attempt set plugin->sched to a disguised xce_sched_sim_driver_t*,
relying on it sharing xce_scheduler_t's first-member layout to reuse the
existing config_hook path -- rejected on review: this repeats the exact
cross-cast pattern that already corrupted memory once in this codebase (see
core.h's xce_core_cache_clear_cb_t history). Fixed by making
limit_seed_config() take xce_sched_core_t* as an honest parameter instead,
with limit_update_config() (normal mode) and the new sim path both
delegating to a shared limit_seed_config_impl() -- no cast, no aliasing risk,
independently verified by two review rounds reading the actual struct
layouts and call chains.

Also added sim_limit.c to sched_sim_isolation_lib so Req 5 AC 17's
controller-I/O-isolation check covers it (an initial gap, closed in the same
remediation round). Two ripple waves of the same established link-stub
pattern applied to the ~24 white-box test files that #include sim_loop.c.

info limit now shows real configured-limit data against a genuine replay.
Live usage will still correctly show "no active usage instance yet" until a
separate, larger fix (logged as bug-1826) wires limit *matching* into the
sim decision path -- deliberately out of this task's scope.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RLSNJhNoULvVcUhXEFafQC
This commit is contained in:
Rongsong Shen 2026-09-18 04:46:49 +00:00
parent 2f1ffd1e4a
commit 487e7b9419
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
32 changed files with 1377 additions and 28 deletions

View file

@ -22428,7 +22428,7 @@
"error_message": "NOT A CRASH -- significant, verified architectural gap discovered while closing out scheduler-sim-diagnostics task 5 (final validation task for the new `info limit` REPL command). The limit plugin is never registered in ANY simulation-mode driver path, so `info limit <name|all>` will ALWAYS report \"no limits configured\"/\"no such limit\" for a real operator running `xce_scheduler -i`/`-s <replay>` against a genuine recording -- even one whose original live run had limits configured and actively enforced. The entire feature built across scheduler-sim-diagnostics tasks 1-5 is correctly implemented, thoroughly tested, and read-only/deterministic as designed, but is currently INERT in real production usage: every positive-display test path (tasks 2-3, showing a limit's configured setting and live usage) only exercises the code via a test-only helper that hand-registers the plugin into core->plugins directly, which has no production equivalent in sim mode.",
"file": "src/scheduler/src/sim_picker.c",
"root_cause": "register_limit() (plugins/limit.c:609) has exactly one call site in the entire codebase: scheduler.c:605, inside sched_register_plugins(), which is itself called only from scheduler.c:827 -- the normal-mode (controller-connected, live) driver's sched_init() path. Sim mode's own plugin-bootstrap mechanism, sim_bootstrap_picker() (sim_picker.c), exists specifically to register a picker plugin (builtin_fcfs, via plugin_register_core()) for simulation/replay mode, since sim mode deliberately does not run the full normal-mode init sequence -- but it only ever registers the PICKER plugin category. It has no equivalent step for the limit plugin (or, by the same reasoning, potentially other plugin categories registered by sched_register_plugins() that were never audited for this gap). Confirmed via full-codebase grep: register_limit()/sched_register_plugins() are never referenced anywhere under sim.c/sim_loop.c/sim_picker.c/driver.c/repl*.c, and no write into core->plugins occurs anywhere in the sim-mode driver files. This is the same class of gap scheduler-driver-equivalence.md already documents for OTHER driver-parity properties (e.g. bug-1613's sched_account_jobs() gap) -- a sim-mode driver feature silently missing something the normal-mode driver does, discovered only when a REPL feature tried to read state that assumes it.",
"fix": "NOT FIXED -- discovered and characterized during scheduler-sim-diagnostics task 5 (final validation task), explicitly out of that task's test-only boundary per its own instructions (a production-code fix found during validation must be reported, not silently patched). Recommended fix: add a sim-mode limit-plugin bootstrap analogous to sim_bootstrap_picker() -- either a new sim_bootstrap_limit_plugin() called from the same place sim_bootstrap_picker() is invoked (sim_loop.c, per that function's own doc comment), or generalize sched_register_plugins() itself to be safely callable from sim mode (auditing whether builtin_fcfs's own registration would double-register if sim_bootstrap_picker() and sched_register_plugins() both ran). Needs a decision on whether the sim driver should register ALL production plugin categories (matching normal mode exactly, closing this gap generally) or add limit-specific bootstrap only (narrower, faster, but leaves the general problem for the next plugin type that needs sim-mode REPL visibility). Also needs the actual config source: sim mode's replay files must carry the original limit configuration if there is to be anything for a bootstrap step to register from -- confirm whether `.replay` files / ReplaySession already embed limit config, or whether that itself is a second, sequential gap. This second question was not investigated in this session -- flagged as the first thing a fix attempt needs to check.",
"fix": "FIXED (2026-09-17, scheduler-sim-diagnostics task 6). Added limit_new_plugin() factory (extracted from register_limit(), mirroring builtin_fcfs_new_plugin()'s established pattern) and a new sim_bootstrap_limit_plugin() (new file src/scheduler/src/sim_limit.c, mirroring sim_picker.c's structure exactly): registers the limit plugin via plugin_register_core() and seeds it with sim->core.config (the snapshot's own borrowed-reference config, which already carries the original session's limit configuration -- no recording/replay format change needed) via a new limit_seed_config(plugin, xce_sched_core_t* core, config) entry point. Wired into sim_loop.c alongside the existing sim_bootstrap_picker(sim) call.\n\nFirst attempt (round 1) set plugin->sched = (void*)sim (a disguised xce_sched_sim_driver_t*) to reuse the existing config_hook path -- REJECTED on independent review: this repeats the exact pointer-to-first-member cross-cast pattern that already corrupted memory once in this codebase (see this bug's own root_cause note and core.h's xce_core_cache_clear_cb_t doc comment for that prior incident). Round 2 eliminated the cast entirely: limit_seed_config() takes xce_sched_core_t* core as an honest parameter (a real field both xce_scheduler_t and xce_sched_sim_driver_t genuinely have), delegating to a shared limit_seed_config_impl() also used by normal-mode limit_update_config(). No cast, no aliasing risk -- confirmed structurally sound by two independent reviewers, each reading the actual struct definitions and full call chain rather than trusting the report.\n\nAlso closed: sim_limit.c added to sched_sim_isolation_lib's source list (meson.build) so Req 5 AC 17's mechanical controller-I/O-isolation check covers it (round 1 omitted this; round 2 fixed it, sched_sim_isolation_test independently re-verified passing with the new file included).\n\nRipple: two waves of an identical, minimal link-satisfying stub pattern across the ~24 white-box test files that #include \"../src/sim_loop.c\" (one for limit_new_plugin(), one for limit_seed_config()) -- same established 'duplicate, don't share' idiom this file family already used for the earlier scheduler-sim-diagnostics tasks' repl_info.c ripple.\n\nTDD: RED->GREEN verified for both the config-seed test and the remediation itself (a temporary limit_seed_config() no-op stub reproduced the exact expected single-test failure, reverted cleanly). Full scheduler suite: 78/78 Ok, 0 Fail; sched_repl_limit_table_test 26/26; sched_limit_plugin_test 5/5 (zero regression to normal-mode registration); sched_sim_isolation_test 5/5 (genuinely covering the new file).\n\nDeliberately NOT fixed (separate, larger, tracked as a distinct follow-up item in .kiro/specs/scheduler-sim-diagnostics/tasks.md, not this bug): limit *matching* (limit_job_election()) is still never invoked during replay -- it's only reachable via plugin_job_election(), which is normal-mode-only (one caller, scheduler.c) and would itself need a core-safe variant before it could be wired into sim_loop.c's decision path. So 'info limit' now shows real CONFIGURED settings against a real replay, but live USAGE will still correctly show 'no active usage instance yet' until that separate, larger fix lands.",
"tags": [
"scheduler",
"sim-mode",
@ -22442,6 +22442,29 @@
],
"occurrences": 1,
"last_seen": "2026-09-17T07:10:00Z"
},
{
"id": "bug-1826",
"timestamp": "2026-09-17T21:50:00Z",
"error_message": "NOT A CRASH -- known, deliberately out-of-scope follow-up discovered while fixing bug-1825 (sim-mode limit plugin bootstrap, scheduler-sim-diagnostics task 6). Bug-1825's fix makes the limit plugin's CONFIGURED SETTING visible in sim mode (info limit now shows a real replay's actual limit configuration). But limit *matching* -- the mechanism that actually populates live usage instances (xce_vres_t) as jobs get evaluated -- is still never invoked during replay. So info limit's live-usage display will correctly show 'no active usage instance yet' for every limit against any real replay, even one whose original live session actively enforced and charged against that limit.",
"file": "src/scheduler/src/plugin.c",
"root_cause": "limit_job_election() (plugins/limit.c, the function that calls match_job_global_limit_()/match_job_host_limit_() and populates xce_vres_t usage instances) is only ever invoked via plugin_job_election() (plugin.c:554), which -- like plugin_config_update()/plugin_prepare_allocation() -- takes a `void* arg` cast unconditionally to `xce_scheduler_t*` and reads `sched->core.plugins`. plugin_job_election() has exactly ONE call site in the entire codebase: scheduler.c (normal-mode only). Nothing in sim_loop.c's replay decision path calls plugin_job_election() or any core-safe equivalent, so no limit ever gets matched against any job during simulation/replay, regardless of whether the limit plugin is registered (bug-1825 fixes registration+config, not this).",
"fix": "NOT FIXED -- deliberately deferred, tracked in `.kiro/specs/scheduler-sim-diagnostics/tasks.md`'s Implementation Notes as a known non-goal of task 6, not bundled into that task's scope. Fixing this properly needs: (1) a core-safe variant of plugin_job_election() (mirroring the plugin_register()/plugin_register_core() split bug-1825 already established for registration, and the limit_update_config()/limit_seed_config() split for config-seeding) that takes xce_sched_core_t* honestly instead of casting a void* to xce_scheduler_t*; (2) wiring a call to it into sim_loop.c's replay decision path, at whatever point a job becomes eligible for dispatch -- this touches the scheduler's most invariant-sensitive area (see .kiro/steering/scheduler-driver-equivalence.md's documented state-equivalence contract and its required resource-contention-fixture testing discipline, plus the known related gaps bug-1611/bug-1613 already track for other normal-vs-sim parity issues in this same decision path). This is a materially larger, higher-risk change than bug-1825's registration+config fix and should go through this project's full TDD + contention-fixture validation discipline, not be rushed in as a drive-by addition.",
"tags": [
"scheduler",
"sim-mode",
"repl",
"limit-plugin",
"limit-matching",
"plugin-job-election",
"architectural-gap",
"unfixed",
"discovered-during-scheduler-sim-diagnostics",
"driver-parity",
"follow-up-to-bug-1825"
],
"occurrences": 1,
"last_seen": "2026-09-17T21:50:00Z"
}
]
}