feat(scheduler-sim-live-allocation): make limit matching core-safe (task 2.3)

Migrates match_job_global_limit_()/limit_job_election()/
match_job_host_limit_() (plugins/limit.c) to read plugin->core (added
task 1.2) instead of deriving xce_scheduler_t* from plugin->sched,
which stays NULL for a plugin registered via plugin_register_core()
(simulation mode). Removes the dead ctx.sched write flagged during
task 2.1's review -- verified this ctx never has .sched read anywhere
downstream.

Fixes bug-1828 (found while writing this task's own TDD test for a
previously untested code path): match_job_host_limit_() had the same
key_find()-negative-rc-treated-as-fatal defect bug-1786 already fixed
in its global-limit sibling, but never applied here -- meaning
host-scoped resource limits were never actually enforced, in any
mode, until this fix (not merely a display gap; the limit's vres was
never added to obj->hlimits, so sched_fill_limit_requests() always
saw an empty list for it).

TDD: new tests drive the real hook-registered entry points against a
plugin_register_core()-registered plugin, asserting real usage
instances via limit_get_usage_instances() for both global and
host-scoped limits.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RLSNJhNoULvVcUhXEFafQC
This commit is contained in:
Rongsong Shen 2026-09-18 16:21:25 +00:00
parent ecdea8bf55
commit fc23ea8d31
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
4 changed files with 347 additions and 27 deletions

View file

@ -22465,6 +22465,50 @@
],
"occurrences": 1,
"last_seen": "2026-09-17T21:50:00Z"
},
{
"id": "bug-1827",
"timestamp": "2026-09-18T07:15:00Z",
"error_message": "NOT A CRASH -- coverage gap found during an /xce-audit-invariants scheduler run (dimension 2: core module I/O isolation), while investigating driver state-equivalence for bug-1826. test_core_isolation.c's own header comment claims core_clock.c joins ONLY the nm(1) undefined-symbol enforcement layer, not the wall-clock text-scan check -- implying it is still covered by the OTHER two text-scan checks (forbidden #include list, forbidden symbol substrings). In reality, core_clock.c is absent from all three text-scan functions (test_core_never_includes_forbidden_headers(), test_core_never_calls_forbidden_symbols(), and the wall-clock scan), not just the wall-clock one -- no XCE_CORE_CLOCK_C_PATH macro is defined anywhere in test_core_isolation.c or testsuite/meson.build's c_args for that test.",
"file": "src/scheduler/testsuite/test_core_isolation.c",
"root_cause": "core_clock.c was added to the mechanically-enforced sched_core_isolation_lib archive (meson.build) and given a documented single exception (the wall-clock text scan, for its one sanctioned clock_gettime() call), but the two OTHER text-scan functions (forbidden-headers, forbidden-symbols) were never extended to also read core_clock.c's source text -- only core.h/core.c/core_events.c/core_session.c are read by those two checks (via XCE_CORE_H_PATH/_C_PATH/_EVENTS_C_PATH/_SESSION_C_PATH macros; no clock-file variant exists). The nm(1) undefined-symbol layer (which DOES include core_clock.c in the archive) only catches an actual undefined-symbol reference at link time -- it cannot catch a bare forbidden #include (e.g. client.h/netio.h/xceapi.h/delay.h/idgen.h/xce/cluster.h) that resolves to no undefined symbol (a macro-only or inline-only header), so the doc comment's implied coverage is narrower in practice than stated. No evidence found that core_clock.c's actual current content violates isolation today -- this is a gap in what the mechanism would CATCH if it did, not a confirmed live violation.",
"fix": "NOT FIXED -- audit-only finding, reported to the user for triage, not bundled into the concurrent scheduler-sim-limit-matching spec work. Straightforward fix when picked up: add a XCE_CORE_CLOCK_C_PATH macro (testsuite/meson.build's c_args for test_core_isolation) and extend test_core_never_includes_forbidden_headers()/test_core_never_calls_forbidden_symbols() to also scan core_clock.c's source text, matching the coverage the file's own doc comment already claims it has.",
"tags": [
"scheduler",
"core-isolation",
"test-coverage-gap",
"test_core_isolation",
"core_clock",
"audit-finding",
"unfixed",
"discovered-during-xce-audit-invariants"
],
"occurrences": 1,
"last_seen": "2026-09-18T07:15:00Z"
},
{
"id": "bug-1828",
"error_message": "IMPACT CORRECTED 2026-09-18 (independent review of scheduler-sim-live-allocation task 2.3): this is NOT just a display/accounting gap -- it meant host-scoped resource limits were NEVER ACTUALLY ENFORCED, in ANY mode (normal or simulation), for this function's entire history. Originally discovered while writing scheduler-sim-live-allocation spec task 2.3's own required TDD test (a host-scoped limit match, via the simulation-mode-registered limit plugin, must create a real usage instance verifiable through limit_get_usage_instances()). limit_prepare_allocation()/match_job_host_limit_() consistently returned rc=-2 (-ENOENT) instead of >= 0, and no usage instance was ever created, even though match_limit() itself genuinely matched (rc=1) and is_host_based_limit() correctly identified the limit as host-based. Tracing the call chain (plugin.c's plugin_prepare_allocation() hook loop -> sched_fill_limit_requests()) shows the missing usage instance meant a host-scoped limit's vres was never added to obj->hlimits, so sched_fill_limit_requests() always iterated an empty list -- the limit's numeric cap was silently never applied to any job's resource request, in production, since this code existed. This does not corrupt the dispatch/pend decision machinery itself (plugin_prepare_allocation()'s hook-calling loop discards the hook's return value, so the bug never fed into a control-flow branch) -- it is a dormant-limit-never-binds defect, but a real enforcement gap, not merely a cosmetic one.",
"root_cause": "match_job_host_limit_() (src/scheduler/src/plugins/limit.c) is the host-scoped sibling of match_job_global_limit_() and has the same defect CLASS bug-1786 fixed in that function alone (misinterpreting key_find()'s negative \"not found\" return as fatal, rather than the correct not-yet-instantiated signal) -- not byte-for-byte identical wording (bug-1786's own guard was `if (rc <= 0) { return rc; }`; this sibling's was `if (rc < 0) { return rc; }`), same underlying mistake. Right after a successful match_limit(), it does `rc = key_find(privdata->instances, vresname, &vres); if (rc < 0) { return rc; }` before the `if (vres == NULL) { instantiate_limit(...) }` block below it. On the FIRST-EVER lookup of a given vresname (always true the first time a host-scoped limit ever matches), key_find() signals \"not found\" as a NEGATIVE rc (-ENOENT), not 0 -- this codebase's established kvmgr convention (get_group(), is_host_based_limit(), and bug-1786/bug-1495's own root causes) -- so this early-return unconditionally caught the expected miss and returned before instantiate_limit() could ever run. bug-1786 (2026-09-07) fixed this exact defect class in match_job_global_limit_() but never touched its host-scoped sibling, which retained the bug because no test exercised match_job_host_limit_()/limit_prepare_allocation() with real matching until this task added one (verified via grep: zero prior test call sites for either function anywhere in the test suite).",
"fix": "FIXED (2026-09-18, scheduler-sim-live-allocation task 2.3). Removed the erroneous `if (rc < 0) { return rc; }` guard in match_job_host_limit_() entirely, mirroring match_job_global_limit_()'s own bug-1786 fix exactly: `vres` (calloc'd NULL, only ever reassigned by a successful key_find) is already the correct, sufficient signal for \"needs instantiation\" via the existing `if (vres == NULL)` block immediately below. TDD: new test test_limit_prepare_allocation_matches_host_limit_via_sim_registered_plugin (test_repl_limit_table.c) drives the real limit_prepare_allocation()/match_job_host_limit_() path against a plugin registered via plugin_register_core() and a real percent-based host-scoped limit; RED (rc=-2, confirmed via temporary debug instrumentation tracing is_host_based_limit()=1, match_limit()=1, but limit_prepare_allocation()=-2) before this fix, GREEN (rc>=0, usage instance created and visible via limit_get_usage_instances()) after. Full scheduler suite: 79/79, zero regressions.",
"files": [
"src/scheduler/src/plugins/limit.c (match_job_host_limit_)",
"src/scheduler/testsuite/test_repl_limit_table.c (new test)"
],
"tags": [
"scheduler",
"limit-plugin",
"kvmgr",
"error-code-convention",
"not-found-vs-error",
"match_job_host_limit_",
"sibling-of-bug-1786",
"scheduler-sim-live-allocation",
"task-2.3",
"fixed"
],
"occurrences": 1,
"last_seen": "2026-09-18T15:14:00Z"
}
]
}