chore(buglog): log bug-1830, mem-based limits don't populate usage via live replay

Discovered during live manual verification of scheduler-sim-diagnostics
Requirement 6 (human-readable info limit formatting) against a real
deployed cluster with a real captured replay. A mem-based limit
(limits2/ug1_global_mem/night_mem_pct) never shows a live usage
instance even when its matching job is genuinely dispatched with an
intact rusage[mem=...] reservation -- while a slots-based limit
(limit1) in the same replay correctly populates one. Suspected timing/
ordering gap between WORKERS_PUSH replay-event application and
match_job_global_limit_()/match_job_host_limit_()'s resmgr lookup, not
confirmed. Out of Requirement 6's boundary (display-formatting only,
proven correct independently) and out of bug-1826's already-closed
scope (matching invocation itself, not this resource-type-specific
timing gap). Needs its own dedicated investigation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RLSNJhNoULvVcUhXEFafQC
This commit is contained in:
Rongsong Shen 2026-09-20 09:02:27 +00:00
parent bf61f5954e
commit 38d9fe3583
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2

View file

@ -24606,6 +24606,33 @@
],
"occurrences": 1,
"last_seen": "2026-09-19T09:00:00Z"
},
{
"id": "bug-1830",
"timestamp": "2026-09-20T09:00:00Z",
"error_message": "NOT A CRASH -- live-verification finding while manually demonstrating scheduler-sim-diagnostics Requirement 6 (human-readable info limit formatting) against a real deployed cluster. A mem-based limit (limits2, ug1_global_mem, night_mem_pct -- all from the real src/configs/limits.yaml) never populates a live usage instance via `info limit all` against a real captured replay, even when the matching job is genuinely dispatched and its rusage[mem=...] reservation is confirmed intact (via `info job <id>`). A slots-based limit (limit1) in the SAME replay, against a matching dispatched job, DOES correctly populate a usage instance (accounted=0.00, capacity=3.00) -- so this is resource-type-specific to mem, not a general replay/limit-matching failure.",
"file": "src/scheduler/src/plugins/limit.c",
"root_cause": "Suspected, not confirmed: match_job_global_limit_()/match_job_host_limit_() (plugins/limit.c) key_find() the limit's constrained resource by name against core->resmgr at match time. For `slots`, this resource is always already present in core->resmgr from session start. For `mem`, the resource may only become known to core->resmgr via a WORKERS_PUSH replay event reporting the worker's live mem value -- if that push event is decoded/applied AFTER the job's own dispatch decision is replayed (or never lands in the resmgr in a way key_find() can see before the match attempt), the mem-based limit's match silently misses, leaving no xce_vres_t/usage instance ever created for it. test_sim_e2e_limit_visibility.c's own acceptance test (scheduler-sim-live-allocation task 5.2) only proves this works when the mem resource is hand-baked directly into the snapshot's resmgr BEFORE any replay event is processed -- it does not cover the ordinary real-deployment shape (resource known only via a live WORKERS_PUSH event that arrives as part of the replay stream itself, not pre-seeded in the snapshot). This needs direct tracing of match timing vs. WORKERS_PUSH application order in sim_loop.c's real round sequence to confirm.",
"fix": "NOT FIXED -- discovered during live manual verification, out of scheduler-sim-diagnostics Requirement 6's boundary (that feature is display-formatting only, and is proven correct independent of this finding: the configured-value line and any usage line that DOES populate both render human-readable correctly, confirmed live). Also out of scheduler-sim-live-allocation's already-closed boundary (bug-1826 there was specifically about limit MATCHING never being invoked at all during replay -- now fixed; this is a narrower, resource-type-specific timing/ordering gap in when a match against core->resmgr can succeed for a resource whose registration itself arrives via replay rather than being pre-seeded). Needs its own dedicated investigation: trace WORKERS_PUSH application order against match_job_global_limit_()/match_job_host_limit_()'s call timing in a real multi-round replay for a mem-based (not slots-based) limit.",
"tags": [
"scheduler",
"sim-mode",
"repl",
"limit-plugin",
"limit-matching",
"mem-resource",
"live-verification-finding",
"not-fixed",
"related-to-bug-1825",
"related-to-bug-1826",
"discovered-during-scheduler-sim-diagnostics-requirement-6-live-test"
],
"related_bugs": [
"bug-1825",
"bug-1826"
],
"occurrences": 1,
"last_seen": "2026-09-20T09:00:00Z"
}
]
}