chore(buglog): bug-1885..1889 fixed
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
9c51de88c6
commit
7b001cb0f7
GPG key ID:
80533536B0EB651C
6 changed files with 214 additions and 36 deletions
|
|
@ -16310,9 +16310,9 @@
|
|||
},
|
||||
{
|
||||
"id": "bug-1885",
|
||||
"status": "open",
|
||||
"status": "fixed",
|
||||
"status_updated": "2026-09-29",
|
||||
"status_note": "newly logged",
|
||||
"status_note": "",
|
||||
"component": "scheduler",
|
||||
"severity": "leak",
|
||||
"timestamp": "2026-09-29T09:48:44Z",
|
||||
|
|
@ -16320,8 +16320,8 @@
|
|||
"occurrences": 1,
|
||||
"error_message": "scheduler normal mode never frees decision structs: 64 bytes leaked per published decision\n\n**Observed:** (code reading) the normal-mode round loop in\n`xce_sched_driver_run_normal_mode_round()` (src/scheduler/src/driver.c)\ndrains each `xce_decision_t*` from `core->decisions`, records it and passes\nit to `on_decision`, but never frees the struct `make_*_decision()`\n(decision.c) calloc()s:\n```\ncbuf_get(&(sched->core.decisions), &decision);\n... sched->core.on_decision(sched->core.driver_ctx, decision);\n/* decision never freed */\n```\n\n**Expected:** the round loop frees the decision struct (not its alloc/pend reason, which the job owns) once published.\n**Impact:** leak of 64 bytes per dispatch/pend decision for the life of a normal-mode scheduler. Simulation mode was fixed in bug-1884 (ebfe4794). Three tests (test_core_full_cycle.c, test_core_session.c, test_snapshot_save_load.c) keep decision pointers from their callback, so the fix changes the callback contract (\"valid only during the call\") and those tests.\n**Found by:** review (while fixing bug-1884, 2026-09-29)",
|
||||
"file": "src/scheduler/src/driver.c, src/scheduler/src/decision.c",
|
||||
"root_cause": "TODO: required for fixed/partial",
|
||||
"fix": "TODO: required for fixed/partial\n\nVerification:\n- Test: TODO: name + file; RED before the fix, GREEN after\n- Suites: TODO: suite results and build profile\n- Live: TODO: soak/deploy run, or n/a and why",
|
||||
"root_cause": "`xce_sched_driver_run_normal_mode_round()` (driver.c) drains each decision from `core->decisions`, records it and passes it to `on_decision`, and never frees the struct `make_*_decision()` (decision.c) allocates.",
|
||||
"fix": "The round frees the decision struct after the callback returns (d21b3d8f). Its allocation/pend reason belong to the job; the publish callback passes those pointers on and the recorder copies, so nothing keeps the struct. Tests with their own drain loops (test_core_full_cycle.c, test_core_session.c, test_snapshot_save_load.c) are unaffected.\n\nVerification:\n- Test: test_round_frees_decision_after_callback in src/scheduler/testsuite/test_driver_session_ordering.c (drives the real round with a stand-in allocator emitting one PEND; checks the struct is freed via __asan_address_is_poisoned and the reason is not): RED before, GREEN after\n- Suites: ASAN debug build (src/xce.build.discrete), `meson test --no-suite xceadmin`: 117/117 OK, 0 sanitizer reports; xcelib_test 695 tests 0 failures (2026-09-29)\n- Live: 3-round sequential HA soak on rebuilt ASAN binaries (/home/claude/tmp/xce-bug1885-soak, 2026-09-29): PASS, 15 jobs, zero job loss, all 3 nodes led; SIMVERIFY 4 files 0 mismatches; each recording replays with detect_leaks=1 at exit 0, 0 mismatches; no sanitizer report in the scheduler or controller stderr",
|
||||
"tags": [
|
||||
"scheduler",
|
||||
"leak",
|
||||
|
|
@ -16330,14 +16330,16 @@
|
|||
"related_bugs": [
|
||||
"bug-1884"
|
||||
],
|
||||
"fix_commits": [],
|
||||
"fix_commits": [
|
||||
"d21b3d8f"
|
||||
],
|
||||
"record": ".wolf/bugs/bug-1885.md"
|
||||
},
|
||||
{
|
||||
"id": "bug-1886",
|
||||
"status": "open",
|
||||
"status": "fixed",
|
||||
"status_updated": "2026-09-29",
|
||||
"status_note": "newly logged",
|
||||
"status_note": "",
|
||||
"component": "scheduler",
|
||||
"severity": "leak",
|
||||
"timestamp": "2026-09-29T09:48:47Z",
|
||||
|
|
@ -16345,8 +16347,8 @@
|
|||
"occurrences": 1,
|
||||
"error_message": "scheduler leaks the rejected queue on each config re-apply and the old config on each CONFIG_PUSH\n\n**Observed:** (code reading)\n- `sched_init_queues()` (src/scheduler/src/scheduler.c) builds a new\n `xce_sched_queue_t` for every configured queue on each config apply and\n `sched_add_queue()`s it; for an existing name the add fails and the new\n queue (with its interned name/resreq) is still handed to the picker but\n owned by nothing.\n- `CONFIG_PUSH` (core_events.c) sets `core->config = config` without freeing\n the previous config.\n\n**Expected:** re-applied queues update the existing queue in place (as `core_upsert_queue()` does for QUEUE_UPDATE), and a replaced config is freed once nothing points into it.\n**Impact:** leak per config push/re-apply (one queue per configured queue plus one whole `xce_conf4sched_t`). Freeing the old config needs care: the picker and limit plugin may keep pointers into it. The queue table's datakiller must stay a no-op for as long as `sched_init_queues()` keeps using a rejected queue (bug-1884 found a use-after-free when it freed).\n**Found by:** review (while fixing bug-1884, 2026-09-29)",
|
||||
"file": "src/scheduler/src/scheduler.c, src/scheduler/src/core_events.c",
|
||||
"root_cause": "TODO: required for fixed/partial",
|
||||
"fix": "TODO: required for fixed/partial\n\nVerification:\n- Test: TODO: name + file; RED before the fix, GREEN after\n- Suites: TODO: suite results and build profile\n- Live: TODO: soak/deploy run, or n/a and why",
|
||||
"root_cause": "`sched_init_queues()` built a new `xce_sched_queue_t` for every configured queue on each config apply; for an existing name `sched_add_queue()` failed and the unregistered queue was handed to the picker (which then tracked it instead of the registered one) and leaked, while the registered queue kept its old settings. `sched_init_config()` and the core's CONFIG_PUSH replaced `core->config` without freeing the previous one.",
|
||||
"fix": "`sched_init_queues()` updates the registered queue in place (attrs, status, priority, accept_interval, sched_delay -- what make_sched_queue() copies), as QUEUE_UPDATE's core_upsert_queue() does; a replaced config is freed after the new one is applied -- nothing points into it (queues, resources, groups and limits built from it are copies) (3e873f0a).\n\nVerification:\n- Test: test_config_reapply_updates_queue_in_place_and_frees_old in src/scheduler/testsuite/test_config_push_deferred.c (applies a one-queue config twice, priority 1 then 7): RED before (priority stayed 1), GREEN after (same queue object, priority 7, first config freed per ASAN)\n- Suites: ASAN debug build (src/xce.build.discrete), `meson test --no-suite xceadmin`: 117/117 OK, 0 sanitizer reports; xcelib_test 695 tests 0 failures (2026-09-29)\n- Live: 3-round sequential HA soak on rebuilt ASAN binaries (/home/claude/tmp/xce-bug1885-soak, 2026-09-29): PASS, 15 jobs, zero job loss, all 3 nodes led; SIMVERIFY 4 files 0 mismatches; each recording replays with detect_leaks=1 at exit 0, 0 mismatches; no sanitizer report in the scheduler or controller stderr",
|
||||
"tags": [
|
||||
"scheduler",
|
||||
"leak",
|
||||
|
|
@ -16355,14 +16357,16 @@
|
|||
"related_bugs": [
|
||||
"bug-1884"
|
||||
],
|
||||
"fix_commits": [],
|
||||
"fix_commits": [
|
||||
"3e873f0a"
|
||||
],
|
||||
"record": ".wolf/bugs/bug-1886.md"
|
||||
},
|
||||
{
|
||||
"id": "bug-1887",
|
||||
"status": "open",
|
||||
"status": "fixed",
|
||||
"status_updated": "2026-09-29",
|
||||
"status_note": "newly logged",
|
||||
"status_note": "",
|
||||
"component": "lib",
|
||||
"severity": "wrong-result",
|
||||
"timestamp": "2026-09-29T09:48:49Z",
|
||||
|
|
@ -16370,8 +16374,8 @@
|
|||
"occurrences": 1,
|
||||
"error_message": "resreq parser ignores a primary's select/rusage parse error and accepts the job without that clause\n\n**Observed:** (code reading) the `primary_resreq` action in\nsrc/lib/xce_resreq.peg calls\n`parse_resreq_primary(primary, $0, auxil->resmgr, &(auxil->errmsg))`\nand ignores its return value; nothing sets `has_error`, so\n`parse_resreq()` returns success. A primary whose select or rusage clause\nfails to parse is kept without that clause (select NULL / rusages empty),\ni.e. the job is scheduled as if it had no such constraint. While fixing\nbug-1884 a LeakSanitizer report showed such an error message being\nproduced during a normal replay fixture (its job's resreq parsed \"OK\").\n\n**Expected:** a primary sub-parse error makes `parse_resreq()` fail with that message (the job is rejected/pended with a parse error), not silently dropped.\n**Impact:** wrong result: a job with a malformed select (e.g. a typo) may dispatch onto hosts its select would have excluded.\n**Found by:** review (while fixing bug-1884, 2026-09-29)",
|
||||
"file": "src/lib/xce_resreq.peg, src/lib/resreq.c",
|
||||
"root_cause": "TODO: required for fixed/partial",
|
||||
"fix": "TODO: required for fixed/partial\n\nVerification:\n- Test: TODO: name + file; RED before the fix, GREEN after\n- Suites: TODO: suite results and build profile\n- Live: TODO: soak/deploy run, or n/a and why",
|
||||
"root_cause": "The `primary_resreq` action in src/lib/xce_resreq.peg ignored `parse_resreq_primary()`'s result and nothing set `has_error`, so `parse_resreq()` succeeded for a primary whose select/rusage clause failed to parse and the job kept no such clause. Confirmed with a probe (2026-09-29): `select[type==]`, `select[mem>]`, `select[type==any && ]` all returned rc=1 with select NULL; `select[gpu>1]` with no gpu resource likewise.",
|
||||
"fix": "A malformed clause now fails the whole resreq and `parse_resreq()` returns the clause's own message (b5712c05). A select naming an unregistered resource is still dropped rather than rejected: parse_select() reports it as -ENOENT (new select_context_t.unknown_resource) and the action lets it pass, because jobs rely on it -- the scheduler's own \"type == any\" default among them when no \"type\" resource is registered (making it fatal failed 36 scheduler suites). Whether an unknown resource in a select should reject the job is left as a separate decision. Making syntax errors fatal exposed bug-1888.\n\nVerification:\n- Test: test_resreq_primary_select_error_fails_parse in src/lib/testsuite/test_resreq.c (xkvmgr resmgr with type/mem; syntax errors incl. inside a compound fail with a message; unknown resource still parses with select NULL; valid select kept): RED before (all accepted), GREEN after\n- Suites: ASAN debug build (src/xce.build.discrete), `meson test --no-suite xceadmin`: 117/117 OK, 0 sanitizer reports; xcelib_test 695 tests 0 failures (2026-09-29)\n- Live: 3-round sequential HA soak on rebuilt ASAN binaries (/home/claude/tmp/xce-bug1885-soak, 2026-09-29): PASS, 15 jobs, zero job loss, all 3 nodes led; SIMVERIFY 4 files 0 mismatches; each recording replays with detect_leaks=1 at exit 0, 0 mismatches; no sanitizer report in the scheduler or controller stderr",
|
||||
"tags": [
|
||||
"lib",
|
||||
"resreq",
|
||||
|
|
@ -16380,8 +16384,63 @@
|
|||
"related_bugs": [
|
||||
"bug-1884"
|
||||
],
|
||||
"fix_commits": [],
|
||||
"fix_commits": [
|
||||
"b5712c05"
|
||||
],
|
||||
"record": ".wolf/bugs/bug-1887.md"
|
||||
},
|
||||
{
|
||||
"id": "bug-1888",
|
||||
"status": "fixed",
|
||||
"status_updated": "2026-09-29",
|
||||
"status_note": "",
|
||||
"component": "scheduler",
|
||||
"severity": "wrong-result",
|
||||
"timestamp": "2026-09-29T10:32:13Z",
|
||||
"last_seen": "2026-09-29T10:32:13Z",
|
||||
"occurrences": 1,
|
||||
"error_message": "snapshots save a job's resreq as a debug dump; restored jobs lose select and rusage\n\n**Observed:** saving a snapshot of a pending job whose resreq is `rusage[mem=2]` stores `submit->resreq` as\n```\nPrimary:\n Tasks: min = 1, max = 1\n Rusages:\n res mem value 2.000000 per 3 duration -1.000000 decay 0\n```\nand the loaded job has no rusage (and no select). With bug-1887 fixed the loader rejects that text outright (snapshot tests failed: \"parse_resreq failed for jobid 201: Primary: ...\").\n\n**Expected:** the snapshot stores the resreq text the job was submitted with, and the restored job parses to the same request.\n**Impact:** wrong result: every job restored from a snapshot -- a `.replay` start snapshot (simulator replays), REPL snapshots -- lost its select and rusage, so replays of sessions with pending jobs at snapshot time could decide differently from the live scheduler. A save/restore field-completeness violation (persistence.md).\n**Found by:** test (while fixing bug-1887, 2026-09-29)",
|
||||
"file": "src/scheduler/src/snapshot_wire.c, src/scheduler/src/snapshot_load.c",
|
||||
"root_cause": "`build_wire_job()` (snapshot_wire.c) filled `submit->resreq` with `format_resreq()`, a human-readable debug dump, because the scheduler job kept only the parsed resreq, not its text. The loader's parser accepted the dump as a primary with neither select nor rusage (the bug-1887 leniency).",
|
||||
"fix": "`xce_sched_job_t` keeps the resreq text (`resreq_str`, set at ingest, JOB_UPDATE and snapshot load, freed at teardown) and the snapshot saves it; a job without it restores as the loader's \"type == any\" default (077b1213).\n\nVerification:\n- Test: test_snapshot_round_trip_keeps_job_resreq_text in src/scheduler/testsuite/test_snapshot_save_load.c (push a job with rusage[mem=2], save, load into a second core): RED before (saved text was the dump), GREEN after (text kept, restored rusage count 1)\n- Suites: ASAN debug build (src/xce.build.discrete), `meson test --no-suite xceadmin`: 117/117 OK, 0 sanitizer reports; xcelib_test 695 tests 0 failures (2026-09-29)\n- Live: 3-round sequential HA soak on rebuilt ASAN binaries (/home/claude/tmp/xce-bug1885-soak, 2026-09-29): PASS, 15 jobs, zero job loss, all 3 nodes led; SIMVERIFY 4 files 0 mismatches; each recording replays with detect_leaks=1 at exit 0, 0 mismatches; no sanitizer report in the scheduler or controller stderr",
|
||||
"tags": [
|
||||
"scheduler",
|
||||
"snapshot",
|
||||
"persistence",
|
||||
"resreq"
|
||||
],
|
||||
"related_bugs": [
|
||||
"bug-1887"
|
||||
],
|
||||
"fix_commits": [
|
||||
"077b1213"
|
||||
],
|
||||
"record": ".wolf/bugs/bug-1888.md"
|
||||
},
|
||||
{
|
||||
"id": "bug-1889",
|
||||
"status": "fixed",
|
||||
"status_updated": "2026-09-29",
|
||||
"status_note": "",
|
||||
"component": "lib",
|
||||
"severity": "wrong-result",
|
||||
"timestamp": "2026-09-29T10:32:13Z",
|
||||
"last_seen": "2026-09-29T10:32:13Z",
|
||||
"occurrences": 1,
|
||||
"error_message": "xcelib_test exits 0 on failures: nested UNITY_BEGIN in test_libxce_retry resets counts\n\n**Observed:** `xcelib_test` printed\n```\n../lib/testsuite/test.c:479:test_fmt_limit_units_percent_limit_ignores_units_mode:FAIL: a percentage-based limit's value must render identically regardless of units_mode ...\n56 Tests 0 Failures 0 Ignored\n```\nand exited 0, so meson reported the libxce suite OK on every run.\n\n**Expected:** a failing test fails xcelib_test and the suite.\n**Impact:** wrong result (test harness): any failure in a test group run before test_libxce_retry was silently dropped. One real failure was hidden: fmt_limit_units() printed a 0.8 percent limit as \"80.000001%\".\n**Found by:** test (reported by the user as a failing test, 2026-09-29)",
|
||||
"file": "src/lib/testsuite/test_libxce_retry.c, src/lib/testsuite/test.c, src/lib/xcelimit.c",
|
||||
"root_cause": "`test_libxce_retry()`, one group of test.c's single runner, called `UNITY_BEGIN()`/`UNITY_END()` itself; `UNITY_BEGIN()` resets Unity's test and failure counters, so failures from earlier groups were lost and the final `UNITY_END()` returned 0. The hidden failure: fmt_limit_units() widened the float limit value before multiplying by 100 (0.8f -> 0.800000011920929 -> \"80.000001%\").",
|
||||
"fix": "Removed the nested UNITY_BEGIN/UNITY_END (the runner now reports 695 tests; this was the only hidden failure), and fmt_limit_units() scales in float then widens, giving the configured 80 (9c51de88). A scan of every testsuite found no other nested runner.\n\nVerification:\n- Test: test_fmt_limit_units_percent_limit_ignores_units_mode in src/lib/testsuite/test_limit.c: FAIL (now counted: 695 tests, 1 failure, exit 1) before the formatter fix, PASS after\n- Suites: ASAN debug build (src/xce.build.discrete), `meson test --no-suite xceadmin`: 117/117 OK, 0 sanitizer reports; xcelib_test 695 tests 0 failures (2026-09-29)\n- Live: 3-round sequential HA soak on rebuilt ASAN binaries (/home/claude/tmp/xce-bug1885-soak, 2026-09-29): PASS, 15 jobs, zero job loss, all 3 nodes led; SIMVERIFY 4 files 0 mismatches; each recording replays with detect_leaks=1 at exit 0, 0 mismatches; no sanitizer report in the scheduler or controller stderr",
|
||||
"tags": [
|
||||
"lib",
|
||||
"test",
|
||||
"harness"
|
||||
],
|
||||
"related_bugs": [],
|
||||
"fix_commits": [
|
||||
"9c51de88"
|
||||
],
|
||||
"record": ".wolf/bugs/bug-1889.md"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,8 +1,8 @@
|
|||
---
|
||||
id: "bug-1885"
|
||||
status: "open"
|
||||
status: "fixed"
|
||||
status_updated: "2026-09-29"
|
||||
status_note: "newly logged"
|
||||
status_note: ""
|
||||
component: "scheduler"
|
||||
severity: "leak"
|
||||
timestamp: "2026-09-29T09:48:44Z"
|
||||
|
|
@ -11,6 +11,7 @@ occurrences: 1
|
|||
files: ["src/scheduler/src/driver.c", "src/scheduler/src/decision.c"]
|
||||
tags: ["scheduler", "leak", "decision"]
|
||||
related: ["bug-1884"]
|
||||
fix_commits: ["d21b3d8f"]
|
||||
---
|
||||
|
||||
# bug-1885: scheduler normal mode never frees decision structs: 64 bytes leaked per published decision
|
||||
|
|
@ -46,15 +47,15 @@ cbuf_get(&(sched->core.decisions), &decision);
|
|||
- Evidence: not kept; code reading
|
||||
|
||||
## Root cause
|
||||
TODO: required for fixed/partial
|
||||
`xce_sched_driver_run_normal_mode_round()` (driver.c) drains each decision from `core->decisions`, records it and passes it to `on_decision`, and never frees the struct `make_*_decision()` (decision.c) allocates.
|
||||
|
||||
## Fix
|
||||
TODO: required for fixed/partial
|
||||
The round frees the decision struct after the callback returns (d21b3d8f). Its allocation/pend reason belong to the job; the publish callback passes those pointers on and the recorder copies, so nothing keeps the struct. Tests with their own drain loops (test_core_full_cycle.c, test_core_session.c, test_snapshot_save_load.c) are unaffected.
|
||||
|
||||
## Verification
|
||||
- Test: TODO: name + file; RED before the fix, GREEN after
|
||||
- Suites: TODO: suite results and build profile
|
||||
- Live: TODO: soak/deploy run, or n/a and why
|
||||
- Test: test_round_frees_decision_after_callback in src/scheduler/testsuite/test_driver_session_ordering.c (drives the real round with a stand-in allocator emitting one PEND; checks the struct is freed via __asan_address_is_poisoned and the reason is not): RED before, GREEN after
|
||||
- Suites: ASAN debug build (src/xce.build.discrete), `meson test --no-suite xceadmin`: 117/117 OK, 0 sanitizer reports; xcelib_test 695 tests 0 failures (2026-09-29)
|
||||
- Live: 3-round sequential HA soak on rebuilt ASAN binaries (/home/claude/tmp/xce-bug1885-soak, 2026-09-29): PASS, 15 jobs, zero job loss, all 3 nodes led; SIMVERIFY 4 files 0 mismatches; each recording replays with detect_leaks=1 at exit 0, 0 mismatches; no sanitizer report in the scheduler or controller stderr
|
||||
|
||||
## References
|
||||
- Related: bug-1884
|
||||
|
|
|
|||
|
|
@ -1,8 +1,8 @@
|
|||
---
|
||||
id: "bug-1886"
|
||||
status: "open"
|
||||
status: "fixed"
|
||||
status_updated: "2026-09-29"
|
||||
status_note: "newly logged"
|
||||
status_note: ""
|
||||
component: "scheduler"
|
||||
severity: "leak"
|
||||
timestamp: "2026-09-29T09:48:47Z"
|
||||
|
|
@ -11,6 +11,7 @@ occurrences: 1
|
|||
files: ["src/scheduler/src/scheduler.c", "src/scheduler/src/core_events.c"]
|
||||
tags: ["scheduler", "leak", "config"]
|
||||
related: ["bug-1884"]
|
||||
fix_commits: ["3e873f0a"]
|
||||
---
|
||||
|
||||
# bug-1886: scheduler leaks the rejected queue on each config re-apply and the old config on each CONFIG_PUSH
|
||||
|
|
@ -44,15 +45,15 @@ related: ["bug-1884"]
|
|||
- Evidence: not kept; code reading plus the test_config_push_deferred UAF seen with a freeing killer
|
||||
|
||||
## Root cause
|
||||
TODO: required for fixed/partial
|
||||
`sched_init_queues()` built a new `xce_sched_queue_t` for every configured queue on each config apply; for an existing name `sched_add_queue()` failed and the unregistered queue was handed to the picker (which then tracked it instead of the registered one) and leaked, while the registered queue kept its old settings. `sched_init_config()` and the core's CONFIG_PUSH replaced `core->config` without freeing the previous one.
|
||||
|
||||
## Fix
|
||||
TODO: required for fixed/partial
|
||||
`sched_init_queues()` updates the registered queue in place (attrs, status, priority, accept_interval, sched_delay -- what make_sched_queue() copies), as QUEUE_UPDATE's core_upsert_queue() does; a replaced config is freed after the new one is applied -- nothing points into it (queues, resources, groups and limits built from it are copies) (3e873f0a).
|
||||
|
||||
## Verification
|
||||
- Test: TODO: name + file; RED before the fix, GREEN after
|
||||
- Suites: TODO: suite results and build profile
|
||||
- Live: TODO: soak/deploy run, or n/a and why
|
||||
- Test: test_config_reapply_updates_queue_in_place_and_frees_old in src/scheduler/testsuite/test_config_push_deferred.c (applies a one-queue config twice, priority 1 then 7): RED before (priority stayed 1), GREEN after (same queue object, priority 7, first config freed per ASAN)
|
||||
- Suites: ASAN debug build (src/xce.build.discrete), `meson test --no-suite xceadmin`: 117/117 OK, 0 sanitizer reports; xcelib_test 695 tests 0 failures (2026-09-29)
|
||||
- Live: 3-round sequential HA soak on rebuilt ASAN binaries (/home/claude/tmp/xce-bug1885-soak, 2026-09-29): PASS, 15 jobs, zero job loss, all 3 nodes led; SIMVERIFY 4 files 0 mismatches; each recording replays with detect_leaks=1 at exit 0, 0 mismatches; no sanitizer report in the scheduler or controller stderr
|
||||
|
||||
## References
|
||||
- Related: bug-1884
|
||||
|
|
|
|||
|
|
@ -1,8 +1,8 @@
|
|||
---
|
||||
id: "bug-1887"
|
||||
status: "open"
|
||||
status: "fixed"
|
||||
status_updated: "2026-09-29"
|
||||
status_note: "newly logged"
|
||||
status_note: ""
|
||||
component: "lib"
|
||||
severity: "wrong-result"
|
||||
timestamp: "2026-09-29T09:48:49Z"
|
||||
|
|
@ -11,6 +11,7 @@ occurrences: 1
|
|||
files: ["src/lib/xce_resreq.peg", "src/lib/resreq.c"]
|
||||
tags: ["lib", "resreq", "parser"]
|
||||
related: ["bug-1884"]
|
||||
fix_commits: ["b5712c05"]
|
||||
---
|
||||
|
||||
# bug-1887: resreq parser ignores a primary's select/rusage parse error and accepts the job without that clause
|
||||
|
|
@ -45,15 +46,15 @@ produced during a normal replay fixture (its job's resreq parsed "OK").
|
|||
- Evidence: not kept; LeakSanitizer stack from the bug-1884 fixture (strdup in parse_select error path under core_build_job)
|
||||
|
||||
## Root cause
|
||||
TODO: required for fixed/partial
|
||||
The `primary_resreq` action in src/lib/xce_resreq.peg ignored `parse_resreq_primary()`'s result and nothing set `has_error`, so `parse_resreq()` succeeded for a primary whose select/rusage clause failed to parse and the job kept no such clause. Confirmed with a probe (2026-09-29): `select[type==]`, `select[mem>]`, `select[type==any && ]` all returned rc=1 with select NULL; `select[gpu>1]` with no gpu resource likewise.
|
||||
|
||||
## Fix
|
||||
TODO: required for fixed/partial
|
||||
A malformed clause now fails the whole resreq and `parse_resreq()` returns the clause's own message (b5712c05). A select naming an unregistered resource is still dropped rather than rejected: parse_select() reports it as -ENOENT (new select_context_t.unknown_resource) and the action lets it pass, because jobs rely on it -- the scheduler's own "type == any" default among them when no "type" resource is registered (making it fatal failed 36 scheduler suites). Whether an unknown resource in a select should reject the job is left as a separate decision. Making syntax errors fatal exposed bug-1888.
|
||||
|
||||
## Verification
|
||||
- Test: TODO: name + file; RED before the fix, GREEN after
|
||||
- Suites: TODO: suite results and build profile
|
||||
- Live: TODO: soak/deploy run, or n/a and why
|
||||
- Test: test_resreq_primary_select_error_fails_parse in src/lib/testsuite/test_resreq.c (xkvmgr resmgr with type/mem; syntax errors incl. inside a compound fail with a message; unknown resource still parses with select NULL; valid select kept): RED before (all accepted), GREEN after
|
||||
- Suites: ASAN debug build (src/xce.build.discrete), `meson test --no-suite xceadmin`: 117/117 OK, 0 sanitizer reports; xcelib_test 695 tests 0 failures (2026-09-29)
|
||||
- Live: 3-round sequential HA soak on rebuilt ASAN binaries (/home/claude/tmp/xce-bug1885-soak, 2026-09-29): PASS, 15 jobs, zero job loss, all 3 nodes led; SIMVERIFY 4 files 0 mismatches; each recording replays with detect_leaks=1 at exit 0, 0 mismatches; no sanitizer report in the scheduler or controller stderr
|
||||
|
||||
## References
|
||||
- Related: bug-1884
|
||||
|
|
|
|||
59
.wolf/bugs/bug-1888.md
Normal file
59
.wolf/bugs/bug-1888.md
Normal file
|
|
@ -0,0 +1,59 @@
|
|||
---
|
||||
id: "bug-1888"
|
||||
status: "fixed"
|
||||
status_updated: "2026-09-29"
|
||||
status_note: ""
|
||||
component: "scheduler"
|
||||
severity: "wrong-result"
|
||||
timestamp: "2026-09-29T10:32:13Z"
|
||||
last_seen: "2026-09-29T10:32:13Z"
|
||||
occurrences: 1
|
||||
files: ["src/scheduler/src/snapshot_wire.c", "src/scheduler/src/snapshot_load.c"]
|
||||
tags: ["scheduler", "snapshot", "persistence", "resreq"]
|
||||
related: ["bug-1887"]
|
||||
fix_commits: ["077b1213"]
|
||||
---
|
||||
|
||||
# bug-1888: snapshots save a job's resreq as a debug dump; restored jobs lose select and rusage
|
||||
|
||||
## Description
|
||||
**Observed:** saving a snapshot of a pending job whose resreq is `rusage[mem=2]` stores `submit->resreq` as
|
||||
```
|
||||
Primary:
|
||||
Tasks: min = 1, max = 1
|
||||
Rusages:
|
||||
res mem value 2.000000 per 3 duration -1.000000 decay 0
|
||||
```
|
||||
and the loaded job has no rusage (and no select). With bug-1887 fixed the loader rejects that text outright (snapshot tests failed: "parse_resreq failed for jobid 201: Primary: ...").
|
||||
|
||||
**Expected:** the snapshot stores the resreq text the job was submitted with, and the restored job parses to the same request.
|
||||
**Impact:** wrong result: every job restored from a snapshot -- a `.replay` start snapshot (simulator replays), REPL snapshots -- lost its select and rusage, so replays of sessions with pending jobs at snapshot time could decide differently from the live scheduler. A save/restore field-completeness violation (persistence.md).
|
||||
**Found by:** test (while fixing bug-1887, 2026-09-29)
|
||||
|
||||
## Environment
|
||||
- OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
|
||||
- libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2)
|
||||
- Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
|
||||
- Build: asan (src/xce.build.discrete)
|
||||
- Code: scheduler-framework @ 9c51de88-dirty
|
||||
- Deployment: n/a (unit test)
|
||||
|
||||
## Reproduction
|
||||
1. Build the scheduler tests; run `src/xce.build.discrete/scheduler/testsuite/sched_snapshot_save_load_test` at a commit before 077b1213 with test_snapshot_round_trip_keeps_job_resreq_text added.
|
||||
2. Observe the saved text is the dump, not `rusage[mem=2]`.
|
||||
- Frequency: deterministic
|
||||
- Evidence: test output quoted above
|
||||
|
||||
## Root cause
|
||||
`build_wire_job()` (snapshot_wire.c) filled `submit->resreq` with `format_resreq()`, a human-readable debug dump, because the scheduler job kept only the parsed resreq, not its text. The loader's parser accepted the dump as a primary with neither select nor rusage (the bug-1887 leniency).
|
||||
|
||||
## Fix
|
||||
`xce_sched_job_t` keeps the resreq text (`resreq_str`, set at ingest, JOB_UPDATE and snapshot load, freed at teardown) and the snapshot saves it; a job without it restores as the loader's "type == any" default (077b1213).
|
||||
|
||||
## Verification
|
||||
- Test: test_snapshot_round_trip_keeps_job_resreq_text in src/scheduler/testsuite/test_snapshot_save_load.c (push a job with rusage[mem=2], save, load into a second core): RED before (saved text was the dump), GREEN after (text kept, restored rusage count 1)
|
||||
- Suites: ASAN debug build (src/xce.build.discrete), `meson test --no-suite xceadmin`: 117/117 OK, 0 sanitizer reports; xcelib_test 695 tests 0 failures (2026-09-29)
|
||||
- Live: 3-round sequential HA soak on rebuilt ASAN binaries (/home/claude/tmp/xce-bug1885-soak, 2026-09-29): PASS, 15 jobs, zero job loss, all 3 nodes led; SIMVERIFY 4 files 0 mismatches; each recording replays with detect_leaks=1 at exit 0, 0 mismatches; no sanitizer report in the scheduler or controller stderr
|
||||
|
||||
## References
|
||||
- Related: none
|
||||
57
.wolf/bugs/bug-1889.md
Normal file
57
.wolf/bugs/bug-1889.md
Normal file
|
|
@ -0,0 +1,57 @@
|
|||
---
|
||||
id: "bug-1889"
|
||||
status: "fixed"
|
||||
status_updated: "2026-09-29"
|
||||
status_note: ""
|
||||
component: "lib"
|
||||
severity: "wrong-result"
|
||||
timestamp: "2026-09-29T10:32:13Z"
|
||||
last_seen: "2026-09-29T10:32:13Z"
|
||||
occurrences: 1
|
||||
files: ["src/lib/testsuite/test_libxce_retry.c", "src/lib/testsuite/test.c", "src/lib/xcelimit.c"]
|
||||
tags: ["lib", "test", "harness"]
|
||||
related: []
|
||||
fix_commits: ["9c51de88"]
|
||||
---
|
||||
|
||||
# bug-1889: xcelib_test exits 0 on failures: nested UNITY_BEGIN in test_libxce_retry resets counts
|
||||
|
||||
## Description
|
||||
**Observed:** `xcelib_test` printed
|
||||
```
|
||||
../lib/testsuite/test.c:479:test_fmt_limit_units_percent_limit_ignores_units_mode:FAIL: a percentage-based limit's value must render identically regardless of units_mode ...
|
||||
56 Tests 0 Failures 0 Ignored
|
||||
```
|
||||
and exited 0, so meson reported the libxce suite OK on every run.
|
||||
|
||||
**Expected:** a failing test fails xcelib_test and the suite.
|
||||
**Impact:** wrong result (test harness): any failure in a test group run before test_libxce_retry was silently dropped. One real failure was hidden: fmt_limit_units() printed a 0.8 percent limit as "80.000001%".
|
||||
**Found by:** test (reported by the user as a failing test, 2026-09-29)
|
||||
|
||||
## Environment
|
||||
- OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
|
||||
- libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2)
|
||||
- Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
|
||||
- Build: asan (src/xce.build.discrete)
|
||||
- Code: scheduler-framework @ 9c51de88-dirty
|
||||
- Deployment: n/a (unit test)
|
||||
|
||||
## Reproduction
|
||||
1. `src/xce.build.discrete/lib/xcelib_test; echo $?` at a commit before 9c51de88.
|
||||
2. Observe the FAIL line, "0 Failures" and exit 0.
|
||||
- Frequency: deterministic
|
||||
- Evidence: output quoted above
|
||||
|
||||
## Root cause
|
||||
`test_libxce_retry()`, one group of test.c's single runner, called `UNITY_BEGIN()`/`UNITY_END()` itself; `UNITY_BEGIN()` resets Unity's test and failure counters, so failures from earlier groups were lost and the final `UNITY_END()` returned 0. The hidden failure: fmt_limit_units() widened the float limit value before multiplying by 100 (0.8f -> 0.800000011920929 -> "80.000001%").
|
||||
|
||||
## Fix
|
||||
Removed the nested UNITY_BEGIN/UNITY_END (the runner now reports 695 tests; this was the only hidden failure), and fmt_limit_units() scales in float then widens, giving the configured 80 (9c51de88). A scan of every testsuite found no other nested runner.
|
||||
|
||||
## Verification
|
||||
- Test: test_fmt_limit_units_percent_limit_ignores_units_mode in src/lib/testsuite/test_limit.c: FAIL (now counted: 695 tests, 1 failure, exit 1) before the formatter fix, PASS after
|
||||
- Suites: ASAN debug build (src/xce.build.discrete), `meson test --no-suite xceadmin`: 117/117 OK, 0 sanitizer reports; xcelib_test 695 tests 0 failures (2026-09-29)
|
||||
- Live: 3-round sequential HA soak on rebuilt ASAN binaries (/home/claude/tmp/xce-bug1885-soak, 2026-09-29): PASS, 15 jobs, zero job loss, all 3 nodes led; SIMVERIFY 4 files 0 mismatches; each recording replays with detect_leaks=1 at exit 0, 0 mismatches; no sanitizer report in the scheduler or controller stderr
|
||||
|
||||
## References
|
||||
- Related: none
|
||||
Loading…
Add table
Add a link
Reference in a new issue