From fcd9fc7666ec1c8c3e45e23029aa4be077e9c489 Mon Sep 17 00:00:00 2001 From: Rongsong Shen Date: Tue, 29 Sep 2026 08:20:55 +0000 Subject: [PATCH] chore(buglog): bug-1884 logged (ASAN --sim exits 1 on exit-time LeakSanitizer report) Co-Authored-By: Claude Opus 5.5 (1M context) --- .wolf/bugs/bug-1884.md | 78 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 78 insertions(+) create mode 100644 .wolf/bugs/bug-1884.md diff --git a/.wolf/bugs/bug-1884.md b/.wolf/bugs/bug-1884.md new file mode 100644 index 00000000..92d1f73d --- /dev/null +++ b/.wolf/bugs/bug-1884.md @@ -0,0 +1,78 @@ +--- +id: "bug-1884" +status: "open" +status_updated: "2026-09-29" +status_note: "newly logged" +component: "scheduler" +severity: "leak" +timestamp: "2026-09-29T08:20:03Z" +last_seen: "2026-09-29T08:20:03Z" +occurrences: 1 +files: ["src/scheduler/src/sim_loop.c", "src/scheduler/src/sim.c", "src/scheduler/src/plugins/limit.c", "src/scheduler/src/main.c"] +tags: ["scheduler", "sim", "asan", "lsan", "leak"] +related: ["bug-1632", "bug-1763"] +--- + +# bug-1884: standalone ASAN xce_scheduler --sim exits 1: LeakSanitizer reports 17 MB still allocated at exit + +## Description +**Observed:** an ASAN build of `xce_scheduler --sim= --sim-verify` +replays the whole recording (0 MISMATCH lines) and then exits 1 because +LeakSanitizer reports at exit: +``` +==2234882==ERROR: LeakSanitizer: detected memory leaks +... +SUMMARY: AddressSanitizer: 17041405 byte(s) leaked in 1187 allocation(s). +``` +Of the 17,041,405 bytes, 17,036,261 are indirect and 5,144 direct. +Grouped by first scheduler/lib frame (bytes, leak groups): +- `kvtab_init` (lib/kv.c:106): 16,941,432 in 15 -- core kv tables, reachable only from leaked roots +- `sim_process_replay_event_message` (sim_loop.c:403): 49,688 in 54 -- decoded replay events (e.g. `decode_JobStateChangeEvent_ptr`) +- `xkvmgr_add` (lib/kvmgr.c:114): 21,376 in 9 +- `sim_locate_and_load_snapshot` (sim.c:224): 14,347 in 106 -- snapshot decode (`decode_Conf4Sched` / `decode_QueueConf_list`) +- `put_resvals` (resource.c:99) via limit.c `update_limit_res_value` and `sched_update_res_accounts`: 2,048 in 2 +- smaller: `parse_resreq`, `core_build_job`, `sched_ctx_init`, `match_job_host_limit_` + +The indirect bulk fits the simulator never tearing down its core at exit +(its picker/allocator threads are never joined, see bug-1632/bug-1763); +the direct groups may include real per-event leaks. + +**Expected:** a clean replay exits 0 under ASAN with LeakSanitizer enabled, +or the exit-time teardown frees what the driver owns. +**Impact:** leak; for scripts, any ASAN `--sim` replay must set +`ASAN_OPTIONS=detect_leaks=0` or its exit code is always 1, which masks real +failures and hides any genuine per-event leak among the exit-time noise. +Release builds are unaffected at exit; a per-event leak would still grow a +long replay's memory. +**Found by:** spec task (libxce-alloc-preason-equality 5.3, live +default-settings replay, 2026-09-29) + +## Environment +- OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64 +- libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2) +- Toolchain: gcc (GCC) 15.2.0, meson 1.9.1 +- Build: asan (src/xce.build.discrete, debug + b_sanitize=address) +- Code: scheduler-framework @ 4500ec46-dirty +- Deployment: n/a (standalone simulation mode, no cluster); the replay file was recorded by a local 3-node HA soak (run-ha-soak.sh, 2 sequential rounds, xcecd_virtual worker) at /home/claude/tmp/xce-alloceq-53 + +## Reproduction +1. `bash src/tools/xce-build.sh scheduler asan` (or any ASAN build dir) +2. `src/xce.build/scheduler/xce_scheduler --sim=/home/claude/tmp/bug-1884/scheduler-replay-20260929-064412.replay --sim-verify --logfile /tmp/sim.log; echo rc=$?` +3. Observe `rc=1` and the LeakSanitizer SUMMARY on stderr. With `ASAN_OPTIONS=detect_leaks=0` the same run exits 0. +- Frequency: deterministic (2/2 runs, identical totals, 2026-09-29); also seen on every recording of /home/claude/tmp/xce-ha-dc-soak +- Evidence: /home/claude/tmp/bug-1884/lsan-report.txt (full report), /home/claude/tmp/bug-1884/scheduler-replay-20260929-064412.replay + +## Root cause +TODO: required for fixed/partial + +## Fix +TODO: required for fixed/partial + +## Verification +- Test: TODO: name + file; RED before the fix, GREEN after +- Suites: TODO: suite results and build profile +- Live: TODO: soak/deploy run, or n/a and why + +## References +- Related: bug-1632, bug-1763 (sim picker/allocator threads never joined) +- run-ha-soak.sh replay gate already runs `--sim` with `ASAN_OPTIONS=detect_leaks=0`