Rongsong Shen
1f932bf22d
Feature validation GO (ASAN scheduler suite 96/96, smoke replay clean on the allocation path). Roadmap entry marked implemented with follow-ups. bug-1897 (found by the validation smoke run, not caused by this feature): xce_scheduler --sim segfaults in limit_job_election on a job whose snapshot resreq failed to parse (pre-bug-1888-fix replays). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
62 lines
3.4 KiB
Markdown
62 lines
3.4 KiB
Markdown
---
|
|
id: "bug-1897"
|
|
status: "open"
|
|
status_updated: "2026-09-30"
|
|
status_note: "newly logged"
|
|
component: "scheduler"
|
|
severity: "crash"
|
|
timestamp: "2026-09-30T10:19:59Z"
|
|
last_seen: "2026-09-30T10:19:59Z"
|
|
occurrences: 1
|
|
files: ["src/scheduler/src/plugins/limit.c", "src/scheduler/src/snapshot_load.c"]
|
|
tags: ["scheduler", "sim", "snapshot", "limit"]
|
|
related: ["bug-1887", "bug-1888"]
|
|
---
|
|
|
|
# bug-1897: xce_scheduler --sim segfaults in limit_job_election on a job whose snapshot resreq failed to parse
|
|
|
|
## Description
|
|
**Observed:** replaying `data/scheduler-replay-20260928-064842.replay` in simulation mode crashes before any decision, 3/3 runs:
|
|
```
|
|
[ERROR] [xce_sched_core_load_snapshot] xce_sched_core_load_snapshot parse_resreq failed for jobid 4667606771186008064: Primary: ...
|
|
==2723571==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000008 (pc 0x555555a7cb6e ... T3)
|
|
#0 limit_job_election ../scheduler/src/plugins/limit.c:336
|
|
#1 plugin_job_election ../scheduler/src/plugin.c:601
|
|
#2 sched_allocator ../scheduler/src/sched_election.c:236
|
|
```
|
|
The replay's snapshot stores resreqs in the debug-dump format that bug-1888 wrote before its fix. Since the bug-1887 fix, `parse_resreq` rejects that text, so the snapshot loader logs the error but keeps the job with no schedobj. The allocator thread's job election then reads `job->schedobj` as NULL in the limit plugin.
|
|
|
|
**Expected:** a job whose resreq cannot be parsed from a snapshot is either dropped (with the error logged) or kept pending with a reason, and never reaches election with a NULL schedobj; the simulator must not crash on an old replay.
|
|
|
|
**Impact:** crash (SIGSEGV) of `xce_scheduler --sim` on any replay whose snapshot has unparsable resreqs, including every replay captured before the bug-1888 fix. Suspected: the same path could crash a normal-mode scheduler loading such a snapshot (not tried).
|
|
|
|
**Found by:** feature-level validation smoke run of spec `scheduler-allocator-runner` (2026-09-30); not caused by that feature (limit.c, snapshot_load.c, sched_election.c untouched by it).
|
|
|
|
## Environment
|
|
- OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
|
|
- libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2)
|
|
- Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
|
|
- Build: asan (src/build.alloc, dedicated scheduler-allocator-runner build dir); ASAN_OPTIONS=detect_leaks=0
|
|
- Code: scheduler-framework @ 28cff179-dirty
|
|
- Deployment: none (simulation mode, `xce_scheduler --sim`, no controller or workers)
|
|
|
|
## Reproduction
|
|
1. `git checkout 28cff179` (or later); `bash src/tools/xce-build.sh scheduler asan build.alloc --jobs 8`
|
|
2. `cd src && nix develop . --command bash -c 'ASAN_OPTIONS=detect_leaks=0 ./build.alloc/scheduler/xce_scheduler --sim=/home/claude/tmp/bug-1897/scheduler-replay-20260928-064842.replay -l /tmp/sim.log'`
|
|
3. Observe `parse_resreq failed` errors in the log, then the ASan SEGV in `limit_job_election`.
|
|
- Frequency: deterministic (3/3 runs, 2026-09-30)
|
|
- Evidence: `/home/claude/tmp/bug-1897/` (replay file copy, `run.out` with the ASan report, `sim.log`)
|
|
|
|
## Root cause
|
|
TODO: required for fixed/partial
|
|
|
|
## Fix
|
|
TODO: required for fixed/partial
|
|
|
|
## Verification
|
|
- Test: TODO: name + file; RED before the fix, GREEN after
|
|
- Suites: TODO: suite results and build profile
|
|
- Live: TODO: soak/deploy run, or n/a and why
|
|
|
|
## References
|
|
- Related: none
|