Rongsong Shen
d26ff8870d
The buglog was one 27K-line JSON file whose fields covered only symptom,
root cause and fix, as free text: 2% of entries named the code version a
bug was found on, 12% the build profile, 36% a way to reproduce, and 58% of
the file was OpenWolf hook auto-detections ("Missing guard clause",
"Function not marked async" on .md files).
- .claude/skills/buglog/bug-record.md: the record standard. Eight
sections (Summary as the title, Description, Environment, Reproduction;
Root cause, Fix, Verification, References once fixed), each with the
content it must contain -- e.g. Environment needs the OS, the libc the
failing binary loads (from its ELF interpreter: glibc 2.42 in the nix
shell vs 2.40 on the host), toolchain, build profile and commit.
- .wolf/bugs/<id>.md: 500 records migrated from buglog.json; every field
verified to round-trip. They carry legacy: true and an explicit "not
recorded" where the old log had no such field; component derived from
file paths, severity from keywords (else unknown), fix_commits from
hashes in status_note/fix that git resolves (91 of 463 fixed).
- .wolf/buglog-archive.json: the 710 hook auto-detections, verbatim.
- .wolf/buglog.json is now an index generated from the records, keeping
the fields OpenWolf's hooks and `openwolf bug search` read.
- scripts/buglog.py rewritten: new (template + Environment collected from
the machine and --binary), check, status (refuses a status the record
doesn't support; fixed needs Root cause, Fix, Verification and
--commit hashes git resolves), note, recur, set, show, list, triage,
env, index. 29 unittest cases in scripts/tests/test_buglog.py.
- scripts/git-hooks/pre-commit runs `check --staged`: incomplete staged
records, a legacy record hand-changed to fixed without the fix-time
sections, a new record claiming legacy: true, or a stale index refuse
the commit.
- scripts/gitea-sync.py reads the records: issue body = the record's
sections plus linked fix commits (origin remote, GITEA_COMMIT_REPO to
override), and a body hash so an issue's text is refreshed when the
record changes.
- bug-1866 rewritten to the full standard as the worked example.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
3.8 KiB
| id | status | status_updated | status_note | component | severity | timestamp | last_seen | occurrences | files | tags | related | fix_commits | legacy | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| bug-1572 | fixed | 2026-09-24 | c9acd502 (bug-1585) |
scheduler | crash | 2026-08-18T01:30:00Z | 2026-09-19T09:00:00Z | 1 |
|
|
|
true |
bug-1572: xce_scheduler silently exits (no ASAN report captured -- backgrounded via setsid nohup ...…
Description
xce_scheduler silently exits (no ASAN report captured -- backgrounded via setsid nohup ... >/dev/null 2>&1) a few seconds after a worker's SECOND periodic resource_sync report for the same resource(s); first report processes fine, second triggers the crash with zero "resval applied" TRACE lines logged for that batch and "handle_scheduler_leave" observed on the controller ~1s later.
Environment
Not recorded (migrated from buglog.json).
Reproduction
Not recorded (migrated from buglog.json).
Root cause
Discovered live during resource-value-verification task 4.1 round-2 remediation smoke test (unrelated to the fix being validated -- reproduces identically with round-1's code, since round-1 only added a TRACE line AFTER the same zarray_put call). xce_core_push_events()'s XCE_CORE_EVENT_RESVALS_DELTA case (core_events.c ~line 786) does zarray_put(&(w->resvals), res->idx, rv->val) -- storing the decoded delta's xce_val_t* pointer DIRECTLY into scheduler-owned state, WITHOUT a MOVE_PTR/copy (unlike core_apply_config_resvals()'s sibling handling of the same ambiguity, which explicitly does MOVE_PTR(val, nv->val); zarray_put(&(core->sharedInst), r->idx, val);). Meanwhile driver.c's release_translated_payload() (called at driver.c:586 with applied=(push_rc>=0) after every successfully-pushed batch) has a case XCE_SCHED_OP_UPDATE_RESVALS: free_ResValList_ptr(&(req->u.resvalupdate.resvals)) with NO if (!applied) guard -- unlike the sibling XCE_SCHED_OP_UPDATE_CONFIG case, which explicitly skips the free when applied ("core->config now owns it permanently -- do not free"). Net effect: every successfully-applied RESVALS_DELTA batch immediately frees the exact xce_val_t objects that were just aliased into w->resvals[idx], leaving a dangling pointer there. The dangling pointer is inert until the SAME (worker, resource) index is updated again -- at that point zarray_put's killer (kill_sched_resval -> free_Value) is invoked on the already-freed pointer, a double-free that aborts the ASAN-instrumented process. This is why it manifests specifically on a worker's second periodic report, not the first.
Fix
NOT FIXED -- out of scope for resource-value-verification task 4.1 round-2 remediation, whose boundary was explicitly restricted to src/scheduler/src/core_events.c's Value_u_which else-branch bug only. Needs its own task: either (a) MOVE_PTR the value out of rv before zarray_put in core_events.c's RESVALS_DELTA case (mirroring core_apply_config_resvals()'s existing pattern) so release_translated_payload()'s free no longer touches live scheduler state, or (b) add an if (!applied) guard to driver.c's XCE_SCHED_OP_UPDATE_RESVALS case (mirroring XCE_SCHED_OP_UPDATE_CONFIG) so a successfully-applied batch is never freed at all -- (a) is likely preferable since RESVALS_FULL's sibling case has the identical unguarded-free shape and should be audited too. Worked around during round-2 validation by running the smoke test's resupdate quickly after deploy, before any real worker's second periodic report could trigger the crash.
Verification
Not recorded (migrated from buglog.json).
References
Not recorded (migrated from buglog.json).