Rongsong Shen
d26ff8870d
The buglog was one 27K-line JSON file whose fields covered only symptom,
root cause and fix, as free text: 2% of entries named the code version a
bug was found on, 12% the build profile, 36% a way to reproduce, and 58% of
the file was OpenWolf hook auto-detections ("Missing guard clause",
"Function not marked async" on .md files).
- .claude/skills/buglog/bug-record.md: the record standard. Eight
sections (Summary as the title, Description, Environment, Reproduction;
Root cause, Fix, Verification, References once fixed), each with the
content it must contain -- e.g. Environment needs the OS, the libc the
failing binary loads (from its ELF interpreter: glibc 2.42 in the nix
shell vs 2.40 on the host), toolchain, build profile and commit.
- .wolf/bugs/<id>.md: 500 records migrated from buglog.json; every field
verified to round-trip. They carry legacy: true and an explicit "not
recorded" where the old log had no such field; component derived from
file paths, severity from keywords (else unknown), fix_commits from
hashes in status_note/fix that git resolves (91 of 463 fixed).
- .wolf/buglog-archive.json: the 710 hook auto-detections, verbatim.
- .wolf/buglog.json is now an index generated from the records, keeping
the fields OpenWolf's hooks and `openwolf bug search` read.
- scripts/buglog.py rewritten: new (template + Environment collected from
the machine and --binary), check, status (refuses a status the record
doesn't support; fixed needs Root cause, Fix, Verification and
--commit hashes git resolves), note, recur, set, show, list, triage,
env, index. 29 unittest cases in scripts/tests/test_buglog.py.
- scripts/git-hooks/pre-commit runs `check --staged`: incomplete staged
records, a legacy record hand-changed to fixed without the fix-time
sections, a new record claiming legacy: true, or a stale index refuse
the commit.
- scripts/gitea-sync.py reads the records: issue body = the record's
sections plus linked fix commits (origin remote, GITEA_COMMIT_REPO to
override), and a body hash so an issue's text is refreshed when the
record changes.
- bug-1866 rewritten to the full standard as the worked example.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
6.3 KiB
| id | status | status_updated | status_note | component | severity | timestamp | last_seen | occurrences | files | tags | related | legacy | gitea_issue | gitea_synced_status | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| bug-1452 | fixed | 2026-09-24 | controller | crash | 2026-07-18T00:00:00Z | 2026-09-19T09:00:00Z | 1 |
|
|
|
true | 36 | fixed |
bug-1452: AddressSanitizer: SEGV on unknown address 0x000000000038 in commit_advance_locked…
Description
AddressSanitizer: SEGV on unknown address 0x000000000038 in commit_advance_locked (controller/commit.c:208), called from commit_on_durable_advance (commit.c:341) <- fsync_worker (persist.c:399), a background ASAN thread started by persist_start (persist.c:723) inside test_persist_truncate.c's make_fresh_persist / test_truncate_above_resolves_pending_continuations_as_failed. Reproduced by temporarily flipping BOTH g_drive_async_demotion_offload and g_drive_async_catchup_offload (bug-1450/1451) from their default 0 to 1 and running the standalone controller_unit_test binary directly (976-test suite) -- crashed immediately, before completing the suite. Confirmed absent with both flags at their committed default of 0: the same binary/build (just reverted) completed cleanly, 976 Tests 0 Failures 0 Ignored, twice.
Environment
Not recorded (migrated from buglog.json).
Reproduction
Not recorded (migrated from buglog.json).
Root cause
ROOT-CAUSED 2026-07-19: the g_drive_async_demotion_offload/g_drive_async_catchup_offload flags were a RED HERRING -- this exact SEGV signature (NULL deref in the fsync_worker -> commit_on_durable_advance -> commit_advance_locked chain) was already independently observed on 2026-07-07, long before those flags existed (see memory: 'Pre-existing ASAN SEGV in test_persist_truncate Unrelated to Bug-1417 Work'). The flags only perturbed thread-scheduling timing enough to make a PRE-EXISTING, always-latent test-teardown race fire more reliably; they do not participate in the crash's causal chain at all. Actual root cause: in src/controller/testsuite/test_persist_truncate.c, the three continuation-related tests (test_truncate_above_resolves_pending_continuations_as_failed, test_truncate_noop_does_not_resolve_pending_continuations, test_truncate_concurrent_burst_resolves_above_cutoff_exactly_once) called commit_engine_fini(&ce) BEFORE teardown_persist(p) (which calls persist_stop(p)) -- the exact opposite of the contract documented in commit_engine_fini's own comment (commit.c:464-468: 'Callers are expected to have stopped the flusher (persist_stop) first'). Because persist_stop had not yet run, the fsync_worker background thread was still alive and could be mid-flight invoking the durable-advance callback (fsync_worker, persist.c:398-399, calls p->on_durable_advance(p->durable_cb_ctx) with NO lock held) concurrently with commit_engine_fini destroying ce->lock/ce->cond and, as its final step, setting ce->p = NULL. This produces a genuine TOCTOU data race with no synchronization between the two threads: commit_on_durable_advance's own guard (if (ce == NULL || ce->p == NULL) return;) can read ce->p as still non-NULL (racing ahead of or concurrent with commit_engine_fini's teardown), pass the guard, and then commit_advance_locked's SEPARATE, later re-read of ce->p (commit.c:208, atomic_load(&ce->p->committed_lsn)) observes the NOW-NULLED value written by the concurrently-running commit_engine_fini on the main thread -- producing a NULL-pointer dereference at exactly offset 0x38 (offsetof(persist_t, committed_lsn)), matching the crash address bit-for-bit. persist_set_durable_cb's unhook (called from commit_engine_fini) is 'defensive and cheap' per its own comment but is NOT sufficient on its own without persist_stop's pthread_join guaranteeing the flusher thread cannot be in-flight at all -- confirmed by the fact that persist_set_durable_cb existed and ran in this path the whole time and the crash still occurred.
Fix
FIXED 2026-07-19 (commit pending): added an explicit persist_stop(p); call immediately before commit_engine_fini(&ce); in all three affected test functions in test_persist_truncate.c (previously commit_engine_fini ran first, then teardown_persist -> persist_stop ran second -- backwards from the documented contract). persist_stop joins the fsync_worker thread, so by the time commit_engine_fini tears down ce->lock/ce->cond/ce->p, no thread can be mid-callback referencing them. teardown_persist's own internal persist_stop(p) call becomes a safe idempotent no-op on the second invocation (fsync_thread_running is already 0 and wal_fd is already -1, so both of its internal branches skip). No production code changed -- this was purely a test-fixture ordering bug, not a bug in commit.c/persist.c itself. Verified: full ASAN controller_unit_test suite (976 tests) run 6 times total (1 via meson + 5 standalone direct binary runs per project convention of not trusting meson's aggregate alone) -- 976 Tests 0 Failures 0 Ignored on every run, zero ASAN reports (SEGV/UAF/etc.), all 8 test_persist_truncate.c tests PASS every run. NOTE for a future session: the identical commit_engine_fini-before-persist_stop ordering also exists in test_demotion_convergence.c (line 233 commit_engine_fini(&g_dc.commit) before line 235 persist_stop(&g_dc.persist), against a real persist_start'd fixture) -- same latent race, not yet observed to crash there (likely a narrower window against a static/global fixture rather than a per-test stack fixture), out of scope for this fix but worth a follow-up sweep of all commit_engine_fini call sites.
Verification
Not recorded (migrated from buglog.json).
References
- Related: bug-1429, bug-1450, bug-1451
- Gitea: #36