feat(xce-build): add non-ASAN debug + release profiles; root-cause bug-1286 with the debug build

/xce-build now takes a build PROFILE: asan (default, debug+ASAN -> xce.build,
backward-compatible), debug (non-ASAN -O0 -g3 -fno-inline -fno-omit-frame-pointer
-> build.dbg, for gdb/cores), release (-O3 -> build.release). Each profile uses
its own default build dir so variants don't clobber each other; arg2 may still be
a legacy build_dir. asan runs the unit suites; debug/release are build-only (the
suites are ASAN-oriented and the meson test harness is overlay-fs fragile, bug-924)
and print how to run a suite directly.

Why: bug-1282/bug-1286 are heisenbugs ASAN perturbs away (no SIGSEGV) and that
gdb-on-release turns into a hang/timeout. The non-ASAN -O0 -g3 build reproduces
them AND yields a symbolized backtrace under gdb.

It immediately paid off: the debug profile ROOT-CAUSED bug-1286 (the round-4
segfault) — myreq_handler (cluster.c:251) does `c = ep->owner` then
`ha_dispatch_gate(&c->ha,…)`; on chaotic restart a walCatchupEntry (op 37) vtask
runs with ep->owner==NULL, so &c->ha = 0x3ff0 -> deref crash at ha.c:3194. Fix is
a NULL-guard in myreq_handler (+ owner-lifecycle). 7.2 now has one remaining
blocker (bug-1286, root-caused); bug-1282 deadlock is fixed.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019dbTHR9CsEYK9RsqmQpQpC
This commit is contained in:
Rongsong Shen 2026-06-30 09:15:36 +00:00
parent e90bf835f9
commit 02e9275706
5 changed files with 208 additions and 25 deletions

View file

@ -21304,6 +21304,96 @@
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-30T08:24:48.860Z"
},
{
"id": "bug-1286",
"timestamp": "2026-06-30T05:30:00Z",
"error_message": "xce_controller SIGSEGV in ha_dispatch_gate during chaotic round-4 restart — Thread received SIGSEGV at ha.c:3194 (atomic_load &ha->role) with ha=0x3ff0 (garbage). The release-only crash that blocks raft-ha-core-safety 7.2; DISTINCT from the bug-1282 deadlock (which is fixed).",
"file": "src/controller/cluster.c:251 (myreq_handler) + src/controller/ha.c:3194 (ha_dispatch_gate)",
"root_cause": "NULL ep->owner. myreq_handler does `cluster_t* c = (cluster_t*)ep->owner;` then `ha_dispatch_gate(&c->ha, opcode)`. On a chaotic restart an inbound endpoint dispatches a walCatchupEntry (op 37) vtask while ep->owner==NULL, so c=NULL and &c->ha = NULL+offsetof(cluster_t,ha) = 0x3ff0 — a garbage pointer ha_dispatch_gate dereferences (atomic_load &ha->role, ha.c:3194) -> SIGSEGV. Backtrace (debug -O0 -g3 under gdb): ha_dispatch_gate(ha=0x3ff0,opcode=37) <- myreq_handler cluster.c:251 <- vtask_entry cluster.c:693 <- run_task/vcpu_entry. ep->owner is NULL because the accept/dispatch path runs a request before owner is wired (or owner cleared mid-flight during teardown).",
"fix": "NOT YET FIXED — root cause nailed via the non-ASAN debug build + gdb. Minimal: NULL-guard `c = ep->owner` in myreq_handler (drop/reject the request if owner not attached) instead of dereferencing. Deeper: set ep->owner at accept time before the endpoint is monitored for input, and never clear it while a vtask is in flight (endpoint-owner lifecycle). Blocks raft-ha-core-safety 7.2 (the second of the two round-4 blockers; bug-1282 deadlock already fixed).",
"tags": [
"segfault",
"null-deref",
"ep-owner",
"myreq_handler",
"cluster.c",
"ha_dispatch_gate",
"restart",
"7.2",
"ROOT-CAUSED"
],
"related_bugs": [
"bug-1282",
"bug-1281",
"bug-1197"
],
"occurrences": 3,
"last_seen": "2026-06-30T05:30:00Z"
},
{
"id": "bug-1287",
"timestamp": "2026-06-30T09:00:43.427Z",
"error_message": "Significant refactor of ",
"file": "src/tools/xce-build.sh",
"root_cause": "4 lines replaced/restructured",
"fix": "Rewrote 19→53 lines (4 removed)",
"tags": [
"auto-detected",
"refactor",
"sh"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-30T09:00:43.427Z"
},
{
"id": "bug-1288",
"timestamp": "2026-06-30T09:02:01.055Z",
"error_message": "Significant refactor of ",
"file": ".claude/skills/xce-build/SKILL.md",
"root_cause": "3 lines replaced/restructured",
"fix": "Rewrote 17→26 lines (3 removed)",
"tags": [
"auto-detected",
"refactor",
"md"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-30T09:02:01.055Z"
},
{
"id": "bug-1289",
"timestamp": "2026-06-30T09:10:50.002Z",
"error_message": "Significant refactor of ",
"file": "src/tools/xce-build.sh",
"root_cause": "3 lines replaced/restructured",
"fix": "Rewrote 5→10 lines (3 removed)",
"tags": [
"auto-detected",
"refactor",
"sh"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-30T09:10:50.002Z"
},
{
"id": "bug-1290",
"timestamp": "2026-06-30T09:15:12.861Z",
"error_message": "Incorrect value in code",
"file": ".kiro/specs/raft-ha-core-safety/tasks.md",
"root_cause": "Had 't crash) — STILL blocks the gate; likely an endpo",
"fix": "Changed to 't crash) — is now **ROOT-CAUSED** (via the new `",
"tags": [
"auto-detected",
"wrong-value",
"md"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-30T09:15:12.861Z"
}
]
}