lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs(raft-ha-core-safety): 5.8 validated (bug-1298 UAF fixed via ep refcount); round-4..6 reveals pre-existing commit-loss (task 5.9)

Browse source 
5.8 endpoint refcount fix validated: 182 unit tests/0 fail, 6-round ASAN chaotic
soak ASan-clean, all rounds elect. bug-1298 marked fixed.

The soak now reaches rounds 4-6 for the first time (every prior run crashed at
round 4) and FAILs the verdict on committed_job_loss=4/divergence=7. Evidence shows
this is the pre-existing bug-795/828/851 failover catch-up gap, NOT a regression:
one job missing from round 4 onward across 3 primaries; rounds 1-3 clean; ZERO
WAL/commit/catch-up code changed this session; ASAN clean. Routed to task 5.9 /
ha-commit-durability.
This commit is contained in:
Rongsong Shen 2026-06-30 14:27:59 +00:00
parent 6e05dc1523
commit da06ba437f
3 changed files with 224 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

103
.wolf/buglog.json
View file

@ -21522,7 +21522,7 @@
"error_message": "AddressSanitizer: heap-use-after-free READ size 4 at endpoint_is_task_blocked netio.c:1008 (vtask dispatcher T3) — endpoint freed by endpoint_killer cluster.c:813 (io_worker T2); ctrl-3 ABORTs mid-chaotic-soak round 3→4, dropping the cluster to 1/3 (no quorum) so round 4 'NO new primary within window'",
"file": "src/lib/netio.c / src/lib/vtaskmgr.c / src/controller/cluster.c",
"root_cause": "Teardown-vs-dispatch race: io_worker (T2) tears down a connection on error — handle_endpoint_error → zarray_del → endpoint_killer free()s the endpoint_t — while the vtask dispatcher (T3) is concurrently scanning the dataq (dataq_deq_first_match → vtask_eligible → endpoint_is_task_blocked) and reads ep->busy on that just-freed endpoint. The endpoint is freed while still referenced by a pending vtask in the dataq; nothing drains/guards the ep's queued vtasks before the free, and no refcount keeps it alive across the dispatcher read. Same netio surface as bug-1282 (ABBA, ep->busy made lockless) and the bug-1197 dispatch_inflight precedent; the bug-1282 lockless read removed the (incidental) lock contention so the dispatcher reaches the read faster, exposing the latent UAF.",
"fix": "NOT FIXED — proposed: drain/cancel an endpoint's queued vtasks from the dataq BEFORE endpoint_killer frees it, or hold a refcount/epoch on endpoint_t across the dispatcher eligibility read so the free is deferred until no dispatcher references remain (mirror bug-1197 dispatch_inflight). Must preserve the bug-1282 lockless busy read and not reintroduce the ABBA ep->lock/q->lock inversion.",
"fix": "FIXED (commit 6e05dc1): added endpoint_t reference count (refcnt _Atomic + heap_alloc flag). endpoint_get starts refcnt=1 (owner); vtask_enq takes a per-vtask ref before publishing the task to the dispatcher, run_task/taskq_item_killer release it; every heap-ep teardown site drops its owner ref via endpoint_unref (last unref fini+frees). The dispatcher can now read ep->busy on a queued vtask without the io_worker freeing it underneath. Embedded eps (heap_alloc=0) never freed by unref. Validated: 182 unit tests/0 fail; 6-round ASAN chaotic soak ZERO ASan reports, ctrl-3 survives round 4, all rounds elect.",
"tags": [
"netio",
"vtaskmgr",
@ -21533,7 +21533,8 @@
"raft-ha-core-safety",
"soak",
"quorum",
"bug-1282-family"
"bug-1282-family",
"FIXED"
],
"related_bugs": [
"bug-1282",
@ -21541,7 +21542,7 @@
"bug-1286"
],
"occurrences": 1,
"last_seen": "2026-06-30T13:36:00-07:00"
"last_seen": "2026-06-30T14:25:00-07:00"
},
{
"id": "bug-1299",
@ -21558,6 +21559,102 @@
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-30T13:39:09.194Z"
},
{
"id": "bug-1300",
"timestamp": "2026-06-30T14:04:05.998Z",
"error_message": "Significant refactor of ",
"file": "src/lib/netio.c",
"root_cause": "4 lines replaced/restructured",
"fix": "Rewrote 11→11 lines (4 removed)",
"tags": [
"auto-detected",
"refactor",
"c"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-30T14:04:05.998Z"
},
{
"id": "bug-1301",
"timestamp": "2026-06-30T14:05:02.703Z",
"error_message": "Significant refactor of ",
"file": "src/controller/cluster.c",
"root_cause": "2 lines replaced/restructured",
"fix": "Rewrote 9→12 lines (2 removed) | Also: /* zarray_del will finalize ep */; endpoint_fini(ep); | Also: endpoint_fini(ep2);; FREEUP(ep2);",
"tags": [
"auto-detected",
"refactor",
"c"
],
"related_bugs": [],
"occurrences": 4,
"last_seen": "2026-06-30T14:05:54.573Z"
},
{
"id": "bug-1302",
"timestamp": "2026-06-30T14:07:16.602Z",
"error_message": "Significant refactor of ",
"file": "src/controller/ha.c",
"root_cause": "2 lines replaced/restructured",
"fix": "Rewrote 8→7 lines (2 removed) | Also: endpoint_fini(ep);; free(ep); | Also: endpoint_fini(ep);; free(ep); /* endpoint_get() calloc'd ep; ",
"tags": [
"auto-detected",
"refactor",
"c"
],
"related_bugs": [],
"occurrences": 3,
"last_seen": "2026-06-30T14:09:43.010Z"
},
{
"id": "bug-1303",
"timestamp": "2026-06-30T14:14:35.148Z",
"error_message": "Significant refactor of ",
"file": "src/controller/testsuite/test_ha_vep_teardown.c",
"root_cause": "4 lines replaced/restructured",
"fix": "Rewrote 8→9 lines (4 removed) | Also: /* ep was not torn down by a vtask here — free it ; * by the drop path). */",
"tags": [
"auto-detected",
"refactor",
"c"
],
"related_bugs": [],
"occurrences": 2,
"last_seen": "2026-06-30T14:16:12.183Z"
},
{
"id": "bug-1304",
"timestamp": "2026-06-30T14:24:36.874Z",
"error_message": "Function not marked async",
"file": ".kiro/specs/raft-ha-core-safety/research.md",
"root_cause": "Function uses await but wasn't declared async",
"fix": "Added async modifier",
"tags": [
"auto-detected",
"async-fix",
"md"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-30T14:24:36.874Z"
},
{
"id": "bug-1305",
"timestamp": "2026-06-30T14:25:55.429Z",
"error_message": "Missing error handling in unknown",
"file": ".kiro/specs/raft-ha-core-safety/tasks.md",
"root_cause": "Code path had no error handling — exceptions would propagate uncaught",
"fix": "Added try/catch block",
"tags": [
"auto-detected",
"error-handling",
"md"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-30T14:25:55.429Z"
}
]
}