test(raft-drive-async): receiver-side dispatch liveness integration tests (task 3.4b)

With the dispatch flags flipped ON in-test, prove the three receiver-side
liveness properties (production defaults stay OFF):

- P1: while a snapshot (and config-push) transfer is parked in flight on the
  slow lane, another endpoint's peer-control message is still processed
  promptly on the vtask pool (different thread), with the transfer still
  parked — the offloaded transfer does not block the drive path.
- P2: a peer-control ack (classified VTASK_LIVENESS) runs promptly even while
  the pool is saturated to the LivenessAdmission reservation boundary with
  REQ_BLOCK tasks (one more REQ_BLOCK fast-rejects with -E_XCE_NOT_READY,
  proving real saturation). Narrowed to the liveness-dispatch-under-saturation
  assertion; commit-advance itself is covered by the commit-engine tests.
- P3: while one endpoint's peer-control body is parked (holding a worker + that
  ep busy), another endpoint's body runs concurrently on a different worker —
  per-endpoint FIFO does not serialize across endpoints.

All deterministic (cond-var gates, bounded polls). Fixed a flaky teardown-race
hang (bug-1224): the tests released a parked vtask body then immediately called
the cancel-based vtaskmgr_fini while the just-released body was still unwinding
(P2's REQ_BLOCK bodies racing blk_lock teardown the worst). Fix adds end-of-body
completion counters polled to the full expected count before fini, so every
pool body has fully returned (worker idle) before the cancel. slowlane_fini is
graceful (join) so the lane side was already safe.

Verified: 5/5 deterministic isolated runs; full controller suite EXIT=0;
ASan-clean. Completes task 3.4 (3.4a UAF fix + 3.4b tests); the production
flag-flip remains deferred to after the 4.1 partition soak + 4.2 regression
sweep.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S6fDCtXkkEgupu7rUah1y2
This commit is contained in:
Rongsong Shen 2026-06-27 00:36:22 +00:00
parent 6d510d8cf4
commit d2830ce514
6 changed files with 693 additions and 2 deletions

View file

@ -20224,6 +20224,61 @@
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-26T17:12:06.544Z"
},
{
"id": "bug-1222",
"timestamp": "2026-06-26T17:44:25.670Z",
"error_message": "Incorrect value in code",
"file": "src/controller/meson.build",
"root_cause": "Had 'testsuite/test_waldump.c'",
"fix": "Changed to 'testsuite/test_ha_dispatch_integration.c'",
"tags": [
"auto-detected",
"wrong-value",
"build"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-26T17:44:25.670Z"
},
{
"id": "bug-1223",
"timestamp": "2026-06-27T00:33:17.036Z",
"error_message": "Significant refactor of ",
"file": "src/controller/testsuite/test_ha_dispatch_integration.c",
"root_cause": "2 lines replaced/restructured",
"fix": "Rewrote 5\u21923 lines (2 removed) | Also: /* Release ep_A so its worker finishes before the ; TEST_ASSERT_EQUAL_INT_MESSAGE(2, poll_atomic_ge(&g",
"tags": [
"auto-detected",
"refactor",
"c"
],
"related_bugs": [],
"occurrences": 2,
"last_seen": "2026-06-27T00:34:00.682Z"
},
{
"id": "bug-1224",
"timestamp": "2026-06-26",
"error_message": "raft-drive-async 3.4b integration tests (test_ha_dispatch_integration.c) hang intermittently \u2014 controller_unit_test never completes (timeout-killed)",
"file": "src/controller/testsuite/test_ha_dispatch_integration.c",
"root_cause": "Test teardown race: P1/P2/P3 released a parked vtask body (gate_open) then immediately called vtaskmgr_fini, which tears down workers via pthread_cancel. The tests polled a run-counter incremented MID-body, so fini raced the just-released body still unwinding (esp. P2 REQ_BLOCK bodies in vtaskmgr_blocking_release racing blk_lock teardown) \u2192 flaky deadlock. slowlane_fini is graceful (join) so the lane side was safe; vtaskmgr_fini (cancel) was the hazard.",
"fix": "Add end-of-body completion counters (g_pc_completed on both pc_probe paths incl. the park path; g_block_done at the end of block_task_entry) and poll them to the full expected count AFTER gate_open and BEFORE vtaskmgr_fini, so every pool body has fully returned (worker idle in cbuf_get) before the cancel-based fini. Verified deterministic 5/5 isolated runs + full suite EXIT=0, ASan-clean.",
"tags": [
"raft-drive-async",
"task-3.4b",
"test",
"flaky",
"vtaskmgr_fini",
"pthread_cancel",
"teardown-race",
"deadlock"
],
"related_bugs": [
"bug-924"
],
"occurrences": 1,
"last_seen": "2026-06-26"
}
]
}