lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(controller): standby-endpoint err handler freed ep->data under a live vtask (bug-1839)

Browse source 
Live 3-node HA soak crashed xce_controller: SIGSEGV in
peer_ctrl_msg_handler_inline (ha.c:7035) dereferencing a NULL
ep->data. gdb analysis of the actual core dump showed the endpoint
itself fully alive (refcnt=191964, tearing_down=0) while ep->data had
been synchronously freed and nulled by standby_ep_err_handler_primary,
the endpoint's err_handler, with zero coordination with the vtask pool
that was concurrently dispatching a peer-control message for the same
endpoint (g_drive_async_peerctrl_dispatch_enabled defaults ON).

The sibling handlers (vote_ep_err_handler, pre_vote_ep_err_handler)
already avoid this exact hazard, with an explicit comment: "Do NOT
free vctx here -- peer_ctrl_msg_handler may still reference ep->data."
standby_ep_err_handler_primary was never updated to the same
discipline when the vtask-pool dispatch machinery (bug-1197/bug-1298)
was introduced.

Fix: route standby_ep_err_handler_primary through the same
already-tested vote_ep_teardown_async() deferred, per-endpoint-FIFO-
serialized teardown that vote-endpoint demotion already uses, plus an
idempotency guard (endpoint_is_tearing_down) since the old "ep->data
== NULL" reentry check no longer applies once the free is deferred.
Also fixes a second-order double-unref this surfaced in
on_standby_connected's stale-duplicate eviction path, caught by ASAN.

TDD: two new tests prove the teardown is deferred until an in-flight
same-ep vtask completes, and that a repeated error callback on the
same endpoint is a safe no-op. Full controller suite: 1219/1219, zero
regressions.

Also logs bug-1840 (a separate scheduler heap-corruption crash from
the same soak run) as not-fixed pending an ASAN reproduction -- a
non-instrumented core dump cannot localize glibc's malloc_consolidate
abort back to the actual out-of-bounds write.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-09-22 03:21:14 +00:00
parent 4f94638a1d
commit e1da9f1e9b
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
7 changed files with 299 additions and 38 deletions
Download patch file Download diff file Expand all files Collapse all files

8
.wolf/memory.md
View file

@ -5626,3 +5626,11 @@
| 04:26 | Edited src/scheduler/src/sim_loop.c | added 1 condition(s) | ~128 |
| 04:33 | Edited src/scheduler/src/sim_loop.c | modified sim_reconcile_invoke_on_dispatch() | ~90 |
| 04:41 | Edited .kiro/specs/scheduler-sim-eager-reconciliation/design.md | modified Correction() | ~557 |
| 19:40 | Edited src/controller/testsuite/test_ha_vep_teardown.c | 4→9 lines | ~159 |
| 19:41 | Edited src/controller/testsuite/test_ha_vep_teardown.c | added 1 condition(s) | ~2024 |
| 19:41 | Edited src/controller/testsuite/test_ha_vep_teardown.c | poll_until_ge() → poll_until() | ~41 |
| 19:53 | Edited src/controller/ha.c | added 1 condition(s) | ~558 |
| 19:53 | Edited src/controller/testsuite/test_ha_liveness.c | 8→11 lines | ~178 |
| 19:53 | Edited src/controller/testsuite/test_ha_liveness.c | 4→4 lines | ~60 |
| 19:54 | Edited src/controller/testsuite/test_ha_liveness.c | 6→4 lines | ~69 |
| 20:01 | Edited src/controller/ha.c | modified XCE_TESTING() | ~235 |