lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(controller): standby-endpoint err handler freed ep->data under a live vtask (bug-1839)

Browse source 
Live 3-node HA soak crashed xce_controller: SIGSEGV in
peer_ctrl_msg_handler_inline (ha.c:7035) dereferencing a NULL
ep->data. gdb analysis of the actual core dump showed the endpoint
itself fully alive (refcnt=191964, tearing_down=0) while ep->data had
been synchronously freed and nulled by standby_ep_err_handler_primary,
the endpoint's err_handler, with zero coordination with the vtask pool
that was concurrently dispatching a peer-control message for the same
endpoint (g_drive_async_peerctrl_dispatch_enabled defaults ON).

The sibling handlers (vote_ep_err_handler, pre_vote_ep_err_handler)
already avoid this exact hazard, with an explicit comment: "Do NOT
free vctx here -- peer_ctrl_msg_handler may still reference ep->data."
standby_ep_err_handler_primary was never updated to the same
discipline when the vtask-pool dispatch machinery (bug-1197/bug-1298)
was introduced.

Fix: route standby_ep_err_handler_primary through the same
already-tested vote_ep_teardown_async() deferred, per-endpoint-FIFO-
serialized teardown that vote-endpoint demotion already uses, plus an
idempotency guard (endpoint_is_tearing_down) since the old "ep->data
== NULL" reentry check no longer applies once the free is deferred.
Also fixes a second-order double-unref this surfaced in
on_standby_connected's stale-duplicate eviction path, caught by ASAN.

TDD: two new tests prove the teardown is deferred until an in-flight
same-ep vtask completes, and that a repeated error callback on the
same endpoint is a safe no-op. Full controller suite: 1219/1219, zero
regressions.

Also logs bug-1840 (a separate scheduler heap-corruption crash from
the same soak run) as not-fixed pending an ASAN reproduction -- a
non-instrumented core dump cannot localize glibc's malloc_consolidate
abort back to the actual out-of-bounds write.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-09-22 03:21:14 +00:00
parent 4f94638a1d
commit e1da9f1e9b
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
7 changed files with 299 additions and 38 deletions
Download patch file Download diff file Expand all files Collapse all files

9
.wolf/anatomy.md
View file

@ -1,7 +1,7 @@
# anatomy.md
> Auto-maintained by OpenWolf. Last scanned: 2026-09-21T11:41:46.809Z
> Files: 670 tracked | Anatomy hits: 0 | Misses: 0
> Auto-maintained by OpenWolf. Last scanned: 2026-09-22T03:01:53.188Z
> Files: 671 tracked | Anatomy hits: 0 | Misses: 0
## ./
@ -884,7 +884,7 @@
## src/controller/
- `ctrlstatus.c` — include "config.h" (~4588 tok)
- `ha.c` — define _GNU_SOURCE (~127669 tok)
- `ha.c` — define _GNU_SOURCE (~127991 tok)
- `ha.h` — ifndef HA_H (~34444 tok)
- `meson.build` (~3405 tok)
@ -895,9 +895,10 @@
- `test_ctrlstatus.c` — include <stdatomic.h> (~12594 tok)
- `test_e2e_capnp.c` — include <stdint.h> (~20187 tok)
- `test_ha_heartbeat_committed.c` — include <stdatomic.h> (~5775 tok)
- `test_ha_liveness.c` — Declares directly (~25885 tok)
- `test_ha_liveness.c` — Declares directly (~26012 tok)
- `test_ha_partition.c` (~5500 tok)
- `test_ha_role.c` — include <signal.h> (~13998 tok)
- `test_ha_vep_teardown.c` (~8206 tok)
- `test_rcfile_integration.sh` — test_rcfile_integration.sh — xce-test-rc-file integration test, shell-driven (~4117 tok)
## src/harness/