lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(netio): bug-1447 same-batch endpoint use-after-free in io_worker dispatch

Browse source 
Raft-drive-async's reconnect-offload burst (Task 2.1/2.2) exposed a
latent race in io_worker's epoll dispatch loop: ha.c's
on_standby_connected() evicts a stale duplicate peer channel via
iomgr_unmonitor()+endpoint_unref(), but epoll_ctl(DEL) only stops
FUTURE epoll_wait() calls -- it does not invalidate an event for that
same endpoint already sitting in the CURRENT batch's events[] array.
A later event in the same batch then dereferences the freed
endpoint_t/SSL*, producing the ASAN SEGV observed deep inside
OpenSSL's ssl3_read_bytes during raft-drive-async's round-9 soak.

The original "concurrent SSL_do_handshake from two io_worker threads"
hypothesis was wrong and is retracted -- iomgr_start() creates exactly
one io_worker thread per iomgr_t.

Fix: extract io_worker_dispatch_batch(), which takes endpoint_ref() on
every endpoint in the batch before dispatching any of them and
endpoint_unref()s each only after the whole batch completes -- the
same lock-free ref/unref bracket already used for bug-1298 and
INVOKE_MSG_HANDLER, just never applied to io_worker's own loop. Pure
atomics, no locks, no blocking added to the hot path.

Adds test_netio_batch_dispatch_uaf.c, which drives the real dispatch
loop with a synthetic epoll_event[] batch (TDD RED confirmed via ASAN
heap-use-after-free before the fix, GREEN after). Verified further
with two independent clean 10-round live sequential HA soaks (33 and
36 jobs, zero loss) and full ASAN suite runs (xcelib_test 294/0,
controller_unit_test 972/0). raft-drive-async's kiro-validate-impl
re-run flips NO-GO to GO.
This commit is contained in:
Rongsong Shen 2026-07-18 13:28:04 +00:00
parent 4d2c90389f
commit 426728bb98
6 changed files with 249 additions and 8 deletions
Download patch file Download diff file Expand all files Collapse all files

8
src/include/netio.h
View file

@ -207,4 +207,12 @@ extern int iomgr_unmonitor(iomgr_t*, endpoint_t*);
extern int iomgr_start(iomgr_t*);
extern int iomgr_fini(iomgr_t*);
extern int iomgr_stop(iomgr_t*);
/* bug-1447: expose io_worker's per-batch dispatch loop so a test can drive it
* directly with a synthetic epoll_event[] array (no real epoll_wait()/kernel
* needed) and verify that one event's handler cannot free an endpoint that a
* LATER event in the same batch still references. Not XCE_TESTING-gated:
* netio.c builds once into libxce.a with no per-target defines (see netio.c
* for why), matching every other function in this header. */
extern void io_worker_dispatch_batch_for_test(struct epoll_event*, int);
#endif