wdgzyjx/myxce
1
0
Fork 0
forked from lei.chen/myxce
Code Pull requests Activity

fix(libxce-failover): correct endpoint reconnect on rotate — re-bind iomgr + re-arm handlers (fixes endpoint_connect SEGV during live failover)

Browse source 
xce_failover_rotate reconnected with endpoint_fini+endpoint_init+endpoint_connect
but never re-bound ep->iomgr (reset by endpoint_init) nor re-armed the owner/
msg/err handlers + iomgr_monitor. endpoint_connect then dereferenced a NULL
iomgr (->ssl_ctx) → SEGV at netio.c:672 during live master failover. Added
xce_ctx_reconnect() in xcectx.c that mirrors xcecli_start's establishment
(without restarting iomgr/vtaskmgr or touching ctx->state); rotate delegates to
it. Found via live 3-node E2E. bug-610.
This commit is contained in:
Rongsong Shen 2026-06-17 06:53:46 +00:00
parent 312bd6be34
commit ffc2c590f9
4 changed files with 95 additions and 18 deletions
Download patch file Download diff file Expand all files Collapse all files

24
.wolf/buglog.json
View file

@ -10446,6 +10446,30 @@
],
"occurrences": 1,
"last_seen": "2026-06-17T04:45:00Z"
},
{
"id": "bug-610",
"timestamp": "2026-06-17T15:00:00Z",
"error_message": "ASan SEGV in endpoint_connect (netio.c:672) via xce_failover_rotate \u2192 xcecli_retry_next \u2192 xcecli_joblist during live master failover (NULL ep->iomgr deref)",
"file": "src/lib/client_failover.c",
"root_cause": "xce_failover_rotate reconnected via endpoint_fini + endpoint_init + endpoint_connect, but endpoint_init (do_endpoint_init) resets the endpoint struct including ep->iomgr, and the rotate never re-bound it. The client's endpoint gets ep->iomgr = &ctx->iomgr only via xce_ctx_set_endpoint (called once in xcecli_start). endpoint_connect derefs iomgr->ssl_ctx when ep->ssl==NULL (true after endpoint_fini frees the SSL) \u2192 NULL deref crash. Also the rotate never re-set owner/msg_handler/err_handler nor iomgr_monitor'd the new fd, so even without the crash the reconnected endpoint would not route replies. Pure unit tests could not exercise the live socket/iomgr path; only the live failover E2E hit it.",
"fix": "Added xce_ctx_reconnect(ctx,host,port) in xcectx.c (where static handle_msg/handle_disconnect live) that mirrors the connection-establishment of xcecli_start/xce_ctx_start without restarting iomgr/vtaskmgr or touching ctx->state: endpoint_fini \u2192 endpoint_init \u2192 endpoint_set_peer \u2192 re-bind ep->iomgr=&ctx->iomgr \u2192 endpoint_connect \u2192 re-set owner+msg/err handlers \u2192 iomgr_monitor. xce_failover_rotate now delegates the reconnect to it.",
"tags": [
"libxce",
"failover",
"crash",
"SEGV",
"endpoint_connect",
"iomgr",
"reconnect",
"E2E"
],
"related_bugs": [
"bug-609",
"bug-608"
],
"occurrences": 1,
"last_seen": "2026-06-17T15:00:00Z"
}
]
}