lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs(buglog): bug-1914, bug-1917, bug-1918 fixed; refresh STATUS.md

Browse source 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-04 23:59:29 +00:00
parent 06e99b9081
commit 8e66d463ea
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
5 changed files with 67 additions and 50 deletions
Download patch file Download diff file Expand all files Collapse all files

17
.wolf/bugs/bug-1917.md
View file

@ -1,8 +1,8 @@
---
id: "bug-1917"
status: "investigating"
status_updated: "2026-10-03"
status_note: "code-review finding from bug-1916; watch loaded soaks for reconnect stacks without free_packet"
status: "fixed"
status_updated: "2026-10-04"
status_note: ""
component: "lib"
severity: "crash"
timestamp: "2026-10-03T22:55:30Z"
@ -11,6 +11,7 @@ occurrences: 1
files: ["src/lib/xcectx.c", "src/lib/netio.c"]
tags: ["race", "failover", "client", "ssl"]
related: ["bug-1916"]
fix_commits: ["431a6d70"]
---
# bug-1917: xce_ctx_reconnect re-inits the endpoint on the caller thread while the io_worker may still use it
@ -39,15 +40,15 @@ UPDATE (2026-10-03): Review of the bug-1916 fix (2026-10-03) adds a second hazar
- Evidence: code reading of src/lib/xcectx.c xce_ctx_reconnect and src/lib/netio.c endpoint_fini / handle_input / handle_output
## Root cause
TODO: required for fixed/partial
`xce_ctx_reconnect` (src/lib/xcectx.c) ran `endpoint_fini`/`endpoint_init` on `ctx->ep` straight away, while work dispatched for the old connection could still be running: the io_worker inside its epoll batch (it holds an endpoint reference for the whole batch: `SSL_read`/`handle_output` on the fd and SSL being freed), and VTASK_IO reply tasks still queued on the context's pool (each holds a reference and reads `ep->owner` without a lock). A queued reply task of the old connection that ran after the reconnect found `ep->owner` re-armed to the context and was delivered to the new session.
## Fix
TODO: required for fixed/partial
`xce_ctx_reconnect` first calls `xce_ctx_quiesce_ep`: unmonitor the old fd (no further io_worker events), clear `ep->owner` under `ep->lock` (stale reply tasks are dropped; `vtask_entry` frees their packet), then wait until the endpoint's reference count is back at its base value (0 for the embedded client endpoint, 1 for a heap one), polling every 1 ms. The wait is bounded at 2000 ms: if the caller is itself a task of this endpoint its own reference never drains, and the reconnect then proceeds as before with a WARN ("reconnecting anyway") rather than deadlocking. `vtask_entry` now reads `ep->owner` under `ep->lock`.
## Verification
- Test: TODO: name + file; RED before the fix, GREEN after
- Suites: TODO: suite results and build profile
- Live: TODO: soak/deploy run, or n/a and why
- Test: `test_reconnect_drains_old_connection_tasks` (new src/lib/testsuite/test_ctx_reconnect_quiesce.c): reconnect to a local listener, queue an old-connection packet via INVOKE_MSG_HANDLER -> handle_msg with the pool stopped, reconnect again while a helper starts the pool 100 ms in. RED before the fix (the stale task reached the context's request handler: expected 0 calls, was 1), GREEN and LSan-clean after.
- Suites: `xce-build.sh all` asan, 127/127 meson suites OK, Go tests ok (2026-10-04).
- Live: loaded `/xce-test-ha` sequential 30 rounds (40 busy loops on 32 cores), 2026-10-04: PASS, zero job loss, all 3 nodes led, no ASAN report in any controller/worker/scheduler log; client failover reconnects in worker/scheduler/xce_test throughout, and the quiesce never needed its timeout (no "reconnecting anyway" WARN).
## References
- Related: none