lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
myxce/.kiro/specs/ha-elect-ctx-lifecycle/tasks.md
Rongsong Shen 6529dcf08f feat(ha): reference-counted ha_elect_ctx_t lifecycle (ha-elect-ctx-lifecycle) 
Replace the deliberate election-carrier leak with an atomic refcount:
a bias reference held by ha->elect_ctx plus one reference per captured
vote endpoint, released via a new endpoint_t.on_fini teardown-completion
hook (fired once, before ep->data is freed). Both real-election and
PreVote capture sites (raft_cb_send_vote_request,
raft_cb_send_pre_vote_request) increment on capture; both publish sites
(ha_start_election_async, ha_start_prevote_async) now atomic_exchange
the outgoing carrier and release it instead of leaking or freeing
outright, so a still-outstanding vote endpoint keeps the superseded
carrier alive until its own teardown completes.

Verified: 758+ controller tests green, isolated LeakSanitizer run shows
zero ha_elect_ctx_t leaks across many election/PreVote rounds, no
premature-free/UAF under supersession, and a live 10-round HA soak
(36 jobs, zero loss, all 3 nodes served as primary).
2026-07-12 04:25:42 +00:00

6.6 KiB
Raw Blame History

Implementation Plan: ha-elect-ctx-lifecycle

  • 1. Foundation: endpoint teardown-completion hook

  • 1.1 Add a generic completion hook to the endpoint abstraction, fired exactly once when an endpoint's teardown reaches its final free

    • Add an on_fini function pointer field to endpoint_t (src/include/netio.h), mirroring the existing on_connected fire-once hook
    • In endpoint_fini (src/lib/netio.c), call the hook (if set) immediately before the existing FREEUP(ep->data) line — no other change to the function's sequence
    • New unit test: set on_fini on a test endpoint, drive it through teardown, assert the hook fires exactly once and that ep->data is still valid (non-freed) at the moment the hook runs
    • Observable: the new unit test passes; every other existing endpoint (which never sets on_fini) tears down with zero behavioral change (hook is a no-op NULL check)
    • Requirements: 2.2

  • 2. Core: election carrier reference-counted lifecycle

  • 2.1 Add a reference count to the election carrier and a single release helper

    • Add an _Atomic int refcount field to ha_elect_ctx_t (src/controller/ha.h)
    • Add a static helper in ha.c that decrements the refcount and, on reaching zero, destroys the mutex/semaphore and frees the carrier
    • New unit test: call the helper once on a freshly allocated carrier (refcount=1) and assert it is freed; call it once on a carrier with refcount=2 and assert it is still valid with refcount=1
    • Observable: the helper is the only place ha_elect_ctx_t is destroyed/freed in ha.c
    • Requirements: 1.1
    • Boundary: ElectionCarrierLifecycle (ha.c/ha.h)

  • 2.2 (P) Increment the carrier refcount and wire the completion hook at the real-election vote-request capture site

    • In raft_cb_send_vote_request, at the point ep->data = vctx is committed (only for vctx instances that reach a live endpoint), atomically increment the carrier's refcount and set ep->on_fini to a new hook function that releases the carrier via the 2.1 helper
    • The two pre-endpoint_get() alloc-fail bailouts in this function do not increment (they never attach ep->data) and need no matching release
    • Observable: after a multi-node election round runs to completion and every vote endpoint's teardown finishes, the carrier's refcount reflects only the still-outstanding supersession bias reference (no endpoint-attributable count remains)
    • Requirements: 1.1, 2.1, 2.3, 2.4
    • Boundary: raft_cb_send_vote_request
    • Depends: 1.1, 2.1

  • 2.3 (P) Increment the carrier refcount and wire the completion hook at the PreVote vote-request capture site

    • Same mechanism as 2.2, applied to raft_cb_send_pre_vote_request
    • Observable: identical post-round refcount behavior as 2.2, but for PreVote rounds
    • Requirements: 1.1, 2.1, 2.3, 2.4, 3.1
    • Boundary: raft_cb_send_pre_vote_request
    • Depends: 1.1, 2.1

  • 2.4 Release the superseded carrier when a new election or PreVote round publishes its own

    • In ha_start_election_async and ha_start_prevote_async, change the carrier-publish assignment from atomic_store_explicit to atomic_exchange_explicit to retrieve the outgoing carrier pointer, then release it via the 2.1 helper if non-NULL
    • Route the two existing no-vote-endpoint fast-exit paths in ha_start_election_async (no-local-config bail-out, single-node case) through the same 2.1 helper instead of a direct free(), for one consistent destruction path
    • Update the "Do NOT free a prior ctx" comment at the PreVote publish site to describe the new mechanism instead of documenting the leak as deliberate
    • Observable: starting a second round while the first round's carrier already has zero endpoint references frees the first round's carrier immediately at supersession; starting a second round while the first still has an outstanding vote endpoint leaves the first carrier alive until that endpoint's teardown completes
    • Requirements: 1.1, 1.2, 3.1, 3.2
    • Boundary: ElectionCarrierLifecycle (ha.c publish sites)
    • Depends: 2.1, 2.2, 2.3

  • 3. Validation: leak closure and premature-free safety

  • 3.1 (P) Prove zero carrier leaks across many consecutive real-election rounds under leak-detection-enabled ASAN

    • New unit test drives N consecutive multi-node election rounds with ASAN leak detection enabled (distinct from the suite's current detect_leaks=0 default)
    • Temporarily revert the 2.2/2.4 changes to confirm this test fails (leak reported) pre-fix, then restore and confirm it passes (RED→GREEN)
    • Observable: zero ha_elect_ctx_t bytes reported leaked across the run
    • Requirements: 1.3
    • Boundary: election test coverage
    • Depends: 2.2, 2.4

  • 3.2 (P) Prove zero carrier leaks across many consecutive PreVote rounds under leak-detection-enabled ASAN

    • Same shape as 3.1, driving PreVote rounds via ha_start_prevote_async instead
    • Observable: zero ha_elect_ctx_t bytes reported leaked across the run
    • Requirements: 1.3, 3.1, 3.2
    • Boundary: PreVote test coverage
    • Depends: 2.3, 2.4

  • 3.3 (P) Prove the carrier is never freed while an outstanding vote endpoint still references it across a round supersession

    • Construct a round with one captured, not-yet-torn-down vote endpoint, then start a new round (triggering supersession) before that endpoint's teardown completes
    • Assert the original carrier is still valid (not freed) immediately after supersession, and becomes freed only after the outstanding endpoint's on_fini fires
    • Covers both the real-election and PreVote capture paths (parametrized or duplicated per design's Testing Strategy)
    • Observable: a naive "free on supersession" implementation would fail this test (UAF or premature free caught by ASAN); the refcounted implementation passes it
    • Requirements: 2.1, 2.2, 2.3, 3.1, 3.2
    • Boundary: no-premature-free test coverage
    • Depends: 2.2, 2.3, 2.4

  • 3.4 Run the full existing controller unit suite and confirm no behavioral change

    • Run the complete controller_unit_test suite; assertions in pre-existing election, PreVote, and failover test files must be byte-for-byte unchanged
    • Observable: full suite green, zero modified assertions in pre-existing election/prevote/failover tests
    • Requirements: 4.1, 4.2
    • Depends: 3.1, 3.2, 3.3

  • 3.5 Run a full HA sequential and chaotic regression soak and confirm no new job loss, divergence, or liveness stall

    • Run /xce-test-ha sequential and chaotic regression against a live 3-node cluster
    • Observable: zero committed job loss, zero divergence, no liveness stall attributable to this change; bug-339/bug-340 vote-freshness behavior unaffected
    • Requirements: 4.3
    • Depends: 3.4