diff --git a/.kiro/specs/ha-elect-ctx-lifecycle/requirements.md b/.kiro/specs/ha-elect-ctx-lifecycle/requirements.md new file mode 100644 index 0000000..7ac762d --- /dev/null +++ b/.kiro/specs/ha-elect-ctx-lifecycle/requirements.md @@ -0,0 +1,55 @@ +# Requirements Document + +## Introduction + +The HA election carrier (`ha_elect_ctx_t`, allocated as `mc` in both `ha_start_election_async` and `ha_start_prevote_async`, stored in `ha->elect_ctx`) is never freed today. Each failover, real-election, or PreVote round allocates a fresh carrier and deliberately leaks the previous one, because per-peer vote endpoints (`vote_ep_ctx_t.ctx`) hold a bare pointer into the carrier and are torn down asynchronously by `endpoint_fini` on the iomgr thread — a time the round-starting thread cannot cheaply observe. A naive free at round-start would risk a use-after-free if a still-live endpoint callback later dereferences the superseded carrier. + +This feature gives the election carrier a real, safe lifetime: it is freed once no vote endpoint can still reference it, and never freed while one might. It applies uniformly to both the real-election and PreVote carrier paths, since they share `ha->elect_ctx` and the identical hazard — fixing one without the other leaves the UAF risk in place for the unfixed path. + +This is a memory-lifetime fix only. It must not change election, PreVote, or failover semantics, and must not weaken or alter any existing correctness guarantee (including the bug-339/bug-340 vote-freshness fixes). + +> Full discovery context (problem statement, code pointers, deferred-priority rationale, implementation options) is in `.kiro/specs/ha-elect-ctx-lifecycle/brief.md` and `.kiro/specs/ha-elect-ctx-lifecycle/research.md`. + +## Boundary Context + +- **In scope**: the lifetime of the `ha_elect_ctx_t` carrier allocated by `ha_start_election_async` and `ha_start_prevote_async`; safe reclamation once every `vote_ep_ctx_t` that references it (via `vctx->ctx`) has been freed; coverage of every capture site (`raft_cb_send_vote_request`, `raft_cb_send_pre_vote_request`) and every `vctx` free site (including `endpoint_fini` FREEUP and error/abort paths that never reach a live endpoint); the memory-order discipline between the reclamation mechanism and the existing `atomic_store(&ha->elect_ctx, ..., release)` publish / acquire reads. +- **Out of scope**: the PreVote protocol itself (shipped and validated in `ha-prevote`); any change to election, vote-granting, or failover *semantics*; the `ha_elect_ctx_t` struct's field contents or tally logic; the async endpoint-teardown ordering itself (`endpoint_fini`, `vote_ep_teardown_async`) beyond observing when it completes. +- **Adjacent expectations**: relies on the existing per-endpoint teardown path (`endpoint_fini` freeing `vctx`) continuing to run exactly as it does today — this feature observes that teardown, it does not change when or how an endpoint tears down. Must remain compatible with the existing "do NOT free a prior ctx" UAF-caution comments elsewhere in `ha.c` (the rdhup path, delaymgr teardown) — this feature is what finally makes one of those comments obsolete for the carrier specifically, without touching the others' own hazards. + +## Requirements + +### Requirement 1: Bounded carrier lifetime + +**Objective:** As a maintainer, I want the election carrier to be freed once it is no longer referenced, so that a long-running controller does not accumulate one leaked carrier per failover/election/PreVote round indefinitely. + +#### Acceptance Criteria +1. When every `vote_ep_ctx_t` that captured a reference to an `ha_elect_ctx_t` carrier has been freed, the Controller shall reclaim that carrier. +2. The Controller shall reclaim a superseded carrier without requiring a bounded wait, timeout, or external trigger — reclamation shall be driven by the actual release of the last referencing endpoint. +3. Under an ASAN build with leak detection enabled, running many consecutive election/PreVote rounds shall report zero leaked `ha_elect_ctx_t` allocations. + +### Requirement 2: No use-after-free on the endpoint-teardown path + +**Objective:** As a maintainer, I want carrier reclamation to never race a live endpoint's use of that carrier, so that fixing the leak cannot introduce a use-after-free. + +#### Acceptance Criteria +1. The Controller shall never free an `ha_elect_ctx_t` carrier while any `vote_ep_ctx_t` still holds an unfreed reference to it (`vctx->ctx`). +2. When a vote endpoint's asynchronous teardown (`endpoint_fini` FREEUP) reads or dereferences its captured carrier, that carrier shall still be valid at the time of that read, regardless of whether a newer election/PreVote round has since started. +3. The Controller shall apply the same safe-reclamation mechanism to every capture site of the carrier (`raft_cb_send_vote_request` for real elections, `raft_cb_send_pre_vote_request` for PreVote), so no capture site is left with the old leak-forever behavior once this feature lands. +4. The Controller shall apply the same safe-reclamation mechanism on every `vctx` free path, including error and abort paths that free a `vote_ep_ctx_t` without the endpoint ever reaching a live connected state. + +### Requirement 3: Uniform coverage across real-election and PreVote + +**Objective:** As a maintainer, I want the fix applied identically to both carrier-allocating paths, so that neither path is left with the original UAF-risk-driven leak. + +#### Acceptance Criteria +1. The Controller shall use the same carrier-lifetime mechanism for carriers allocated by `ha_start_election_async` (real election) and `ha_start_prevote_async` (PreVote). +2. When a PreVote round is superseded by a new round (e.g. a suppressed disruptive rejoin or a fresh liveness-triggered campaign), the superseded round's carrier shall become eligible for reclamation exactly as a superseded real-election carrier does. + +### Requirement 4: No regression to election, PreVote, or failover correctness + +**Objective:** As an operator, I want this memory-lifetime change to leave voting and failover behavior completely unchanged, so that fixing a leak cannot introduce a liveness or safety regression. + +#### Acceptance Criteria +1. The Controller shall preserve all existing vote-tally, quorum, and freshness-gate behavior (including the bug-339/bug-340 vote-freshness fixes) unchanged by this feature. +2. The Controller shall preserve all existing election, PreVote, and failover timing and outcome behavior unchanged by this feature — this feature shall not be observable from vote outcomes, election timing, or failover convergence. +3. A full HA soak (sequential and chaotic `/xce-test-ha` regression) run after this feature lands shall show no new job loss, divergence, or liveness stall attributable to this change. diff --git a/.kiro/specs/ha-elect-ctx-lifecycle/spec.json b/.kiro/specs/ha-elect-ctx-lifecycle/spec.json index 398f70f..c89d85d 100644 --- a/.kiro/specs/ha-elect-ctx-lifecycle/spec.json +++ b/.kiro/specs/ha-elect-ctx-lifecycle/spec.json @@ -1,12 +1,12 @@ { "feature_name": "ha-elect-ctx-lifecycle", "created_at": "2026-06-16T03:10:00Z", - "updated_at": "2026-06-16T03:10:00Z", + "updated_at": "2026-07-09T17:15:00Z", "language": "en", - "phase": "initialized", + "phase": "requirements-generated", "approvals": { "requirements": { - "generated": false, + "generated": true, "approved": false }, "design": {