feat(ha-prevote): add preVote request/reply wire opcodes (task 1.1)

Append preVoteRequest @38 / preVoteReply @39 to the OPCode enum, reusing the
existing RequestVoteRequest/RequestVoteReply payload structs and union arms
(no new structs, no isPreVote flag). Keeps the real-vote wire path unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-06-16 03:59:47 +00:00
parent 646aa69aa8
commit 0d660264f5
2 changed files with 94 additions and 0 deletions

View file

@ -0,0 +1,92 @@
# Implementation Plan
> TDD is mandatory (project rule): for each implementation sub-task, add the failing test in the relevant `testsuite/` first (RED), then implement to green. Dependency direction: capnp/types → raft library → controller adapter → dispatch/transport. The real-vote state machine is reused unchanged after PreVote success.
- [ ] 1. Foundation: wire schema and raft-library PreVote scaffolding
- [x] 1.1 Add PreVote opcodes to the election wire protocol
- Add two opcodes for the PreVote request and PreVote reply, reusing the existing vote request/reply payload structures and their message union arms (no new payload structs).
- Regenerate the generated bindings and confirm the build produces the new opcode constants while existing vote opcode ordinals are unchanged.
- _Requirements: 7.1_
- _Boundary: PreVote dispatch (wire schema)_
- [ ] 1.2 (P) Add PreVote types, state, and callback seam to the raft library
- Add PreVote outcome codes (won / lost / pending), a self-contained PreVote tally on the raft state (granted count, per-peer replied tracking, in-progress flag), and an optional send-pre-vote callback in the callback table.
- Header compiles with the new declarations and existing election state/layout is unchanged; no behavior added yet.
- _Requirements: 1.2, 6.2_
- _Boundary: raft pre-vote functions_
- [ ] 2. Core: raft-library pure PreVote decision and tally (test-first, no side effects)
- [ ] 2.1 Implement the voter-side PreVote decision
- RED first: add raft unit tests asserting grant for a fresh, higher-or-equal-term candidate when the eligibility callback allows it, and deny on stale term or when the eligibility callback rejects.
- Reuse the existing vote-eligibility callback (primary-deny, fresh-heartbeat stickiness, catch-up deny) and let the dispatcher apply freshness beforehand, exactly as the real-vote path does.
- Observable: after a granted PreVote the test confirms term, voted-for, and role are all unchanged and no persist callback fired.
- _Requirements: 2.1, 2.2, 2.3, 2.4, 3.1, 3.2, 3.3, 6.2_
- _Boundary: raft pre-vote functions_
- [ ] 2.2 Implement the candidate-side PreVote start
- RED first: add a raft unit test asserting the prospective term used is current+1, that it is never written to the persistent term and never persisted, and that the send-pre-vote callback fires once per peer with a self-grant recorded.
- Observable: test confirms the persistent term/voted-for are byte-for-byte unchanged after starting the round.
- _Requirements: 1.1, 1.2, 1.3, 6.2_
- _Boundary: raft pre-vote functions_
- _Depends: 1.2_
- [ ] 2.3 Implement the PreVote reply tally
- RED first: add raft unit tests asserting the tally lives on the raft state, returns won at quorum, lost when all peers reply without quorum, pending otherwise, and ignores duplicate replies from the same peer.
- Mirror the production real-vote tally pattern, but never fire become-leader/become-follower — return a status the caller acts on.
- Observable: test confirms role and term unchanged across won/lost/pending outcomes.
- _Requirements: 1.4, 1.5, 6.2_
- _Boundary: raft pre-vote functions_
- _Depends: 1.2_
- [ ] 3. Core: controller voter-side PreVote dispatch (test-first)
- [ ] 3.1 Answer incoming PreVote requests with stickiness + freshness, no mutation
- RED first: extend the vote-gate tests so a PreVote from a fresh candidate is denied while a leader is believed alive (primary role, or standby with fresh heartbeat, or catch-up) and granted once stickiness lapses and freshness passes.
- Compute the freshness floor and run the eligibility gate before the raft decision, reusing the real-vote dispatch shape; reply with the voter's unchanged term plus the grant bit and freshness fields; log every denial with candidate id and reason.
- Observable: a denied PreVote reply carries the voter's original term (no term adoption) and a diagnosable denial-reason log line is emitted.
- _Requirements: 2.1, 2.2, 3.1, 3.2, 3.3, 7.1, 7.2_
- _Boundary: PreVote dispatch_
- _Depends: 1.1, 2.1_
- [ ] 4. Core: controller candidate-side orchestration and trigger restructuring (test-first)
- [ ] 4.1 (P) Run the PreVote round and branch on its outcome
- RED first: add controller tests driving the reply handler through the existing vote-reply hook (no real transport): majority grant → real election started; no majority → node stays standby with term unchanged and the liveness watchdog re-armed; round timeout treated as a loss.
- Owns the candidate orchestration in the controller: start the round (no persist, no role change), arm a round timeout bounded by the existing election-probe deadline, and the PreVote-reply handler that feeds the raft tally and branches — on a win perform the existing truncate + electing-transition + start-real-election sequence (unchanged real-election path); on loss log campaign abandonment. The send path emits the PreVote request opcode from task 1.1; transport-side reply routing is wired separately in task 5.1.
- Observable: on no-majority the cluster term is unchanged, role stays standby, and a subsequent genuine failure still triggers a new round.
- _Requirements: 1.1, 1.4, 1.5, 5.4, 7.3_
- _Boundary: PreVote orchestration_
- _Depends: 1.1, 2.2, 2.3_ (parallel-safe with 3.1: orchestration lives in a different file than the voter-side dispatch)
- [ ] 4.2 Route failover triggers through PreVote and defer the destructive step
- RED first: add tests asserting the failover trigger no longer truncates the WAL or increments the term until PreVote wins, and that an incoming primary heartbeat mid-round cancels the round and re-attaches as standby.
- Make both failover entry points (connection RDHUP and heartbeat-timeout poll) begin the PreVote round instead of starting an election directly; move truncate + electing-transition to PreVote success; cancel an in-progress round on heartbeat receipt.
- Observable: a frozen-then-resumed standby that loses PreVote leaves the WAL and term untouched and re-attaches when the next heartbeat arrives.
- _Requirements: 1.1, 4.2, 4.3, 4.4_
- _Boundary: PreVote orchestration (trigger path)_
- _Depends: 4.1_
- [ ] 5. Integration: transport wiring and end-to-end scenarios
- [ ] 5.1 Wire the PreVote reply opcode case to the candidate handler
- Add the transport dispatch case for the incoming PreVote-reply opcode so it routes to the reply handler built in 4.1 (replacing the test hook in production), paralleling the real-vote reply dispatch case.
- Observable: a PreVote reply arriving over the real transport reaches the candidate's tally and can drive the round to a decision end-to-end (not just via the test hook).
- _Requirements: 7.1_
- _Boundary: PreVote dispatch_
- _Depends: 1.1, 3.1, 4.1_ (shares the dispatch file with 3.1; owns only the opcode-to-handler routing, while 4.1 owns the handler)
- [ ] 5.2 Frozen-then-resumed no-op scenario test
- Add a 3-node scenario test: primary alive, a resumed standby starts a PreVote round; both peers deny (primary role + fresh heartbeat).
- Observable: no real vote request is sent, the cluster term is unchanged across the event, and the primary stays primary (no demotion/truncate/self-restart).
- _Requirements: 4.1, 4.3, 4.4_
- _Depends: 3.1, 4.1, 4.2, 5.1_
- [ ] 5.3 Genuine-failover and re-attach scenario tests
- Add scenario tests: dead primary with stale peer heartbeats → PreVotes granted → a new primary is elected within the existing election-timeout bounds; heartbeat mid-round → clean re-attach; failed PreVote → liveness watchdog re-armed so a later real failure still elects.
- Observable: a genuine primary death elects a new primary, and the suppressed cases never do.
- _Requirements: 5.1, 5.4, 4.2_
- _Depends: 4.1, 4.2, 5.1_
- [ ] 6. Validation: safety invariants and regression guard
- [ ] 6.1 Assert Raft safety invariants hold with PreVote
- Add raft-library invariant assertions that the PreVote functions mutate no persistent state (Election Safety and term monotonicity preserved), and record the TLA+/verification-checklist consistency argument (the real-vote path is unchanged, so the higher-term-adopt failover and the bug-339/bug-340 fixes are preserved).
- Observable: invariant tests pass and the existing failover/rejoin regression tests remain green with PreVote enabled.
- _Requirements: 5.2, 5.3, 6.1, 6.2, 6.3, 6.4_
- _Depends: 2.1, 2.2, 2.3_

View file

@ -46,6 +46,8 @@ enum OPCode {
upgradeConfigAck @35;
snapshotTransfer @36;
walCatchupEntry @37;
preVoteRequest @38;
preVoteReply @39;
}
struct PeerInfo $C.mapname("xce_peerinfo_t") {