lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

test(ha-prevote): Raft safety-invariant property tests + RN-1 consistency argument (task 6.1)

Browse source 
Four property-style raft tests proving the PreVote API preserves the safety
invariants: term/voted_for/role never change and on_persist never fires across
mixed begin/request/reply sequences (WON and LOST); a WON round creates no leader
(stays FOLLOWER); and the real-vote higher-term-adopt + step-down + quorum-elect
path is unchanged (bug-339/340 intact). Records the TLA+/checklist consistency
argument (RN-1) in test_raft.c and research.md, citing the tests as evidence.
Raft suite 123/0/0; zero production change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-06-16 07:18:06 +00:00
parent 22fa753fa6
commit e6b234a6a4
3 changed files with 557 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

303
src/lib/testsuite/test_raft.c
View file

@ -3554,6 +3554,303 @@ static void test_pre_vote_reply_5node_won_at_three(void)
TEST_ASSERT_EQUAL_INT(0, g_mock.become_follower_count);
}
/* ====================================================================== *
* Task 6.1 — Raft safety-invariant assertions for the PreVote API
*
* The per-function tests above (tasks 2.1/2.2/2.3) check individual cases.
* The tests below assert the SAFETY INVARIANTS as *properties* that hold
* across the ENTIRE PreVote API, mechanically backing the RN-1 TLA+/
* verification-checklist consistency argument recorded in
* .kiro/specs/ha-prevote/research.md (§"RN-1 resolved").
*
* ---------------------------------------------------------------------- *
* RN-1 CONSISTENCY ARGUMENT (mechanically evidenced below)
* ---------------------------------------------------------------------- *
* Claim: adding the PreVote phase preserves every Raft safety invariant
* the repository's `raft/raft.tla` reference and the controller
* verification checklist require — Election Safety (at most one leader per
* term), Leader Completeness, term monotonicity, and a monotonic commit
* index — so the system stays consistent after the change.
*
* Proof structure (purity ⇒ no new transitions):
*
* 1. The three PreVote functions — raft_begin_pre_vote,
* raft_on_pre_vote_request, raft_on_pre_vote_reply — are PURE with
* respect to the persistent Raft state. Across an arbitrary sequence of
* begin / request / reply calls (mixed grant+deny, duplicates,
* out-of-range peers, and both WON and LOST terminal outcomes), none of
* them ever:
* - changes s->term, s->voted_for, s->voted_for_term, or s->role, or
* - invokes on_persist, or
* - invokes on_become_leader / on_become_follower.
* (Asserted by test_invariant_prevote_sequence_pure_no_persist and
* test_invariant_prevote_won_grants_no_leadership.)
*
* 2. Because the PreVote path mutates no term, it cannot violate term
* monotonicity: the only writer of s->term remains raft_start_election
* (real term bump) and raft_on_vote_request Step 2 (adopt a strictly
* higher term) — both UNCHANGED by this feature. A PreVote can only
* *gate* whether raft_start_election is later called; it never advances
* a term itself. (Asserted by the term-constant checks below.)
*
* 3. Because the PreVote path creates no leader (no on_become_leader, no
* role := RAFT_LEADER), it adds no new way to satisfy the antecedent of
* Election Safety. The sole leadership-creating transitions remain
* raft_start_election's single-node short-circuit and
* raft_on_vote_reply's quorum branch — again UNCHANGED. A PreVote round
* reaching RAFT_PRE_VOTE_WON returns a STATUS only; the node stays
* RAFT_FOLLOWER until the separate, unchanged real-election path runs.
* (Asserted by test_invariant_prevote_won_grants_no_leadership.)
*
* 4. The legitimate higher-term-adopt failover path (the bug-339/bug-340
* fix) lives entirely in raft_on_vote_request / raft_on_vote_reply,
* which this feature does not touch. PreVote sits strictly *before* that
* path and, by (1), leaves its inputs (term/voted_for/role) untouched.
* test_invariant_real_vote_higher_term_adopt_unchanged demonstrates the
* real-vote path still adopts a higher term and fires become_follower /
* become_leader exactly as before, even after a full PreVote round has
* run on the same node — so PreVote did not alter the legitimate
* failover semantics bug-339/bug-340 depend on.
*
* Conclusion: PreVote introduces no transition that writes a term or
* creates a leader. Every term-writing / leadership-creating transition in
* the model is byte-for-byte the pre-PreVote transition. Therefore the set
* of reachable (term, leader) configurations is unchanged, and the TLA+
* invariants (Election Safety, Leader Completeness, term monotonicity,
* monotonic commit index) and the controller verification checklist
* continue to hold. The tests below are the mechanical evidence for
* premises (1)–(4).
* ====================================================================== */
/*
* test_invariant_prevote_sequence_pure_no_persist (Req 6.2, 2.3, 1.2)
*
* PROPERTY (term monotonicity / no persistent mutation): drive a long,
* mixed sequence of PreVote operations through a node and assert that across
* the WHOLE sequence the persistent Raft state (term, voted_for,
* voted_for_term, role) never changes and on_persist is NEVER called.
*
* The sequence deliberately exercises: begin → answer an incoming request
* (grant) → answer a stale-term request (deny) → reply tally with a grant,
* a duplicate of that grant, an out-of-range peer, and a deny — i.e. every
* PreVote entry point and both grant/deny outcomes, including a WON
* terminal. If ANY PreVote code path leaked a term/vote/role write or a
* persist, this property fails — which would be a real Election-Safety /
* term-monotonicity finding, not a test bug.
*/
static void test_invariant_prevote_sequence_pure_no_persist(void)
{
raft_state_t s;
raft_callbacks_t cbs = make_mock_cbs();
cbs.on_vote_eligible = mock_vote_eligible;
cbs.on_trace = mock_trace;
reset_mock();
g_mock.vote_eligible_return_value = 1; /* grant when consulted */
/* 3-node FOLLOWER at term=5, having voted for peer 1 in an earlier term.
* These persistent values are the canary: nothing below may change them. */
TEST_ASSERT_EQUAL_INT(0, init_vote_node(&s, &cbs, 3, 0, 5));
s.voted_for = 1;
s.voted_for_term = 4;
const uint64_t term0 = s.term;
const int voted_for0 = s.voted_for;
const uint64_t voted_for_term0 = s.voted_for_term;
const raft_role_t role0 = s.role;
/* (a) begin a round — self-grant + prospective term, no persist */
TEST_ASSERT_EQUAL_INT(0, raft_begin_pre_vote(&s));
/* (b) answer an incoming PreVote request that PASSES eligibility — grant */
TEST_ASSERT_EQUAL_INT(1, raft_on_pre_vote_request(&s, 2, 6 /* higher */));
/* (c) answer a STALE-term incoming request — deny */
TEST_ASSERT_EQUAL_INT(0, raft_on_pre_vote_request(&s, 2, 3 /* stale */));
/* (d) answer an incoming request the eligibility gate REJECTS — deny */
g_mock.vote_eligible_return_value = 0;
TEST_ASSERT_EQUAL_INT(0, raft_on_pre_vote_request(&s, 2, 6));
g_mock.vote_eligible_return_value = 1;
/* (e) tally replies: a grant (drives to WON quorum on 3-node), then a
* duplicate of the same grant, an out-of-range peer, and a deny. */
TEST_ASSERT_EQUAL_INT(RAFT_PRE_VOTE_WON,
raft_on_pre_vote_reply(&s, 1, 1 /* grant */));
/* round closed after WON; further replies are no-ops on persistent state */
(void)raft_on_pre_vote_reply(&s, 1, 1 /* duplicate grant */);
(void)raft_on_pre_vote_reply(&s, 99, 1 /* out-of-range */);
(void)raft_on_pre_vote_reply(&s, 2, 0 /* late deny */);
/* INVARIANT: persistent Raft state byte-for-byte unchanged. */
TEST_ASSERT_EQUAL_UINT64(term0, s.term);
TEST_ASSERT_EQUAL_INT(voted_for0, s.voted_for);
TEST_ASSERT_EQUAL_UINT64(voted_for_term0, s.voted_for_term);
TEST_ASSERT_EQUAL_INT(role0, s.role);
TEST_ASSERT_EQUAL_INT(RAFT_FOLLOWER, s.role); /* never promoted */
/* INVARIANT: PreVote never persists and never fires leadership callbacks. */
TEST_ASSERT_EQUAL_INT(0, g_mock.persist_count);
TEST_ASSERT_EQUAL_INT(0, g_mock.become_leader_count);
TEST_ASSERT_EQUAL_INT(0, g_mock.become_follower_count);
}
/*
* test_invariant_prevote_lost_sequence_pure_no_persist (Req 6.2, 1.4)
*
* Same purity property as above, but driving the round to the LOST terminal
* outcome (all peers deny). A WON path and a LOST path are the two ways a
* round can end; both must be side-effect-free. This guards the case where
* a frozen-then-resumed standby gets no PreVote majority (Req 4) — the round
* must leave term/vote/role untouched so the node simply re-attaches.
*/
static void test_invariant_prevote_lost_sequence_pure_no_persist(void)
{
raft_state_t s;
raft_callbacks_t cbs = make_mock_cbs();
cbs.on_trace = mock_trace;
reset_mock();
TEST_ASSERT_EQUAL_INT(0, init_vote_node(&s, &cbs, 3, 0, 9));
s.voted_for = 0;
s.voted_for_term = 9;
const uint64_t term0 = s.term;
const int voted_for0 = s.voted_for;
const uint64_t voted_for_term0 = s.voted_for_term;
const raft_role_t role0 = s.role;
TEST_ASSERT_EQUAL_INT(0, raft_begin_pre_vote(&s));
/* both peers deny → LOST, no majority */
TEST_ASSERT_EQUAL_INT(RAFT_PRE_VOTE_PENDING,
raft_on_pre_vote_reply(&s, 1, 0));
TEST_ASSERT_EQUAL_INT(RAFT_PRE_VOTE_LOST,
raft_on_pre_vote_reply(&s, 2, 0));
/* INVARIANT: persistent state unchanged across a LOST round. */
TEST_ASSERT_EQUAL_UINT64(term0, s.term);
TEST_ASSERT_EQUAL_INT(voted_for0, s.voted_for);
TEST_ASSERT_EQUAL_UINT64(voted_for_term0, s.voted_for_term);
TEST_ASSERT_EQUAL_INT(role0, s.role);
TEST_ASSERT_EQUAL_INT(0, g_mock.persist_count);
TEST_ASSERT_EQUAL_INT(0, g_mock.become_leader_count);
TEST_ASSERT_EQUAL_INT(0, g_mock.become_follower_count);
}
/*
* test_invariant_prevote_won_grants_no_leadership (Req 6.1)
*
* ELECTION SAFETY property: a PreVote round that reaches RAFT_PRE_VOTE_WON
* grants NO leadership. The node must remain RAFT_FOLLOWER and no
* on_become_leader / on_become_follower callback may fire — a PreVote WON is
* a STATUS the controller acts on, not a leadership transition. Only the
* separate, unchanged raft_start_election path creates a leader.
*
* The test then demonstrates the contrast: invoking the real
* raft_start_election (the only leadership-creating path) from the same
* still-FOLLOWER node bumps the term and self-votes — proving the leader is
* created exclusively by the real path, never by the PreVote WON itself.
*/
static void test_invariant_prevote_won_grants_no_leadership(void)
{
raft_state_t s;
raft_callbacks_t cbs = make_mock_cbs();
reset_mock();
/* arm a round on a 3-node FOLLOWER at term=5 (self-grant in progress) */
TEST_ASSERT_EQUAL_INT(0, arm_pre_vote_round(&s, &cbs, 3, 0, 5));
const uint64_t term_before_won = s.term;
/* one peer grant → WON on a 3-node cluster (quorum=2) */
int rc = raft_on_pre_vote_reply(&s, 1, 1 /* grant */);
TEST_ASSERT_EQUAL_INT(RAFT_PRE_VOTE_WON, rc);
/* INVARIANT (Election Safety): WON created NO leader. */
TEST_ASSERT_EQUAL_INT(RAFT_FOLLOWER, s.role); /* still a follower */
TEST_ASSERT_EQUAL_UINT64(term_before_won, s.term); /* term not advanced */
TEST_ASSERT_EQUAL_INT(0, g_mock.become_leader_count);
TEST_ASSERT_EQUAL_INT(0, g_mock.become_follower_count);
TEST_ASSERT_EQUAL_INT(0, g_mock.persist_count);
/* CONTRAST: leadership is created ONLY by the unchanged real path.
* Starting the real election now bumps the term + records the self-vote
* (this is the sole term-writing / leadership-creating transition). */
reset_mock();
TEST_ASSERT_EQUAL_INT(RAFT_OK, raft_start_election(&s));
TEST_ASSERT_EQUAL_INT(RAFT_CANDIDATE, s.role);
TEST_ASSERT_EQUAL_UINT64(term_before_won + 1, s.term); /* real bump here */
TEST_ASSERT_EQUAL_INT(1, g_mock.persist_count); /* persisted here */
}
/*
* test_invariant_real_vote_higher_term_adopt_unchanged (Req 5.2, 5.3)
*
* REAL-VOTE-PATH-UNCHANGED property: PreVote must not have altered the
* legitimate higher-term-adopt failover path that the bug-339/bug-340 fixes
* depend on. After running a full PreVote round on a node, the real
* raft_on_vote_request still adopts a strictly higher term and steps a
* leader down (fires on_become_follower), and raft_on_vote_reply still wins
* at quorum (fires on_become_leader) — exactly as before PreVote existed.
*
* This is the library-level guard for bug-339 (higher-term adopt on
* RDHUP failover) and bug-340 (rejoin), confirming PreVote sits beside the
* real-vote path without modifying it.
*/
static void test_invariant_real_vote_higher_term_adopt_unchanged(void)
{
raft_state_t s;
raft_callbacks_t cbs = make_mock_cbs();
uint64_t reply_term = 0;
cbs.on_trace = mock_trace;
/* --- Part 1: higher-term real RequestVote adopts term + steps down a
* leader, even after a PreVote round ran on this node. --- */
reset_mock();
TEST_ASSERT_EQUAL_INT(0, init_vote_node(&s, &cbs, 3, 0, 2));
/* Run a (lost) PreVote round first to prove it leaves the real-vote
* adopt path untouched. */
TEST_ASSERT_EQUAL_INT(0, raft_begin_pre_vote(&s));
(void)raft_on_pre_vote_reply(&s, 1, 0);
(void)raft_on_pre_vote_reply(&s, 2, 0);
TEST_ASSERT_EQUAL_UINT64(2, s.term); /* PreVote did not move the term */
/* Now this node is the leader in term 2 and a real higher-term vote
* request arrives — the unchanged Step 2 must adopt term 5 and step down. */
s.role = RAFT_LEADER;
s.voted_for = 0;
s.voted_for_term = 2;
reset_mock();
int rc = raft_on_vote_request(&s, 1, 5, &reply_term);
/* Higher-term adopt + leader step-down, exactly as bug-339/340 require. */
TEST_ASSERT_EQUAL_UINT64(5, s.term);
TEST_ASSERT_EQUAL_INT(RAFT_FOLLOWER, s.role);
TEST_ASSERT_EQUAL_UINT64(5, reply_term);
TEST_ASSERT_EQUAL_INT(1, g_mock.become_follower_count); /* stepped down */
TEST_ASSERT_EQUAL_INT(RAFT_GRANT, rc); /* then grants */
/* --- Part 2: real raft_on_vote_reply still elects a leader at quorum,
* unaffected by PreVote. --- */
reset_mock();
TEST_ASSERT_EQUAL_INT(0, init_node(&s, &cbs, 3, 0));
s.role = RAFT_CANDIDATE;
s.term = 1;
s.votes_granted = 1; /* self-vote */
s.replies_received = 0;
s.election_decided = 0;
int rcr = raft_on_vote_reply(&s, 1, 1 /* grant */, 1);
TEST_ASSERT_EQUAL_INT(RAFT_BECAME_LEADER, rcr);
TEST_ASSERT_EQUAL_INT(RAFT_LEADER, s.role);
TEST_ASSERT_EQUAL_INT(1, g_mock.become_leader_count); /* real path elects */
}
/* ---------- test_raft entry point ---------- */
void test_raft(void)
@ -3634,6 +3931,12 @@ void test_raft(void)
RUN_TEST(test_pre_vote_reply_out_of_range_peer_ignored);
RUN_TEST(test_pre_vote_reply_5node_won_at_three);
/* Raft safety-invariant assertions for the PreVote API (task 6.1, RN-1) */
RUN_TEST(test_invariant_prevote_sequence_pure_no_persist);
RUN_TEST(test_invariant_prevote_lost_sequence_pure_no_persist);
RUN_TEST(test_invariant_prevote_won_grants_no_leadership);
RUN_TEST(test_invariant_real_vote_higher_term_adopt_unchanged);
/* simulator init (task 4.1) */
#ifdef XCE_TESTING
RUN_TEST(test_sim_init_3node);