fix(ha): election-safety hardening — vote gates, restored-vote term, freshness source

Election protocol fixes validated against raft.tla invariants:

- ha_init_raft(): one-time raft state init at startup (re-init would lose
  durable election progress); n_peers = full cluster size including self
- handle_request_vote(): higher-term step-down moved BEFORE the role gate
  (split-brain fix); standby vote-deny now requires a live primary control
  channel (primary_ctrl_fd >= 0) AND a fresh heartbeat
- ha_heartbeat_is_fresh(): derive freshness from ha->last_activity (the
  field production actually writes); the orphaned last_heartbeat timespec
  had no production writer, leaving the vote-deny gate inert (bug-337);
  field removed from ha_state_t
- handle_request_vote(): restore same-candidate idempotent re-grant —
  deny only a DIFFERENT candidate in the same term, matching raft.c and
  Raft §5.2 votedFor ∈ {Nil, candidateId} (bug-338)
- raft_init(): restore voted_for_term alongside a persisted vote; leaving
  it 0 allowed a double vote in the same term across a restart (bug-339)
- myapp_run(): leader_known check treats a vote recorded under this node's
  own name as a self-vote — production persists the real candidate name,
  the "self" placeholder only appears in test mode
- raft_on_vote_request(): leader steps down on higher-term vote request
  instead of denying it

Tests: test_proto_H6_restored_vote_blocks_second_vote added; H1 restore
assertion strengthened; test_ha_vote_gate.c driven through last_activity +
primary_ctrl_fd (the real production fields). Full /xce-build all green
(lib + controller + scheduler + Go suites, 0 failures, ASAN clean).

deploy-ha.sh: start.sh now forwards extra args to the controller binary.
This commit is contained in:
Rongsong Shen 2026-06-10 01:25:39 +00:00
parent fa66d1ba9c
commit 9821ec4f3c
7 changed files with 207 additions and 91 deletions

View file

@ -2029,6 +2029,9 @@ static void test_proto_H1_init_with_persisted_state(void)
TEST_ASSERT_EQUAL_INT(0, rc);
TEST_ASSERT_EQUAL_UINT64(5, s.term);
TEST_ASSERT_EQUAL_INT(1, s.voted_for);
/* The restored vote belongs to the restored term — without this, the
* duplicate-vote check cannot match after restart (see H6). */
TEST_ASSERT_EQUAL_UINT64(5, s.voted_for_term);
TEST_ASSERT_EQUAL_INT(RAFT_FOLLOWER, s.role);
}
@ -2061,6 +2064,51 @@ static void test_proto_H2_init_first_boot(void)
TEST_ASSERT_EQUAL_INT(RAFT_FOLLOWER, s.role);
}
/*
* test_proto_H6_restored_vote_blocks_second_vote
*
* Election Safety across restart: a node that voted for candidate 1 in
* term 5, then restarted (raft_init with initial_term=5, initial_voted_for=1),
* must DENY a vote request from a DIFFERENT candidate (idx=2) in the same
* term 5. Without restoring voted_for_term alongside voted_for, the
* duplicate-vote check cannot match the restored term and the node would
* grant a second vote in term 5 → two leaders in one term.
*/
static void test_proto_H6_restored_vote_blocks_second_vote(void)
{
raft_state_t s;
raft_callbacks_t cbs = make_mock_cbs();
uint64_t reply_term = 0;
reset_mock();
raft_config_t cfg;
memset(&s, 0, sizeof(s));
memset(&cfg, 0, sizeof(cfg));
cfg.n_peers = 3;
cfg.self_idx = 0;
cfg.election_timeout_ms = 1000;
cfg.initial_term = 5;
cfg.initial_voted_for = 1; /* persisted vote for candidate 1, term 5 */
cfg.cbs = &cbs;
TEST_ASSERT_EQUAL_INT(0, raft_init(&s, &cfg));
/* Different candidate (idx=2) requests a vote in the SAME term 5 */
int rc = raft_on_vote_request(&s, 2, 5, &reply_term);
TEST_ASSERT_EQUAL_INT(RAFT_DENY, rc);
TEST_ASSERT_EQUAL_UINT64(5, reply_term);
/* The restored vote must be untouched */
TEST_ASSERT_EQUAL_INT(1, s.voted_for);
TEST_ASSERT_EQUAL_UINT64(5, s.voted_for_term);
/* Same candidate (idx=1) retransmitting must still be re-granted */
rc = raft_on_vote_request(&s, 1, 5, &reply_term);
TEST_ASSERT_EQUAL_INT(RAFT_GRANT, rc);
}
/*
* test_proto_H3_persist_before_vote_recorded
*
@ -2964,6 +3012,7 @@ void test_raft(void)
RUN_TEST(test_proto_H3_persist_before_vote_recorded);
RUN_TEST(test_proto_H4_persist_fail_denies);
RUN_TEST(test_proto_H5_start_election_self_vote_persisted);
RUN_TEST(test_proto_H6_restored_vote_blocks_second_vote);
RUN_TEST(test_proto_Hnull_required_callback_einval);
/* Group D: Quorum and Result (task 5.3) */