wdgzyjx/myxce
1
0
Fork 0
forked from lei.chen/myxce
Code Pull requests Activity

fix(controller): drop RDHUP while electing — break dueling-candidate term escalation (task 5.6)

Browse source 
After 5.5 let the freshest survivor win PreVote and ENTER the real election,
round 4 still stalled: the dead primary's control fd keeps RDHUP'ing on a
deterministic ~15s cadence, and ha_on_primary_ctrl_fd_rdhup's 5.2 suppress only
catches CTRL_STANDBY — so an ELECTING node fell through to ha_begin_failover and
restarted a fresh PreVote, preempting its own in-progress real election. Every
cycle bumped the term (21->25->28->29...) and kept two survivors dueling in
lockstep, defeating the jittered election-timeout tiebreak.

Guard the RDHUP entry: if role==CTRL_ELECTING or a PreVote round is in progress,
close the dead control fd and return WITHOUT restarting failover — let the
in-progress round finish or time out (its election timeout is jittered, which
desynchronizes the duel so one survivor wins). A node not currently electing
still takes the normal 5.2-suppress / genuine-loss failover path.

RED+PIN in test_ha_core_safety.c (22 Tests / 0 Failures): CORE RED — CTRL_ELECTING
+ RDHUP must not start a new PreVote (Expected 0 Was 1 on baseline); PIN — genuine
loss (STANDBY, not electing) still fails over. Preserves Election Safety, 5.2
suppression, and genuine-loss liveness. Re-running 7.2.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019dbTHR9CsEYK9RsqmQpQpC
This commit is contained in:
Rongsong Shen 2026-06-30 11:47:09 +00:00
parent fa53feaf62
commit 81cb09a8db
2 changed files with 107 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

28
src/controller/ha.c
View file

@ -6739,6 +6739,34 @@ void ha_on_primary_ctrl_fd_rdhup(ha_state_t *ha, cluster_t *c)
{
if (!ha) return;
/* raft-ha-core-safety task 5.6: do NOT restart failover while an election is
* already in flight. A won PreVote enters the real election (CTRL_ELECTING),
* or a PreVote round may be mid-decision (pre_vote_in_progress). The dead
* primary's control fd keeps RDHUP'ing on a deterministic ~15s cadence — if
* each RDHUP re-triggers a fresh ha_begin_failover it PREEMPTS the in-progress
* real election before it can complete, bumping the term every cycle and
* keeping two surviving candidates dueling in lockstep (split vote, unbounded
* term escalation; research.md §"Task 5.5 fix VERIFIED"). Drop the RDHUP:
* close the dead control fd and let the in-progress round finish or time out —
* its election timeout is jittered, which desynchronizes the duel so one
* survivor wins. A node NOT currently electing falls through to the normal
* 5.2 suppress / genuine-loss failover path below. */
{
ctrl_role_t cur = (ctrl_role_t)atomic_load_explicit(&ha->role,
memory_order_acquire);
if (cur == CTRL_ELECTING || ha->raft.pre_vote_in_progress) {
if (ha->primary_ctrl_fd >= 0) {
close(ha->primary_ctrl_fd);
ha->primary_ctrl_fd = -1;
}
INFO("HA: primary_ctrl_fd RDHUP but an election is already in progress "
"(role=%d, prevote=%d) — dropping the RDHUP, not restarting "
"failover (task 5.6: don't preempt the in-progress election / "
"churn the term)", (int)cur, ha->raft.pre_vote_in_progress);
return;
}
}
/* Cancel the liveness poll task first (Req 5.4): prevents a concurrent
* poll fn from triggering a second failover call after EPOLLRDHUP fires. */
ha_disarm_heartbeat_timeout_vtask(ha);