lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(controller): probe a late-joining standby at once so its catch-up starts without waiting a heartbeat (bug-1913)

Browse source 
After an election where the vote connections are not promoted, both
standbys register later via on_standby_connected; the own-term no-op was
appended before any standby existed, and catch-up waited for heartbeat-
driven state reports (config push one 3 s cycle, WAL stream the next), so
the no-op committed ~6 s late and every client write in that window
failed -110.

ha_send_catchup_probe() sends the ordinary heartbeat request (same fields
as primary_heartbeat_fn; encode + enqueue only; primary only) at
registration and once after the standby's first accepted config-push ACK,
so the decision loop's next action follows at ~RTT. The decision loop,
its one-action-per-report pacing, commit/majority rules and the dispatch
gate are unchanged; WAL streaming still runs only on the vtask. The
one-shot arm is cleared on disconnect and on every become_leader /
become_follower.

Tests: test_late_joiner_catchup_kick.c (6). Controller suite 1285/0.
Live: catch-up starts 0-6 ms after a late registration (was up to ~6 s);
/xce-test-ha sequential 20 rounds, chaotic 6 rounds and sequential 8
rounds under 40-spinner load all PASS with zero job loss.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-03 23:18:10 +00:00
parent 6d9ef3d024
commit 2aefa939db
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
5 changed files with 485 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

19
src/controller/ha.h
View file

@ -289,6 +289,25 @@ typedef struct {
* Slot identity (the repl registry slot
* index) is stable across schedule/run.
* 0 at init (cleared by ha_init memset). */
_Atomic int catchup_kick_armed[MAX_HA_CANDIDATES]; /* bug-1913: per-standby
* (absolute peer index) one-shot arm for
* the catch-up probe. Set by
* on_standby_connected when a standby
* registers; consumed (1->0 CAS) by that
* standby's first ACCEPTED config-push
* ACK, which then sends one heartbeat
* probe so the decision loop's next
* action (WAL stream) follows the config
* push at ~RTT instead of on the next
* heartbeat tick. One-shot so a standby
* whose reported tag stays behind cannot
* drive a probe/push/ACK loop.
* Cleared on the standby's disconnect
* (standby_ep_err_handler_primary) and
* on every become_leader/become_follower,
* so an arm never outlives its
* registration or leadership.
* 0 at init (cleared by ha_init memset). */
/* ── New field added by ha-convergence-recovery spec (Task 5.x) ──────── */
_Atomic int quorum_lost; /* Quorum-transition edge detector (Req
* 8.1-8.3): 0 = quorum held (or unknown