lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(raft-drive-async): offload delaymgr peer reconnect to slow lane (task 2.1)

Browse source 
Heartbeat timer detects a disconnected peer, CAS-acquires a per-peer
single-flight flag, slowlane_submits the bounded connect, and returns — the
delaymgr never blocks on connect. Lane worker (reconnect_lane_fn) does the
bounded endpoint_connect + lock-guarded iomgr_monitor; flag cleared on all
terminal paths (incl. -EEXIST race fix). Default-ON kill-switch retains the
inline path for TDD-RED/fallback. Never self-terminates on failure. 2.2 adds
the inbound/outbound race + failure hardening (documented seam). 196/0 green.
This commit is contained in:
Rongsong Shen 2026-06-26 08:08:31 +00:00
parent a36234a103
commit e07667ff9b
4 changed files with 569 additions and 17 deletions
Download patch file Download diff file Expand all files Collapse all files

42
src/controller/ha.h
View file

@ -254,6 +254,19 @@ typedef struct {
* cleared and the config is activated.
* NEVER self-terminates (loud non-serving
* fence only). 0 at init (ha_init memset). */
/* ── New field added by raft-drive-async spec (Task 2.1) ──────── */
_Atomic int reconnect_inflight[MAX_HA_CANDIDATES]; /* Per-peer
* single-flight guard for the offloaded
* delaymgr reconnect (Req 2.5): set (CAS
* 0->1) when a reconnect to peer[i] is
* submitted to the slow lane, cleared by
* the lane worker on completion (success
* or failure) so the next heartbeat tick
* can retry. While set, the heartbeat
* reconnect scan enqueues no duplicate for
* that peer. Indexed by ABSOLUTE candidate
* index (peers_cfg.candidates[]). 0 at
* init (ha_init memset). */
} ha_state_t;
/* ── Public API ────────────────────────────────────────────── */
@ -1196,6 +1209,18 @@ void ha_disarm_heartbeat_timeout_vtask(ha_state_t *ha);
*/
void ha_start_primary_heartbeat(cluster_t *c);
/*
* ha_set_reconnect_offload_for_test / ha_reconnect_offload_enabled:
* Feature-flag accessors (raft-drive-async Task 2.1). The offload flag gates
* whether primary_heartbeat_fn submits the peer reconnect to the slow lane (ON,
* the production default — the delaymgr thread never blocks on connect) or runs
* the legacy inline blocking connect on the delaymgr thread (OFF, retained only
* so the TDD RED test can demonstrate the pre-change stall). These accessors
* are compiled in ALL builds; the setter is used by tests to flip the flag.
*/
void ha_set_reconnect_offload_for_test(int on);
int ha_reconnect_offload_enabled(void);
/* ── Task 2.4: Non-blocking async election initiator ────────────────────── */
/*
@ -1353,6 +1378,23 @@ extern int (*ha_heartbeat_req_hook)(endpoint_t *ep, uint64_t term,
uint64_t committed_lsn,
uint64_t committed_config_tag);
/*
* ha_reconnect_connect_hook: test-only seam for the BLOCKING half of the
* offloaded peer reconnect (raft-drive-async Task 2.1). The reconnect splits
* into a non-blocking prep (alloc endpoint, set callbacks) on the delaymgr
* tick and a blocking connect+monitor that runs on a slow-lane worker. When
* non-NULL, the lane worker calls this INSTEAD OF the real bounded
* endpoint_connect + iomgr_monitor, so a test can drive the reconnect path
* deterministically (e.g. park the worker to simulate an unreachable peer)
* without a real socket.
*
* Signature: int hook(cluster_t *c, int peer_idx)
* returns 0 to simulate a successful connect, <0 to simulate failure.
* The lane worker clears reconnect_inflight[peer_idx] after this returns
* regardless of outcome, so dedup never wedges (Req 2.5).
*/
extern int (*ha_reconnect_connect_hook)(cluster_t *c, int peer_idx);
/*
* ha_repl_push_config_for_standby_hook: test-only seam for the per-standby
* config push triggered by the heartbeat-reply decision loop (Task 3.3).