lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(ha-repl-hardening): batch WAL streaming from decision loop (task 3.4)

Browse source 
ha_stream_wal_to_standby() reads WAL from committed_lsn range, accumulates
entries to 256/64KB per batch, enqueues all via xce_request() (non-blocking).
Partial final batch sent immediately. Wired into STREAM_WAL case.
3 tests: 500-entries/2-batches, partial batch, empty range.
Fixed stale task-number comments per review.
This commit is contained in:
Rongsong Shen 2026-06-04 17:57:55 +00:00
parent d1c7872d73
commit 6b69a47c50
7 changed files with 469 additions and 20 deletions
Download patch file Download diff file Expand all files Collapse all files

40
src/controller/ha.h
View file

@ -535,25 +535,31 @@ void ha_repl_ack_received(ha_state_t *ha, cluster_t *c);
*
* Reads ha->standby_reported_tag[peer_idx] and ha->standby_reported_lsn[peer_idx]
* (stored by ha_process_standby_state on the previous call), calls
* ha_evaluate_repl_action, and if the result is HA_REPL_PUSH_CONFIG:
* - In production builds: calls push_config_to_standbys(c, …).
* - In XCE_TESTING builds: calls ha_repl_push_config_for_standby_hook(c, peer_idx)
* if the hook is non-NULL.
* ha_evaluate_repl_action, and acts on the result:
* - HA_REPL_PUSH_CONFIG: In production builds, calls push_config_to_standbys(c, …).
* In XCE_TESTING builds, calls ha_repl_push_config_for_standby_hook(c, peer_idx).
* - HA_REPL_STREAM_WAL: calls ha_stream_wal_to_standby(ep, wal_path, standby_lsn,
* committed_lsn) to batch-stream WAL entries via xce_request() (non-blocking).
* In XCE_TESTING builds, calls ha_repl_stream_wal_hook if non-NULL.
* - HA_REPL_SEND_SNAPSHOT: stub (Task 3.5).
* - HA_REPL_CONVERGED: no-op.
*
* One action per heartbeat cycle per standby (design §Unified Heartbeat-Driven
* Protocol). Other actions (SEND_SNAPSHOT, STREAM_WAL) are stubs returning 0
* pending Tasks 3.4–3.5; CONVERGED is a no-op.
* Protocol). CONVERGED is a no-op.
*
* ep — outbound endpoint to the standby (used for STREAM_WAL; may be
* NULL in tests that only exercise PUSH_CONFIG or CONVERGED paths)
* peer_idx — 0-based index into ha->standby_reported_* arrays
* committed_tag — primary's committed config_tag (from c->myconf or test fixture)
* committed_lsn — primary's committed WAL LSN (from c->persist or test fixture)
* oldest_wal_lsn— oldest retained WAL LSN (0 = all WAL retained, skip snapshot)
*
* Returns 0 on success; negative on internal error.
* Requirements: 1.4 (ha-repl-hardening spec)
* Requirements: 1.4, 5.1–5.5 (ha-repl-hardening spec)
*/
int ha_dispatch_repl_action_for_standby(ha_state_t *ha,
cluster_t *c,
endpoint_t *ep,
int peer_idx,
uint64_t committed_tag,
uint64_t committed_lsn,
@ -610,6 +616,26 @@ extern int (*ha_push_config_hook)(void *c);
*/
extern int (*ha_repl_push_config_for_standby_hook)(void *c, int peer_idx);
/*
* ha_repl_stream_wal_hook: test-only seam for ha_stream_wal_to_standby.
*
* When non-NULL, ha_dispatch_repl_action_for_standby() calls this INSTEAD OF
* ha_stream_wal_to_standby() when the decision is HA_REPL_STREAM_WAL, so unit
* tests can count batch-send invocations without requiring a real endpoint or
* WAL directory.
*
* Signature: int hook(void *ep, const char *wal_path,
* uint64_t standby_lsn, uint64_t committed_lsn)
* ep — endpoint_t* (may be NULL in test mode)
* wal_path — WAL directory path
* standby_lsn — standby's last applied LSN
* committed_lsn— primary's committed LSN
* Requirements: 5.1–5.5 (ha-repl-hardening)
*/
extern int (*ha_repl_stream_wal_hook)(void *ep, const char *wal_path,
uint64_t standby_lsn,
uint64_t committed_lsn);
/*
* ha_tcp_connect_hook: test-only seam for the raw TCP connect inside
* ha_find_and_connect_to_primary. When non-NULL, the function calls this