lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(ha-repl-hardening): batch WAL streaming from decision loop (task 3.4)

Browse source 
ha_stream_wal_to_standby() reads WAL from committed_lsn range, accumulates
entries to 256/64KB per batch, enqueues all via xce_request() (non-blocking).
Partial final batch sent immediately. Wired into STREAM_WAL case.
3 tests: 500-entries/2-batches, partial batch, empty range.
Fixed stale task-number comments per review.
This commit is contained in:
Rongsong Shen 2026-06-04 17:57:55 +00:00
parent d1c7872d73
commit 6b69a47c50
7 changed files with 469 additions and 20 deletions
Download patch file Download diff file Expand all files Collapse all files

28
src/controller/config_repl.h
View file

@ -127,6 +127,34 @@ int push_config_to_standbys(cluster_t *c, const uint8_t *bytes, size_t blen,
*/
int handle_upgrade_config(cluster_t *c, endpoint_t *ep, xce_request_t *req);
/*
* ha_stream_wal_to_standby: batch-stream WAL entries [standby_lsn+1, committed_lsn]
* to a standby endpoint via xce_request() (non-blocking).
*
* Accumulates WAL entries up to WAL_STREAM_BATCH_ENTRIES (256) entries or
* WAL_STREAM_BATCH_BYTES (65536) bytes per batch, then enqueues each batch
* as a single OPCode_walCatchupEntry message with n_entries set.
* All batches for the full range are enqueued in a single invocation — never
* blocks the vtaskmgr thread.
*
* A partial final batch (fewer entries than the batch limit) is sent immediately
* without waiting for the batch to fill (Req 5.4).
*
* Parameters:
* ep — outbound endpoint to the standby
* wal_path — directory containing WAL segment files (seg-*.wal)
* standby_lsn — standby's last applied LSN (stream starts at standby_lsn+1)
* committed_lsn— primary's committed LSN (stream ends at committed_lsn inclusive)
*
* Returns 0 on success; negative on internal error (e.g., NULL ep).
* Requirements: 5.1, 5.2, 5.4, 5.5 (ha-repl-hardening spec)
* Boundary: WAL Streaming (config_repl.c)
*/
#define WAL_STREAM_BATCH_ENTRIES 256
#define WAL_STREAM_BATCH_BYTES 65536
int ha_stream_wal_to_standby(endpoint_t *ep, const char *wal_path,
uint64_t standby_lsn, uint64_t committed_lsn);
/*
* handle_ctrlhello: primary-side handler for OP_hello from a joining standby.
*