feat(raft-drive-async): snapshot SlowLane offload + WAL-trim cursor guard (task 3.2)

Part B (LIVE, default-ON): trim_wal_segments now bounds WAL-segment deletion
by repl_trim_lsn(&p->repl, active_seg_lsn) = min(active segment LSN, every
active standby acked_lsn, every valid disconnected-slot last_disconnected_lsn),
after repl_expire_disconnect_slots ages out stale slot guards. floor <=
active_seg_lsn structurally, so the guard can only preserve more segments,
never delete one a live standby still needs — closes the bug-795/828
trim-vs-cursor race for connected/recently-disconnected standbys (bug-1147
stopgap; empty/cursor=0 join still needs entity-bearing snapshots).

Part A (DORMANT, g_drive_async_snapshot_offload default OFF): ha_snapshot_send_
dispatch routes HA_REPL_SEND_SNAPSHOT inline (flag OFF — byte-for-byte legacy,
on io_worker) or to slowlane_submit+snapshot_lane_fn (flag ON). Send body
factored into ha_snapshot_do_send. Per-peer single-flight via
ha_state_t.snapshot_inflight[] CAS + slowlane key; in-flight flag cleared on
all four terminal paths, mirroring the reconnect offload (task 2.1).

Lands dormant like task 3.1: snapshot_lane_fn borrows a standby endpoint it
does not own, so flag-ON would race io_worker RDHUP teardown (cross-thread UAF,
bug-1197 family). Flag OFF keeps send+teardown serialized on the io_worker.
Task 3.4 is the single gate that flips both this and the peerctrl flag ON after
establishing endpoint-lifetime discipline; bug-1198 stays open by design until
then. The offload flag was initially authored default-ON (would ship the UAF)
and corrected to OFF in ha.c and the ha.h accessor doc (bug-1210).

Tests (controller 196/0, 0 new leaks): test_snapshot_offload_runs_on_lane,
test_snapshot_inline_when_flag_off, test_snapshot_offload_dedups_per_peer,
test_trim_guard_preserves_standby_segments,
test_trim_without_standbys_proceeds_normally. Adversarial cpp-review: GO.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S6fDCtXkkEgupu7rUah1y2
This commit is contained in:
Rongsong Shen 2026-06-26 15:43:24 +00:00
parent e17a6a613c
commit 6c0523b69b
8 changed files with 2046 additions and 850 deletions

View file

@ -1368,9 +1368,22 @@ static void scan_latest_committed_config_ref(persist_t* p)
}
/* Delete WAL segment files whose start LSN is strictly less than the
* currently-open segment's start LSN. We parse the active segment
* filename (seg_path) to get its start LSN and never delete that file,
* even if its start_lsn < checkpoint_lsn. */
* active segment's start LSN AND strictly less than the trim floor.
*
* Bug-1147 (stopgap, raft-drive-async task 3.2): the trim floor is the
* minimum of the active-segment LSN, every active standby's last acked LSN,
* and every recently-disconnected slot's last_disconnected_lsn (strategy-A
* retention discipline). This ensures we NEVER delete a WAL segment that a
* live standby still needs for catch-up (the bug-795/828 committed-job-loss
* family). It does NOT fix the empty/far-behind join case (cursor=0) —
* that requires entity-bearing snapshots (separate spec).
*
* Lock ordering (verified 2026-06-26): persist_checkpoint takes p->lock only
* to snapshot checkpoint_lsn, then RELEASES it before calling
* trim_wal_segments — so this function runs with NO p->lock held.
* repl_trim_lsn and repl_expire_disconnect_slots take only r->lock
* (p->repl.lock); neither ever acquires p->lock or ce->lock, so there is no
* p->lock↔repl.lock (or ce->lock↔repl.lock) inversion. */
static void trim_wal_segments(persist_t* p)
{
/* Extract the start LSN of the currently-open segment from seg_path.
@ -1387,6 +1400,18 @@ static void trim_wal_segments(persist_t* p)
if (sscanf(seg_basename, "seg-%" SCNx64 ".wal", &active_seg_lsn) != 1)
return;
/* Expire stale disconnected slots so their last_disconnected_lsn guards
* don't hold WAL segments indefinitely (slot guard expires at
* last_disconnect_time + snap_interval_s). */
repl_expire_disconnect_slots(&p->repl, time(NULL),
p->config.snap_interval_s);
/* Compute the trim floor: min(active_seg_lsn, all active standby
* acked_lsn, all valid disconnected-slot last_disconnected_lsn).
* Any segment with seg_start_lsn BELOW this floor is disposable. */
uint64_t floor = repl_trim_lsn(&p->repl, active_seg_lsn,
p->config.snap_interval_s);
DIR* d = opendir(p->wal_path);
if (!d) return;
@ -1395,8 +1420,11 @@ static void trim_wal_segments(persist_t* p)
if (strncmp(e->d_name, "seg-", 4) != 0) continue;
uint64_t seg_start_lsn = 0;
if (sscanf(e->d_name, "seg-%" SCNx64 ".wal", &seg_start_lsn) != 1) continue;
/* Never delete the currently-open segment; only trim older ones. */
if (seg_start_lsn < active_seg_lsn) {
/* Only delete segments below BOTH the active segment AND the
* standby-cursor floor (bug-1147 guard). floor <= active_seg_lsn
* by construction (repl_trim_lsn caps at checkpoint_lsn=active_seg_lsn),
* so this check subsumes the old < active_seg_lsn guard. */
if (seg_start_lsn < floor) {
char seg_path[4096];
snprintf(seg_path, sizeof(seg_path), "%s/%s", p->wal_path, e->d_name);
unlink(seg_path);