wdgzyjx/myxce
1
0
Fork 0
forked from lei.chen/myxce
Code Pull requests Activity

fix(raft-drive-async): resolve snapshot chunk-drop (bug-1386) and chunk misordering (bug-1389)

Browse source 
ha_send_snapshot_to_standby ignored the network send's return value in its
chunk loop, silently truncating a transfer on a mid-stream error while still
logging "complete" (bug-1386). Fixing the silent drop exposed a second,
independent bug in the same pipeline: ha_handle_snapshot_chunk appended
received chunks in arrival order instead of at their wire-carried
chunk_offset, so out-of-order delivery across the vtask pool corrupted the
reassembled file even though every chunk's bytes were individually correct
(bug-1389). Fixed by checking every fallible call in the send loop and
switching the receiver to pwrite-at-offset with a count-based completion
tracker.

Adds snapverify/snapdiff xce_test commands (offline checkpoint-file
structural validator and byte-level comparator) used to diagnose bug-1389 and
kept as standing diagnostic tools for future snapshot-transfer triage.

g_drive_async_configpush_offload flipped back ON after both fixes, validated
via 6-round chaotic + 3/3 force-snapshot soak (raft-drive-async task 3.3).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SDWVNQLKxqTVHh42cH7rV3
This commit is contained in:
Rongsong Shen 2026-07-03 02:40:12 +00:00
parent 02e46f6ba1
commit 5cb36794d2
14 changed files with 1553 additions and 43 deletions
Download patch file Download diff file Expand all files Collapse all files

37
src/controller/ha.h
View file

@ -319,6 +319,43 @@ typedef struct {
* a stale value can never be mistaken
* for a live in-flight LSN. 0 at
* init (ha_init memset). */
/* ── New fields added by bug-1389 fix ──────────────────────────────── */
uint32_t snapshot_recv_count; /* Standby receive side: count of
* OPCode_snapshotTransfer chunks
* structurally accounted for (written
* to snap.part) in the CURRENT
* transfer attempt. The vtask pool can
* dispatch chunks of one transfer to
* different workers, so chunks are NOT
* guaranteed to arrive/process in
* chunk_index order (bug-1389: chunks
* observed out of order in a live
* soak) — ha_handle_snapshot_chunk
* writes each chunk at its own
* chunk_offset (pwrite, order-
* independent) and uses this counter,
* not "this call happened to carry
* is_last", to decide when every chunk
* has actually arrived. Reset to 0
* when chunk_index==0 starts a fresh
* transfer (Req 4.4). Guarded by
* ha->lock. 0 at init (ha_init
* memset). */
uint32_t snapshot_recv_total; /* Standby receive side: total chunk
* count for the current transfer,
* known only once the is_last chunk
* has been seen (= its chunk_index+1,
* chunks are 0-based and contiguous).
* 0 = not yet known (is_last not seen
* yet) OR no transfer in progress —
* ha_handle_snapshot_chunk finalizes
* (verify/rename/load) only once
* snapshot_recv_count reaches this
* value, then resets it to 0 so a
* late straggler chunk cannot trigger
* a second finalize. Guarded by
* ha->lock. 0 at init (ha_init
* memset). */
} ha_state_t;
/* ── Public API ────────────────────────────────────────────── */