myxce

wdgzyjx/myxce

Fork 0

forked from lei.chen/myxce

Commit graph

Author	SHA1	Message	Date
Rongsong Shen	6c0523b69b	feat(raft-drive-async): snapshot SlowLane offload + WAL-trim cursor guard (task 3.2) Part B (LIVE, default-ON): trim_wal_segments now bounds WAL-segment deletion by repl_trim_lsn(&p->repl, active_seg_lsn) = min(active segment LSN, every active standby acked_lsn, every valid disconnected-slot last_disconnected_lsn), after repl_expire_disconnect_slots ages out stale slot guards. floor <= active_seg_lsn structurally, so the guard can only preserve more segments, never delete one a live standby still needs — closes the bug-795/828 trim-vs-cursor race for connected/recently-disconnected standbys (bug-1147 stopgap; empty/cursor=0 join still needs entity-bearing snapshots). Part A (DORMANT, g_drive_async_snapshot_offload default OFF): ha_snapshot_send_ dispatch routes HA_REPL_SEND_SNAPSHOT inline (flag OFF — byte-for-byte legacy, on io_worker) or to slowlane_submit+snapshot_lane_fn (flag ON). Send body factored into ha_snapshot_do_send. Per-peer single-flight via ha_state_t.snapshot_inflight[] CAS + slowlane key; in-flight flag cleared on all four terminal paths, mirroring the reconnect offload (task 2.1). Lands dormant like task 3.1: snapshot_lane_fn borrows a standby endpoint it does not own, so flag-ON would race io_worker RDHUP teardown (cross-thread UAF, bug-1197 family). Flag OFF keeps send+teardown serialized on the io_worker. Task 3.4 is the single gate that flips both this and the peerctrl flag ON after establishing endpoint-lifetime discipline; bug-1198 stays open by design until then. The offload flag was initially authored default-ON (would ship the UAF) and corrected to OFF in ha.c and the ha.h accessor doc (bug-1210). Tests (controller 196/0, 0 new leaks): test_snapshot_offload_runs_on_lane, test_snapshot_inline_when_flag_off, test_snapshot_offload_dedups_per_peer, test_trim_guard_preserves_standby_segments, test_trim_without_standbys_proceeds_normally. Adversarial cpp-review: GO. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S6fDCtXkkEgupu7rUah1y2	2026-06-26 15:43:24 +00:00
Rongsong Shen	a7094fc47f	test: P0-G end-to-end smoke tests — WAL, checkpoint, hooks, GC, Cap'n Proto (P0-15a-e) Add 5 E2E integration test files that cross sub-phase boundaries: - test_e2e_wal.c (P0-15a): WAL write+read-back (entry count), torn-write recovery via ftruncate + persist_recover(), chained-CRC corruption stops replay at last valid entry. - test_e2e_checkpoint.c (P0-15b): 20-entry WAL + checkpoint + 10 post- checkpoint entries + recover asserts current_lsn >= 30 and checkpoint_lsn > 0; WAL trim after two checkpoints; snap_keep_n rotation with 4 rounds. - test_e2e_hooks.c (P0-15c): persist_job_hook_fun(HOOK_DETAIL_JOB_NEW) writes WAL_JOB_SUBMIT entry; persist_worker_hook_fun(HOOK_DETAIL_WORKER_JOIN) writes WAL_WORKER_CTRL; 3-call sequence preserves monotone LSN order and verifies header LSN fields on disk. - test_e2e_gc.c (P0-15d): running job retained; expired finished job not retained; JOB_STAT_TRANSIENT times out; expired unavailable worker not retained; checkpoint-after-WAL integration smoke. - test_e2e_capnp.c (P0-15e): persist_config_t → capnp_persist_cfg_t 8-field round-trip; capnp_snap_hdr_t field values and struct size (48 bytes); capnp_repl_envelope_t fields including WAL_CRC_NONE sentinel. Total: 89 tests, 0 failures (was 72 before this commit).	2026-05-07 13:25:17 +00:00

Author

SHA1

Message

Date

Rongsong Shen

6c0523b69b

feat(raft-drive-async): snapshot SlowLane offload + WAL-trim cursor guard (task 3.2)

Part B (LIVE, default-ON): trim_wal_segments now bounds WAL-segment deletion
by repl_trim_lsn(&p->repl, active_seg_lsn) = min(active segment LSN, every
active standby acked_lsn, every valid disconnected-slot last_disconnected_lsn),
after repl_expire_disconnect_slots ages out stale slot guards. floor <=
active_seg_lsn structurally, so the guard can only preserve more segments,
never delete one a live standby still needs — closes the bug-795/828
trim-vs-cursor race for connected/recently-disconnected standbys (bug-1147
stopgap; empty/cursor=0 join still needs entity-bearing snapshots).

Part A (DORMANT, g_drive_async_snapshot_offload default OFF): ha_snapshot_send_
dispatch routes HA_REPL_SEND_SNAPSHOT inline (flag OFF — byte-for-byte legacy,
on io_worker) or to slowlane_submit+snapshot_lane_fn (flag ON). Send body
factored into ha_snapshot_do_send. Per-peer single-flight via
ha_state_t.snapshot_inflight[] CAS + slowlane key; in-flight flag cleared on
all four terminal paths, mirroring the reconnect offload (task 2.1).

Lands dormant like task 3.1: snapshot_lane_fn borrows a standby endpoint it
does not own, so flag-ON would race io_worker RDHUP teardown (cross-thread UAF,
bug-1197 family). Flag OFF keeps send+teardown serialized on the io_worker.
Task 3.4 is the single gate that flips both this and the peerctrl flag ON after
establishing endpoint-lifetime discipline; bug-1198 stays open by design until
then. The offload flag was initially authored default-ON (would ship the UAF)
and corrected to OFF in ha.c and the ha.h accessor doc (bug-1210).

Tests (controller 196/0, 0 new leaks): test_snapshot_offload_runs_on_lane,
test_snapshot_inline_when_flag_off, test_snapshot_offload_dedups_per_peer,
test_trim_guard_preserves_standby_segments,
test_trim_without_standbys_proceeds_normally. Adversarial cpp-review: GO.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S6fDCtXkkEgupu7rUah1y2

2026-06-26 15:43:24 +00:00

Rongsong Shen

a7094fc47f

test: P0-G end-to-end smoke tests — WAL, checkpoint, hooks, GC, Cap'n Proto (P0-15a-e)

Add 5 E2E integration test files that cross sub-phase boundaries:

- test_e2e_wal.c (P0-15a): WAL write+read-back (entry count), torn-write
  recovery via ftruncate + persist_recover(), chained-CRC corruption stops
  replay at last valid entry.

- test_e2e_checkpoint.c (P0-15b): 20-entry WAL + checkpoint + 10 post-
  checkpoint entries + recover asserts current_lsn >= 30 and checkpoint_lsn
  > 0; WAL trim after two checkpoints; snap_keep_n rotation with 4 rounds.

- test_e2e_hooks.c (P0-15c): persist_job_hook_fun(HOOK_DETAIL_JOB_NEW)
  writes WAL_JOB_SUBMIT entry; persist_worker_hook_fun(HOOK_DETAIL_WORKER_JOIN)
  writes WAL_WORKER_CTRL; 3-call sequence preserves monotone LSN order and
  verifies header LSN fields on disk.

- test_e2e_gc.c (P0-15d): running job retained; expired finished job not
  retained; JOB_STAT_TRANSIENT times out; expired unavailable worker not
  retained; checkpoint-after-WAL integration smoke.

- test_e2e_capnp.c (P0-15e): persist_config_t → capnp_persist_cfg_t 8-field
  round-trip; capnp_snap_hdr_t field values and struct size (48 bytes);
  capnp_repl_envelope_t fields including WAL_CRC_NONE sentinel.

Total: 89 tests, 0 failures (was 72 before this commit).

2026-05-07 13:25:17 +00:00

2 commits