fix(controller): checkpoints no longer punch a hole in the replicated log (bug-1870)

persist_checkpoint() and persist_checkpoint_marker_triggered() wrote a
local-only WAL_CHECKPOINT marker that consumed an LSN of the replicated log
(current_lsn++) but was never replicated. A checkpoint taken by one member
only -- the primary's periodic max-wal-records checkpoint, or the one before
a snapshot send -- left a hole no standby had: the leader's next entry
carried prev_log_index = the marker's LSN, logOk failed on every standby
forever, commits timed out and writes stalled ("could not submit 110 jobs"
in the --force-snapshot soak; reproduced on the pre-bug-1867 baseline). Even
the config-marker checkpoint, taken on every member, stamped the marker with
each member's own current_lsn at apply time, which need not agree.

Nothing reads the marker except to skip it, so checkpoints simply stop
writing it. The recovery, streamer and bug-1475 log_len skip paths stay, for
WALs written by older binaries; persist_test_write_legacy_checkpoint_marker()
(XCE_TESTING only) reproduces such a WAL for the tests that cover them.

- new test_bug1870_leader_only_checkpoint_leaves_no_log_hole (RED before):
  a leader-only checkpoint takes no LSN and a standby that never
  checkpointed catches up past it via the real streamer/receive path.
- test_checkpoint_writes_marker_to_wal inverted to ..._writes_no_marker_...
- bug-1475 x2 and the apply-correctness trailing-marker test now build
  their legacy marker explicitly.

Controller suite 1234/0. HA soaks: --force-snapshot PASS (113 jobs, real
snapshot install -- also live-verifies bug-1867), --config-upgrade-wal-trim
PASS, --sequential 6 rounds PASS (SIMVERIFY 7 files, 0 mismatches).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-09-26 15:16:46 +00:00
parent 725301dd92
commit 63fd3d2afd
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
6 changed files with 139 additions and 20 deletions

View file

@ -1092,6 +1092,12 @@ extern void (*persist_checkpoint_test_hook)(persist_t*);
/* Checkpoint */
int persist_checkpoint(persist_t* p);
#ifdef XCE_TESTING
/* bug-1870: test-only -- write a pre-fix WAL_CHECKPOINT marker (consumes an
* LSN) to reproduce a WAL written by an older binary. */
int persist_test_write_legacy_checkpoint_marker(persist_t* p);
#endif
/*
* persist_set_checkpoint_offload_for_test / persist_checkpoint_offload_enabled:
* Feature-flag accessors (xce-audit-invariants controller pass, dimension 1