lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(controller): refuse replicated WAL writes before this boot's recovery (bug-1920)

Browse source 
A restarted standby accepts the primary's connection during its startup
delay, before persist_recover runs. Catch-up entries written into the
un-recovered persist_t opened seg-...0001 with O_TRUNC and rewrote it,
and after recovery the stale fd kept filing new LSNs under that segment
name. The primary's filename-ordered streamer then skipped them, so a
standby needing that range never caught up and commits stalled.

persist_wal_write_replicated now returns -EBUSY until recovery completes
(the bug-1416 recovery_complete gate). ha_apply_wal_catchup_entry and
ha_handle_snapshot_chunk drop pre-recovery traffic without ACK or NACK.
Catch-up streams from min(NACK-derived join position, standby report),
and the streamer warns when a non-empty range yields no entries.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-04 12:02:37 +00:00
parent f01bb7de34
commit 762c6f1883
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
6 changed files with 242 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

42
src/controller/testsuite/test_convergence_verification.c
View file

@ -479,6 +479,47 @@ static void test_walack_at_parity_completes_catchup_and_clears_in_catch_up(void)
printf("test_walack_at_parity_completes_catchup_and_clears_in_catch_up: checked\n");
}
/* bug-1920: the heartbeat-driven catch-up must never stream from ABOVE the
* standby's own reported position. A log-mismatch NACK walks the primary's
* next_index back one per reject, so the catch-up join position can sit far
* above where the standby's log really ends (soak: join 340, standby at 323).
* Streaming [341..] from there is rejected again at best (one step per
* heartbeat) and silent at worst; the standby's own report is an upper bound
* on what it holds, so the start must be min(join position, report). */
static void test_dispatch_streams_from_min_of_join_and_report(void)
{
cv_setup("b1920", /*n_peers=*/3, /*n_standbys=*/1,
/*appended_lsn=*/342, /*n_vcpus=*/8);
atomic_store(&g_cv.persist.committed_lsn, 341);
repl_ep_t ep = &g_cv_eps[0];
repl_start_catchup(&g_cv.persist.repl, ep, /*join_applied_lsn=*/340);
g_cv.ha.standby_reported_lsn[0] = 323;
g_cv.ha.standby_reported_tag[0] = 0;
ha_dispatch_repl_action_for_standby(&g_cv.ha, &g_cv, (endpoint_t *)ep,
/*peer_idx=*/0, /*committed_tag=*/0,
/*committed_lsn=*/341,
/*oldest_wal_lsn=*/1);
TEST_ASSERT_EQUAL_INT(1, atomic_load(&g_stream_calls));
TEST_ASSERT_EQUAL_UINT64_MESSAGE(
323, atomic_load(&g_stream_last_standby_lsn),
"bug-1920: stream from the standby's reported position (323), not the "
"higher NACK-derived join position (340)");
/* The join position still wins when it is the LOWER one (bug-1479: a
* conflicted suffix the standby's report cannot see). */
atomic_store(&g_stream_calls, 0);
repl_start_catchup(&g_cv.persist.repl, ep, /*join_applied_lsn=*/300);
ha_dispatch_repl_action_for_standby(&g_cv.ha, &g_cv, (endpoint_t *)ep,
0, 0, 341, 1);
TEST_ASSERT_EQUAL_UINT64(300, atomic_load(&g_stream_last_standby_lsn));
cv_teardown();
printf("test_dispatch_streams_from_min_of_join_and_report: checked\n");
}
/* 5.3 decision pin: the heartbeat decision function ha_evaluate_repl_action is
* the fallback initiator of catch-up and the SOLE owner of the snapshot
* decision; the ACK-driven continuation must not change its verdicts. This pins
@ -1457,6 +1498,7 @@ void test_convergence_verification(void)
RUN_TEST(test_walack_single_flight_no_duplicate_continuation);
RUN_TEST(test_walack_reschedules_after_continuation_completes);
RUN_TEST(test_walack_at_parity_completes_catchup_and_clears_in_catch_up);
RUN_TEST(test_dispatch_streams_from_min_of_join_and_report);
RUN_TEST(test_heartbeat_decision_path_unchanged);
printf("\n>> Quorum transition logging tests (RED, task 5.1)\n\n");