myxce/.wolf/bugs/bug-1870.md

id	status	status_updated	status_note	component	severity	timestamp	last_seen	occurrences	files	tags	related	fix_commits	legacy	gitea_issue	gitea_synced_status	gitea_synced_hash
bug-1870	fixed	2026-09-26	63fd3d2a: checkpoints no longer write the LSN-consuming WAL_CHECKPOINT marker; force-snapshot, config-upgrade-wal-trim, sequential soaks PASS.	controller	unknown	2026-09-26T09:55:09Z	2026-09-26T09:55:09Z	1	src/controller/persist.c (checkpoint marker append) src/controller/repl.c / config_repl.c (broadcast / catch-up)	ha replication checkpoint wal force-snapshot soak livelock		63fd3d2a	true	296	fixed	05688d859d6e

bug-1870: xce-test-ha --force-snapshot FAIL: 'could not submit 110 jobs to trigger a checkpoint' -- writes…

Description

xce-test-ha --force-snapshot FAIL: 'could not submit 110 jobs to trigger a checkpoint' -- writes stall right after the first max-wal-records checkpoint. The primary's WAL holds a CHECKPOINT entry (plen 0) at the checkpoint LSN that no standby ever receives (primary ctrl-1: 109 JOB_SUBMIT, 110 CHECKPOINT, 111 JOB_SUBMIT; standbys stop at 109). Every later AppendEntries carries prevIdx=, standbys reply ae_reject_logmismatch, the primary marks catch-up and re-sends the last common entry forever: 'commit: async continuation deadline expired without commit', 'HA quorum lost: connected=0'.

Environment

Not recorded (migrated from buglog.json).

Reproduction

Not recorded (migrated from buglog.json).

Root cause

Not yet root-caused. The checkpoint marker is appended to the local WAL (and consumes an LSN in the replicated log) without being broadcast to / streamed to standbys, so the log has a hole standbys can never fill. Reproduced on a565779d (baseline, 2026-09-26) and on the bug-1867 working tree -- independent of both. Repro: nix develop ./src --command bash .claude/skills/xce-test-ha/run-ha-soak.sh --force-snapshot

(max-wal-records=100, wal-segment-bytes=4096).

Fix

UPDATE (2026-09-27): follow-up cb1415d7: load_snapshot seeded current_lsn = checkpoint_lsn (off by one, hidden by the removed marker) -- broke --config-upgrade soak; fixed

Verification

Not recorded (migrated from buglog.json).

References

• Gitea: #296