42 lines
2.3 KiB
Markdown
42 lines
2.3 KiB
Markdown
---
|
|
id: "bug-1870"
|
|
status: "fixed"
|
|
status_updated: "2026-09-26"
|
|
status_note: "63fd3d2a: checkpoints no longer write the LSN-consuming WAL_CHECKPOINT marker; force-snapshot, config-upgrade-wal-trim, sequential soaks PASS."
|
|
component: "controller"
|
|
severity: "unknown"
|
|
timestamp: "2026-09-26T09:55:09Z"
|
|
last_seen: "2026-09-26T09:55:09Z"
|
|
occurrences: 1
|
|
files: ["src/controller/persist.c (checkpoint marker append)", "src/controller/repl.c / config_repl.c (broadcast / catch-up)"]
|
|
tags: ["ha", "replication", "checkpoint", "wal", "force-snapshot", "soak", "livelock"]
|
|
related: []
|
|
fix_commits: ["63fd3d2a"]
|
|
legacy: true
|
|
gitea_issue: 296
|
|
gitea_synced_status: "fixed"
|
|
gitea_synced_hash: "05688d859d6e"
|
|
---
|
|
|
|
# bug-1870: xce-test-ha --force-snapshot FAIL: 'could not submit 110 jobs to trigger a checkpoint' -- writes…
|
|
|
|
## Description
|
|
xce-test-ha --force-snapshot FAIL: 'could not submit 110 jobs to trigger a checkpoint' -- writes stall right after the first max-wal-records checkpoint. The primary's WAL holds a CHECKPOINT entry (plen 0) at the checkpoint LSN that no standby ever receives (primary ctrl-1: 109 JOB_SUBMIT, 110 CHECKPOINT, 111 JOB_SUBMIT; standbys stop at 109). Every later AppendEntries carries prevIdx=<checkpoint LSN>, standbys reply ae_reject_logmismatch, the primary marks catch-up and re-sends the last common entry forever: 'commit: async continuation deadline expired without commit', 'HA quorum lost: connected=0'.
|
|
|
|
## Environment
|
|
_Not recorded (migrated from buglog.json)._
|
|
|
|
## Reproduction
|
|
_Not recorded (migrated from buglog.json)._
|
|
|
|
## Root cause
|
|
Not yet root-caused. The checkpoint marker is appended to the local WAL (and consumes an LSN in the replicated log) without being broadcast to / streamed to standbys, so the log has a hole standbys can never fill. Reproduced on a565779d (baseline, 2026-09-26) and on the bug-1867 working tree -- independent of both. Repro: nix develop ./src --command bash .claude/skills/xce-test-ha/run-ha-soak.sh --force-snapshot <dir> (max-wal-records=100, wal-segment-bytes=4096).
|
|
|
|
## Fix
|
|
UPDATE (2026-09-27): follow-up cb1415d7: load_snapshot seeded current_lsn = checkpoint_lsn (off by one, hidden by the removed marker) -- broke --config-upgrade soak; fixed
|
|
|
|
## Verification
|
|
_Not recorded (migrated from buglog.json)._
|
|
|
|
## References
|
|
- Gitea: #296
|