From 3fab29114fdaf6f0b877ec7a7c6cc0715c96b0c4 Mon Sep 17 00:00:00 2001 From: Rongsong Shen Date: Wed, 1 Jul 2026 09:04:00 +0000 Subject: [PATCH] =?UTF-8?q?docs(ha-entity-snapshots):=20block=20task=207.1?= =?UTF-8?q?=20=E2=80=94=20committed=5Fdivergence=20is=20upstream=20(raft-h?= =?UTF-8?q?a-core-safety=20task=203)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 10-round chaotic soak: committed_job_loss=0 (core goal confirmed fixed), committed_divergence=11. Root-caused to walcompare having no per-member compaction-floor concept (uniform --committed N for all members), which is exactly raft-ha-core-safety task 3's unimplemented snapshot-boundary reconciliation, not a gap in this spec's boundary. Also: this run never exercised an actual snapshot install (start_node doesn't wipe data dirs) — forcing a real empty-rejoin round is still outstanding for a true signal. --- .kiro/specs/ha-entity-snapshots/tasks.md | 25 ++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/.kiro/specs/ha-entity-snapshots/tasks.md b/.kiro/specs/ha-entity-snapshots/tasks.md index 8c6edde3..1bbcad88 100644 --- a/.kiro/specs/ha-entity-snapshots/tasks.md +++ b/.kiro/specs/ha-entity-snapshots/tasks.md @@ -111,6 +111,31 @@ committed-prefix divergence, and existing comparison tooling confirms the rejoined member's job/queue state matches the rest of the cluster - _Requirements: 4.1, 4.2, 4.3_ + - _Blocked: 10-round chaotic soak run (unmodified, no forced empty-rejoin yet) shows + committed_job_loss=0 (the feature's core goal — CONFIRMED FIXED) but committed_divergence=11. + Root cause is upstream, not in this spec's boundary: `walcompare` (testsuite/walcompare.c) + has no concept of a per-member compaction/checkpoint floor — `wc_compare_committed` requires + EVERY member to physically hold EVERY WAL entry at or below one uniform `--committed N` floor + (the soak passes the same scalar to all 3 members, see `walcheck_count` in run-ha-soak.sh and + `cmd_walcompare` in test.c). A member that legitimately catches up via a full-checkpoint + snapshot will, BY DESIGN, have no raw WAL entries below its own checkpoint LSN (that range was + restored via entity content, not WAL replay) — walcompare has no per-member "compacted below + this LSN, don't expect it" signal, so it will always flag that range as MISSING/divergent + regardless of whether ha-entity-snapshots restored the entities correctly. This is exactly + `raft-ha-core-safety` task 3 ("Snapshot-boundary reconciliation" — "Make snapshot install + establish a verifiable (term, lsn) at the checkpoint so post-install AE intake reconciles + correctly and no committed entry below the checkpoint can diverge", still `[ ]` unimplemented) + and its task 5.10a residual, NOT a gap in this spec — design.md's Boundary Commitments + explicitly place "WAL-trim floor computation" out of this spec's boundary. Additionally, this + specific soak run never actually exercised a full-checkpoint catch-up at all (grepped all 3 + node logs for the `ha.c:2618`/`2621` "HA: snapshot loaded ..." / "... legacy position-only ..." + lines and `ha_handle_snapshot_chunk`: zero matches) — `start_node` in run-ha-soak.sh restarts a + process in place without wiping its data dir, so every rejoin in this run was ordinary + WAL-stream catch-up, never a snapshot install. Forcing the required empty-rejoin scenario is + still outstanding. Do not patch `walcompare`'s per-member floor logic from inside this spec — + route that fix to `raft-ha-core-safety` task 3; this task stays blocked until that contract + lands, then re-run with a forced empty-rejoin round to get a true signal on this spec's own + entity-restore correctness._ - [ ] 8. Validation: rolling-upgrade compatibility - [ ] 8.1 Verify a pre-feature checkpoint file is handled safely by the new binary in a live