diff --git a/.kiro/specs/ha-entity-snapshots/tasks.md b/.kiro/specs/ha-entity-snapshots/tasks.md index 8c6edde3..1bbcad88 100644 --- a/.kiro/specs/ha-entity-snapshots/tasks.md +++ b/.kiro/specs/ha-entity-snapshots/tasks.md @@ -111,6 +111,31 @@ committed-prefix divergence, and existing comparison tooling confirms the rejoined member's job/queue state matches the rest of the cluster - _Requirements: 4.1, 4.2, 4.3_ + - _Blocked: 10-round chaotic soak run (unmodified, no forced empty-rejoin yet) shows + committed_job_loss=0 (the feature's core goal — CONFIRMED FIXED) but committed_divergence=11. + Root cause is upstream, not in this spec's boundary: `walcompare` (testsuite/walcompare.c) + has no concept of a per-member compaction/checkpoint floor — `wc_compare_committed` requires + EVERY member to physically hold EVERY WAL entry at or below one uniform `--committed N` floor + (the soak passes the same scalar to all 3 members, see `walcheck_count` in run-ha-soak.sh and + `cmd_walcompare` in test.c). A member that legitimately catches up via a full-checkpoint + snapshot will, BY DESIGN, have no raw WAL entries below its own checkpoint LSN (that range was + restored via entity content, not WAL replay) — walcompare has no per-member "compacted below + this LSN, don't expect it" signal, so it will always flag that range as MISSING/divergent + regardless of whether ha-entity-snapshots restored the entities correctly. This is exactly + `raft-ha-core-safety` task 3 ("Snapshot-boundary reconciliation" — "Make snapshot install + establish a verifiable (term, lsn) at the checkpoint so post-install AE intake reconciles + correctly and no committed entry below the checkpoint can diverge", still `[ ]` unimplemented) + and its task 5.10a residual, NOT a gap in this spec — design.md's Boundary Commitments + explicitly place "WAL-trim floor computation" out of this spec's boundary. Additionally, this + specific soak run never actually exercised a full-checkpoint catch-up at all (grepped all 3 + node logs for the `ha.c:2618`/`2621` "HA: snapshot loaded ..." / "... legacy position-only ..." + lines and `ha_handle_snapshot_chunk`: zero matches) — `start_node` in run-ha-soak.sh restarts a + process in place without wiping its data dir, so every rejoin in this run was ordinary + WAL-stream catch-up, never a snapshot install. Forcing the required empty-rejoin scenario is + still outstanding. Do not patch `walcompare`'s per-member floor logic from inside this spec — + route that fix to `raft-ha-core-safety` task 3; this task stays blocked until that contract + lands, then re-run with a forced empty-rejoin round to get a true signal on this spec's own + entity-restore correctness._ - [ ] 8. Validation: rolling-upgrade compatibility - [ ] 8.1 Verify a pre-feature checkpoint file is handled safely by the new binary in a live