myxce

lei.chen/myxce

Fork 1

Commit graph

Author	SHA1	Message	Date
Rongsong Shen	20e59d0b42	checkpoint: bug-851 root cause + raft TLA+ conformance spec family Investigation & fixes (failover committed-job loss): - root-cause bug-851 on an owned 3-node cluster: chaotic failover causes silent committed-prefix divergence (WAL hdr.term always 0; two leaders per term; no (term,lsn) reconcile). Evidence in repro-851/ + INVESTIGATION. - live-validated fixes 848/849/850 (recovery-on-promotion, advertise real applied_lsn, persist-layer contiguity guard); add waldump WAL inspector. Specs (strict raft.tla conformance — library owns the log): - TLA-gap-analysis.md: impl vs raft/raft.tla, gaps G1-G6. - raft-log-replication, raft-commit-safety, raft-sim-conformance: extend src/lib/raft to own log/commit/verification via callbacks (specs generated, cross-spec review passed). - ha-term-safety: controller-side adapter + config marker (R1-R7); design done. - config-upgrade-snapshot: R8 descoped here (deps entity-bearing-snapshots, bug-857). - roadmap + buglog (851/854/857) + .wolf knowledge updated.	2026-06-21 14:02:15 +00:00
Rongsong Shen	7d70820aac	docs(ha-commit-durability): root-cause findings for failover committed-job loss (bug-828) WAL-byte decode proves REAL data loss (not commit-index): committed prefix entries physically absent from live nodes (ctrl-2 WAL starts LSN 16, ctrl-3 LSN 13), present only on the down node. Mechanism: empty-rejoin catch-up streams [watermark..committed] not [1..], no snapshot to backfill the prefix -> permanent gap. Predates libxce-failover. Full findings + open forks + xce_test waldump tooling plan in INVESTIGATION-failover-job-loss.md. Not yet fixed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-20 13:51:40 +00:00

Author

SHA1

Message

Date

Rongsong Shen

20e59d0b42

checkpoint: bug-851 root cause + raft TLA+ conformance spec family

Investigation & fixes (failover committed-job loss):
- root-cause bug-851 on an owned 3-node cluster: chaotic failover causes
  silent committed-prefix divergence (WAL hdr.term always 0; two leaders
  per term; no (term,lsn) reconcile). Evidence in repro-851/ + INVESTIGATION.
- live-validated fixes 848/849/850 (recovery-on-promotion, advertise real
  applied_lsn, persist-layer contiguity guard); add waldump WAL inspector.

Specs (strict raft.tla conformance — library owns the log):
- TLA-gap-analysis.md: impl vs raft/raft.tla, gaps G1-G6.
- raft-log-replication, raft-commit-safety, raft-sim-conformance: extend
  src/lib/raft to own log/commit/verification via callbacks (specs generated,
  cross-spec review passed).
- ha-term-safety: controller-side adapter + config marker (R1-R7); design done.
- config-upgrade-snapshot: R8 descoped here (deps entity-bearing-snapshots, bug-857).
- roadmap + buglog (851/854/857) + .wolf knowledge updated.

2026-06-21 14:02:15 +00:00

Rongsong Shen

7d70820aac

docs(ha-commit-durability): root-cause findings for failover committed-job loss (bug-828)

WAL-byte decode proves REAL data loss (not commit-index): committed prefix
entries physically absent from live nodes (ctrl-2 WAL starts LSN 16, ctrl-3 LSN
13), present only on the down node. Mechanism: empty-rejoin catch-up streams
[watermark..committed] not [1..], no snapshot to backfill the prefix -> permanent
gap. Predates libxce-failover. Full findings + open forks + xce_test waldump
tooling plan in INVESTIGATION-failover-job-loss.md. Not yet fixed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-20 13:51:40 +00:00

2 commits