checkpoint: bug-851 root cause + raft TLA+ conformance spec family

Investigation & fixes (failover committed-job loss):
- root-cause bug-851 on an owned 3-node cluster: chaotic failover causes
  silent committed-prefix divergence (WAL hdr.term always 0; two leaders
  per term; no (term,lsn) reconcile). Evidence in repro-851/ + INVESTIGATION.
- live-validated fixes 848/849/850 (recovery-on-promotion, advertise real
  applied_lsn, persist-layer contiguity guard); add waldump WAL inspector.

Specs (strict raft.tla conformance — library owns the log):
- TLA-gap-analysis.md: impl vs raft/raft.tla, gaps G1-G6.
- raft-log-replication, raft-commit-safety, raft-sim-conformance: extend
  src/lib/raft to own log/commit/verification via callbacks (specs generated,
  cross-spec review passed).
- ha-term-safety: controller-side adapter + config marker (R1-R7); design done.
- config-upgrade-snapshot: R8 descoped here (deps entity-bearing-snapshots, bug-857).
- roadmap + buglog (851/854/857) + .wolf knowledge updated.
This commit is contained in:
Rongsong Shen 2026-06-21 14:02:15 +00:00
parent 7d70820aac
commit 20e59d0b42
85 changed files with 10525 additions and 1993 deletions

View file

@ -1356,6 +1356,20 @@ int cluster_init_phase2(cluster_t* c)
return rc;
}
/* bug-848: a (re)started member recovered its own WAL above — make it
* ADVERTISE that recovered position for catch-up (sr->applied_lsn), not the
* ha_init default of 0. Otherwise the primary streams the whole log and the
* receiver overwrites already-consistent records instead of fetching only
* the missing suffix (Raft incremental catch-up). */
ha_sync_applied_lsn_from_recovery(&c->ha, &c->persist);
INFO("HA: persist_recover done — recovered durable_floor=%" PRIu64
" current_lsn=%" PRIu64 "; applied_lsn=%" PRIu64
" advertised for catch-up",
persist_durable_floor(&c->persist),
(uint64_t)c->persist.current_lsn,
(uint64_t)atomic_load_explicit(&c->ha.applied_lsn,
memory_order_acquire));
return 0;
}