checkpoint: bug-851 root cause + raft TLA+ conformance spec family

Investigation & fixes (failover committed-job loss):
- root-cause bug-851 on an owned 3-node cluster: chaotic failover causes
  silent committed-prefix divergence (WAL hdr.term always 0; two leaders
  per term; no (term,lsn) reconcile). Evidence in repro-851/ + INVESTIGATION.
- live-validated fixes 848/849/850 (recovery-on-promotion, advertise real
  applied_lsn, persist-layer contiguity guard); add waldump WAL inspector.

Specs (strict raft.tla conformance — library owns the log):
- TLA-gap-analysis.md: impl vs raft/raft.tla, gaps G1-G6.
- raft-log-replication, raft-commit-safety, raft-sim-conformance: extend
  src/lib/raft to own log/commit/verification via callbacks (specs generated,
  cross-spec review passed).
- ha-term-safety: controller-side adapter + config marker (R1-R7); design done.
- config-upgrade-snapshot: R8 descoped here (deps entity-bearing-snapshots, bug-857).
- roadmap + buglog (851/854/857) + .wolf knowledge updated.

.kiro/specs/raft-log-replication/brief.md

@@ -0,0 +1,41 @@
+# Brief: raft-log-replication
+
+## Problem
+`src/lib/raft/raft.{c,h}` is an **election-only** Raft library (terms, votes, leader election, PreVote) with a clean callback interface and a deterministic simulator (`raft_sim.c`). It has **no log, no AppendEntries, no commit**. The replicated log is hand-rolled in the controller (`persist.c` WAL, `repl.c` catch-up streaming, `ha.c`, `ha_convergence.c`) and **violates the Raft log invariants of `raft/raft.tla`**, which root-caused **bug-851** (silent committed-prefix divergence under chaotic failover). Authoritative gap analysis: `.kiro/specs/ha-term-safety/TLA-gap-analysis.md`.
+
+This spec closes the four **log-half** gaps by making the library own the log, strictly following `raft.tla`:
+- **G1** — per-entry term is dead (WAL `hdr.term` always 0; `p->term` never set from `ha->term`).
+- **G2** — replication matches LSN contiguity only; no `prevLogTerm` check (TLA+ `AppendEntries`/`HandleAppendEntriesRequest` `logOk`).
+- **G3** — no `(term,lsn)` conflict truncation (TLA+ conflict branch: delete the divergent entry + all after).
+- **G5** — a node adopts a higher term only on **vote** RPCs; the AppendEntries/heartbeat path never advances `currentTerm` (TLA+ `UpdateTerm` fires on **any** higher-term RPC). Confirmed live: a rejoined standby reported `term:2` under a term-3 primary → term reuse → two leaders per term.
+
+## Approach
+Extend the library to own the replicated log, **strictly mirroring the `raft.tla` actions**, keeping the core **pure / synchronous / deterministic** (no I/O, no threads) so it stays simulator-verifiable. All side effects go through new callbacks the controller adapter implements over the existing WAL/repl transport.
+
+### TLA+ actions implemented (raft.tla line refs)
+- `ClientRequest(i,v)` (246) → `raft_log_append(...)`: append `[term ↦ currentTerm, value]`.
+- `AppendEntries(i,j)` (204) → emit via `on_send_append_entries` with `prevLogIndex`, `prevLogTerm`, ≤1+ entries, `leaderCommit`. Per-peer `nextIndex`/`matchIndex` (TLA+ leaderVars).
+- `HandleAppendEntriesRequest(i,j,m)` (327) → `raft_on_append_entries_request(...)`: `logOk` = `prevLogIndex==0 ∨ log[prevLogIndex].term==prevLogTerm`; reject (stale term / `¬logOk`), candidate→follower on equal term, accept (already-have / conflict-truncate-one / append).
+- `HandleAppendEntriesResponse(i,j,m)` (393) → `raft_on_append_entries_reply(...)`: success → advance `nextIndex`/`matchIndex`; fail → decrement `nextIndex`.
+- `UpdateTerm(i,j,m)` (406) → applied on **every** incoming RPC including AppendEntries/heartbeat (closes G5): higher term ⇒ adopt term, clear vote, step down to follower.
+
+### Authoritative shared contract (downstream specs MUST reference this verbatim)
+Log index model: the library log index is 1-based (TLA+) and maps 1:1 to the controller WAL **LSN**. Per-entry `term` is `uint64_t`. New REQUIRED callbacks (pure; controller implements I/O):
+- `int  on_persist_entry(void *ctx, uint64_t index, uint64_t term, const void *entry, size_t len)` — durably append/overwrite at index.
+- `uint64_t on_get_entry_term(void *ctx, uint64_t index)` — term of local entry at index (0 if absent).
+- `int  on_send_append_entries(void *ctx, int peer_idx, uint64_t prev_log_index, uint64_t prev_log_term, uint64_t leader_commit, /* entries handle */ ...)` — transmit.
+- `int  on_truncate_from(void *ctx, uint64_t index)` — delete local entries at `index..end`.
+New library entry points: `raft_log_append`, `raft_on_append_entries_request`, `raft_on_append_entries_reply`. Existing election callbacks/behaviour unchanged.
+
+## Scope
+- **In**: library log model with per-entry term; `ClientRequest` append; AppendEntries send (per-peer next/match index); `HandleAppendEntriesRequest` accept/reject/conflict-truncate/append by `(term,index)`; `UpdateTerm` on every RPC; controller adapter that maps callbacks onto the existing WAL/repl transport (WAL `hdr.term` stamped from `ha->term`, carried in the repl envelope, read back on recovery); simulator coverage asserting **Log Matching**.
+- **Out**: `AdvanceCommitIndex` / commit-index safety + `(term,lsn)` vote freshness → `raft-commit-safety`. The full conformance harness → `raft-sim-conformance`. Config-change markers + upgrade snapshot → `ha-term-safety`. Async I/O, fan-out scheduling, the sync commit barrier, snapshot transport → controller callback adapters (not the pure core).
+
+## Preserve as policy (do NOT canonicalize away)
+PreVote, priority leadership, persist-before-ACK durability + synchronous commit barrier (consumed in `raft-commit-safety`), "controller never self-terminates" (library returns statuses; controller owns liveness/fencing).
+
+## Verification
+Extend `raft_sim.c`: drive AppendEntries across partitions, message drop/dup/reorder, and restarts; assert the **Log Matching** invariant (same `(index,term)` ⇒ identical prefix) and that a follower truncates a divergent suffix. Replace the controller's LSN-only reconcile with `(term,index)` matching delegated to the library.
+
+## Dependencies
+`raft-library` (election core, implemented `[x]`). Feeds `raft-commit-safety`, `raft-sim-conformance`, and `ha-term-safety` integration. Closes bug-851 G1/G2/G3/G5.