myxce/.kiro/specs/raft-log-replication/design.md

# Design Document

## Overview

**Purpose**: This feature extends the election-only Raft library (`src/lib/raft/raft.{c,h}`) into a strict implementation of the **log half** of `raft/raft.tla` (Ongaro canonical), giving the library ownership of the replicated log. It closes the four log-half gaps of bug-851 (G1/G2/G3/G5): per-entry term, `prevLogTerm` log-matching, `(term, index)` conflict truncation, and term adoption on the AppendEntries/heartbeat path.

**Users**: The XCE controller (as a Raft node), the deterministic simulator (`raft_sim.c`), and three downstream specs that consume this seam verbatim — `raft-commit-safety`, `raft-sim-conformance`, and `ha-term-safety`.

**Impact**: Today the replicated log is hand-rolled across the controller (`persist.c`/`repl.c`/`ha.c`/`ha_convergence.c`) and matches by LSN contiguity only. After this feature, the library owns log growth, AppendEntries emission/handling, per-peer progress, and `(term, index)` truncation as pure, synchronous, deterministic state transitions; the controller becomes a thin adapter that performs I/O behind four new callbacks. The on-disk WAL format is unchanged (the term is a value fix, not a layout change).

### Goals
- Add a library-owned log model where every entry carries a `uint64_t` term, index 1-based mapping 1:1 to the WAL LSN (G1).
- Implement `ClientRequest`, `AppendEntries` emission, `HandleAppendEntriesRequest`, and `HandleAppendEntriesResponse` exactly as `raft.tla` defines them (G2/G3).
- Apply `UpdateTerm` on **every** RPC including AppendEntries/heartbeat (G5).
- Define the four new callbacks **verbatim** per the brief's Authoritative shared contract.
- Extend `raft_sim.c` to assert the **Log Matching** invariant and divergent-suffix truncation under partitions, drop/dup/reorder, and restart.

### Non-Goals
- **No commit-index advance** (`AdvanceCommitIndex`, G4) and **no Figure-8 current-term commit rule** — owned by `raft-commit-safety`.
- **No `(lastLogTerm, lastLogIndex)` vote freshness** (G6) — owned by `raft-commit-safety`.
- **No config-change markers / upgrade snapshot** — owned by `ha-term-safety`.
- **No async I/O, fan-out scheduling, commit barrier, or snapshot transport** — these live in controller-side callback adapters, not the pure core.
- **No heavy controller WAL/repl/recovery wiring** — this spec defines only the seam contract and a thin adapter description; the integration is `ha-term-safety`.

## Boundary Commitments

### This Spec Owns
- The library log model: per-entry `uint64_t` term, 1-based index, the in-library leader progress vectors (`next_index[]`, `match_index[]`).
- New entry points: `raft_log_append`, `raft_on_append_entries_request`, `raft_on_append_entries_reply`.
- The four new callbacks in `raft_callbacks_t`: `on_persist_entry`, `on_get_entry_term`, `on_send_append_entries`, `on_truncate_from`.
- `UpdateTerm`-on-any-RPC applied on the AppendEntries/heartbeat path.
- The `(term, index)` reconcile seam: the library decides match / truncate / append; the controller routes the decision.
- Simulator extensions in `raft_sim.c` + tests in `src/lib/testsuite/test_raft.c` asserting Log Matching.

### Out of Boundary
- Commit-index advance and current-term commit safety (`raft-commit-safety`, G4).
- Vote-freshness `(term, index)` comparison in `RequestVote`/PreVote (`raft-commit-safety`, G6).
- Configuration markers in the unified log + the change-boundary snapshot (`ha-term-safety`).
- The controller's WAL byte layout, repl envelope serialization, recovery loop, commit barrier, fsync/durability, and snapshot transport internals (`ha-term-safety` adapter). This spec specifies only **what the adapter must route**, not its internal mechanics.

### Allowed Dependencies
- The implemented election core (`raft-library`): `raft_state_t`, `raft_callbacks_t`, `raft_init`, the vote/PreVote path. All unchanged.
- `errno.h`, `stdint.h`, `string.h` only (the library has no other dependencies and must stay free-standing for the simulator).
- The simulator harness (`raft_sim.c`) for verification.
- Must NOT depend on any controller header, WAL type, capnp type, or transport — the core stays pure.

### Revalidation Triggers
Dependent specs (`raft-commit-safety`, `raft-sim-conformance`, `ha-term-safety`) must re-check integration if any of the following change:
- The signature or semantics of any of the four new callbacks (`on_persist_entry`, `on_get_entry_term`, `on_send_append_entries`, `on_truncate_from`).
- The signature or return semantics of `raft_log_append`, `raft_on_append_entries_request`, `raft_on_append_entries_reply`.
- The 1-based-index ↔ WAL-LSN identity, or the "term `0` means absent" convention.
- The per-peer `next_index[]` / `match_index[]` initialization or update rules.
- The placement of `UpdateTerm` before `logOk` on the AppendEntries path.

## Architecture

### Existing Architecture Analysis

The library is a pure, callback-driven state machine (`raft.c`, ~430 lines). `raft_state_t` holds only Raft invariant state; application policy is delegated to `on_vote_eligible`. Persistence is write-only via `on_persist` — **the library never reads storage**; the caller restores `(term, voted_for)` into `raft_config_t` before `raft_init`. This feature follows the same discipline for the log: the library never reads the WAL directly; it reads entry terms only through `on_get_entry_term`, and writes/truncates only through callbacks. The pattern preserved:

- **Pure / synchronous / deterministic core** — no I/O, no threads, no allocation in the hot path. The log model uses fixed bookkeeping (a `Len(log)` counter `log_len` plus per-peer vectors); the entry **bytes** live in the adapter (WAL) / simulator (in-memory array), reached via callbacks. The library never owns entry storage — consistent with how it already does not own `(term, voted_for)` storage.
- **Persist-before-act ordering** — `on_persist` is called before granting a vote / sending RPCs; `on_persist_entry` is called before reporting an append success, identically.
- **Status returns, never self-terminate** — the library returns codes; the controller owns liveness and fencing. No new code path aborts.

### Architecture Pattern & Boundary Map

```mermaid
flowchart TB
    subgraph PureCore["src/lib/raft/raft.c — PURE CORE (this spec)"]
        APP["raft_log_append()<br/>ClientRequest (tla:246)"]
        SEND["AppendEntries emit<br/>(tla:204) next/match index"]
        REQ["raft_on_append_entries_request()<br/>HandleAppendEntriesRequest (tla:327)<br/>UpdateTerm (tla:406) first"]
        REPLY["raft_on_append_entries_reply()<br/>HandleAppendEntriesResponse (tla:393)"]
    end
    subgraph CBs["raft_callbacks_t — NEW seam (verbatim contract)"]
        C1["on_persist_entry(index,term,entry,len)"]
        C2["on_get_entry_term(index) -> term|0"]
        C3["on_send_append_entries(peer,prevIdx,prevTerm,leaderCommit,...)"]
        C4["on_truncate_from(index)"]
    end
    subgraph Consumers["ADAPTERS — implement callbacks (NOT this spec's heavy wiring)"]
        CTRL["controller adapter (ha-term-safety):<br/>WAL hdr.term=ha->term, repl envelope,<br/>recovery read-back, (term,idx) route"]
        SIM["raft_sim.c: in-memory log,<br/>synchronous routing (this spec)"]
    end
    APP --> C1
    SEND --> C3
    REQ --> C2 & C1 & C4
    REPLY -.updates.-> SEND
    C1 & C2 & C3 & C4 --> CTRL
    C1 & C2 & C3 & C4 --> SIM
```

**Architecture Integration**:
- **Selected pattern**: Pure state machine + callback ports (Hexagonal/Ports-and-Adapters), already established by the election core. New log actions are new ports; adapters (controller, simulator) implement them.
- **Domain boundaries**: protocol decisions (what to persist/send/truncate, accept/reject/conflict) stay in the library; I/O (WAL write, transport, recovery, in-memory backing) stays in adapters.
- **Existing patterns preserved**: `raft_config_t` extensibility, persist-before-act, NULL-checked optional callbacks, status-code returns, `XCE_TESTING`-gated simulator.
- **Steering compliance**: matches `src/lib/CLAUDE.md` (free-standing C, gnu17), `ha-async-io.md` (no blocking in the core; async lives in adapters), `testing.md` (TDD: simulator/test first), "controller never self-terminates" (library returns statuses only).

### Technology Stack

| Layer | Choice / Version | Role in Feature | Notes |
|-------|------------------|-----------------|-------|
| Library core | C (gnu17), `errno.h`/`stdint.h`/`string.h` only | Pure log state machine | Free-standing; no controller/capnp deps |
| Callbacks | `raft_callbacks_t` function pointers | Seam for I/O | 4 new required callbacks |
| Verification | `raft_sim.c` (XCE_TESTING) + `test_raft.c` | Deterministic Log Matching | In-memory log backing |
| Adapter (downstream) | controller `persist.c`/`repl.c` (`ha-term-safety`) | WAL/transport I/O | Not built here |

## File Structure Plan

### Modified Files
- `src/lib/raft/raft.h` — Add log-model fields to `raft_state_t` (`log_len`, `next_index[RAFT_MAX_PEERS]`, `match_index[RAFT_MAX_PEERS]`); add the four callbacks to `raft_callbacks_t`; declare `raft_log_append`, `raft_on_append_entries_request`, `raft_on_append_entries_reply`; add an `raft_append_result_t` reply struct; extend the simulator section (an in-memory log per `raft_sim_node_t`, an AppendEntries RPC buffer/route).
- `src/lib/raft/raft.c` — Implement the three new entry points + a private AppendEntries emit helper; add `on_persist_entry` ordering on `raft_init` validation; apply `UpdateTerm` on the AppendEntries path before `logOk`; initialize `next_index`/`match_index` in `raft_start_election`/`on_become_leader` path.
- `src/lib/raft/raft_sim.c` — In-memory per-node log; wire the four callbacks; add `raft_sim_deliver_append_entries` (synchronous request→reply routing honoring partitions); drop/dup/reorder + restart drivers; the Log Matching assertion helper.
- `src/lib/testsuite/test_raft.c` — New tests for each entry point and the simulator invariants (TDD-first).

### Directory Structure (unchanged)
```
src/lib/raft/
├── raft.h          # + log fields, 4 callbacks, 3 entry points, sim log
├── raft.c          # + ClientRequest / AppendEntries / Handle* / UpdateTerm-on-AE
└── raft_sim.c      # + in-memory log, AE routing, Log Matching assert
src/lib/testsuite/
└── test_raft.c     # + log-replication + Log Matching tests
```

> Each file keeps one responsibility: `raft.c` = pure protocol, `raft_sim.c` = deterministic harness + in-memory adapter, `test_raft.c` = assertions. No new file is introduced; the existing module pattern is extended.

## System Flows

### AppendEntries: leader emit → follower handle → leader reply (TLA+ actions in nodes)

```mermaid
sequenceDiagram
    participant L as Leader (raft.c)
    participant LC as Leader adapter (cb)
    participant FC as Follower adapter (cb)
    participant F as Follower (raft.c)
    Note over L: raft_log_append(v) — ClientRequest tla:246
    L->>LC: on_persist_entry(idx, term, v)
    Note over L: AppendEntries tla:204<br/>prevLogIndex=nextIndex[j]-1
    L->>LC: on_send_append_entries(j, prevIdx, prevTerm, leaderCommit, entry)
    LC-->>FC: (transport / sim route)
    Note over F: raft_on_append_entries_request()<br/>UpdateTerm tla:406 FIRST (G5)
    F->>FC: on_get_entry_term(prevIdx)  %% logOk (G2)
    alt term < currentTerm OR (follower AND not logOk)
        F-->>L: reply success=false, matchIndex=0
    else conflict: local[idx].term != entry.term (G3)
        F->>FC: on_truncate_from(idx)
        F->>FC: on_persist_entry(idx, term, entry)
        F-->>L: reply success=true, matchIndex=idx
    else no-conflict append OR already-have
        F->>FC: on_persist_entry(idx, term, entry)  %% append case
        F-->>L: reply success=true, matchIndex=prevIdx+len
    end
    Note over L: raft_on_append_entries_reply() tla:393
    alt success
        Note over L: nextIndex[j]=matchIndex+1; matchIndex[j]=reply.matchIndex
    else failure
        Note over L: nextIndex[j]=Max(nextIndex[j]-1, 1)  %% walk back
    end
```

Flow-level decisions:
- **UpdateTerm fires first** on the request path (before `logOk`), so a strictly-higher-term request is never spuriously rejected for a log mismatch (R6.4, closes G5).
- **`logOk`** is computed via `on_get_entry_term(prevLogIndex)`; a return of `0` is the "absent" sentinel consistent with `prevLogTerm == 0` when `prevLogIndex == 0` (R1.6, R4.1).
- **Conflict truncation** deletes `index..end` via `on_truncate_from` then appends in the same handler invocation (R4.5/R4.6, closes G3) — an intentional, safe collapse of `raft.tla`'s truncate-only conflict step plus the following no-conflict append; see **Note on conflict branch** below.
- **Commit is opaque** — `leaderCommit` is passed through untouched; this spec never advances commit (R3.4, R5.4).

## Requirements Traceability

| Requirement | Summary | Design Elements | raft.tla action |
|-------------|---------|-----------------|-----------------|
| 1.1–1.6 | Library log model, per-entry term | `raft_state_t.log_len`; `on_get_entry_term`; `on_persist_entry` | `ClientRequest` (246), `log[i].term` |
| 2.1–2.5 | ClientRequest append | `raft_log_append()` | `ClientRequest` (246) |
| 3.1–3.5 | AppendEntries emit, next/match | `next_index[]`/`match_index[]`; `on_send_append_entries` | `AppendEntries` (204), `BecomeLeader` (229) |
| 4.1–4.7 | logOk / reject / step-down / accept / conflict | `raft_on_append_entries_request()`; `on_get_entry_term`; `on_truncate_from`; `on_persist_entry` | `HandleAppendEntriesRequest` (327) |
| 5.1–5.4 | reply advance/decrement | `raft_on_append_entries_reply()` | `HandleAppendEntriesResponse` (393) |
| 6.1–6.5 | UpdateTerm on any RPC | term-adoption block in request handler (before logOk); `on_persist`; `on_become_follower` | `UpdateTerm` (406) |
| 7.1–7.7 | Callback contract + pure split | `raft_callbacks_t` additions; `raft_init` validation; adapter description | (seam, not a TLA+ action) |
| 8.1–8.6 | Simulator Log Matching | `raft_sim.c` AE routing, in-memory log, assert helpers | Log Matching invariant |

## Components and Interfaces

| Component | Layer | Intent | Req Coverage | Key Dependencies | Contracts |
|-----------|-------|--------|--------------|------------------|-----------|
| Log model (`raft_state_t`) | Library core | Per-entry term + per-peer progress | 1, 3 | election core (P0) | State |
| `raft_log_append` | Library core | ClientRequest append | 2 | on_persist_entry (P0) | Service |
| `raft_on_append_entries_request` | Library core | Handle+logOk+conflict+UpdateTerm | 4, 6 | on_get_entry_term, on_truncate_from, on_persist_entry, on_persist, on_become_follower (P0) | Service |
| `raft_on_append_entries_reply` | Library core | Advance/decrement progress | 5 | (internal state) | Service |
| New callbacks | Seam | I/O ports | 7 | adapters (P0) | Service |
| Simulator | Verification | Deterministic Log Matching | 8 | all of the above (P0) | State |

### Library Core

#### Log model additions to `raft_state_t`

| Field | Detail |
|-------|--------|
| Intent | Hold the library's view of log length and per-peer replication progress (TLA+ `logVars` length + `leaderVars`) |
| Requirements | 1.1, 1.2, 3.1 |

**Responsibilities & Constraints**
- `uint64_t log_len;` — `Len(log)`; the highest 1-based index the library believes is persisted locally. Incremented on append, set by truncate.
- `uint64_t next_index[RAFT_MAX_PEERS];` — per-peer `nextIndex` (1-based).
- `uint64_t match_index[RAFT_MAX_PEERS];` — per-peer `matchIndex` (0 = nothing matched).
- The library stores **no entry bytes and no per-entry term array** — entry terms are obtained via `on_get_entry_term`; bytes live in the adapter/simulator. This keeps `raft_state_t` fixed-size and the core pure (consistent with not storing `(term, voted_for)` bytes).
- Invariant: `match_index[j] ≤ next_index[j] - 1` for every peer; `log_len` is the 1:1 WAL LSN.

**State Management**
- `log_len` restored at init from the adapter's recovered WAL length (added to `raft_config_t` as `initial_log_len`, default 0; the library never reads storage).
- `next_index[j] = log_len + 1`, `match_index[j] = 0` set when the node becomes leader (mirrors `BecomeLeader`, tla:229).

#### `raft_log_append` — ClientRequest (raft.tla:246)

| Field | Detail |
|-------|--------|
| Intent | Append one entry stamped with `currentTerm` on the leader |
| Requirements | 2.1, 2.2, 2.3, 2.4, 2.5 |

##### Service Interface
```c
/*
 * raft_log_append: leader appends one client entry (raft.tla ClientRequest, 246).
 * Stamps the entry with s->term, assigns index Len(log)+1, persists it via
 * on_persist_entry BEFORE returning success (persist-before-act).
 * Does NOT advance any commit index (owned by raft-commit-safety).
 *
 * out_index: receives the assigned 1-based index (== WAL LSN) on success.
 * Returns RAFT_OK on success; -EPERM if role != RAFT_LEADER; the on_persist_entry
 * return is propagated on persist failure (no log mutation on failure).
 */
int raft_log_append(raft_state_t *s, const void *entry, size_t len,
                    uint64_t *out_index);
```
- Preconditions: `s->role == RAFT_LEADER`.
- Postconditions: `log_len` incremented by 1; `*out_index == log_len`; entry persisted at `(log_len, s->term)`.
- Invariants: appended entry term == `s->term`; no commit-index change.

#### `raft_on_append_entries_request` — HandleAppendEntriesRequest (raft.tla:327) + UpdateTerm (406)

| Field | Detail |
|-------|--------|
| Intent | Apply UpdateTerm first, then logOk + reject/step-down/accept (already-have / conflict-truncate / append) |
| Requirements | 4.1–4.7, 6.1–6.5 |

##### Service Interface
```c
/* Result of handling an AppendEntries request — the adapter sends this back. */
typedef struct {
    uint64_t reply_term;    /* current term to return to the leader */
    int      success;       /* 1 = accepted, 0 = rejected */
    uint64_t match_index;   /* on success: prevLogIndex + len(entries); else 0 */
} raft_append_result_t;

/*
 * raft_on_append_entries_request: handle an incoming AppendEntries / heartbeat.
 * Step order (must not be reordered):
 *  1. UpdateTerm (raft.tla:406, G5): if term > s->term, adopt term, clear vote,
 *     step down to FOLLOWER, on_persist(term,-1), fire on_become_follower if was
 *     leader. Applied on EVERY RPC including a 0-entry heartbeat, BEFORE logOk.
 *  2. logOk (raft.tla:328, G2): prevLogIndex==0 || (prevLogIndex<=log_len &&
 *     on_get_entry_term(prevLogIndex)==prevLogTerm).
 *  3. Reject (raft.tla:333): term<s->term || (term==s->term && FOLLOWER && !logOk)
 *     -> success=0, match_index=0.
 *  4. Candidate step-down (raft.tla:346): term==s->term && CANDIDATE -> FOLLOWER.
 *  5. Accept (raft.tla:351), index = prevLogIndex+1:
 *       already-have: entries empty, or local[index].term == entry.term -> success.
 *       conflict (raft.tla:375, G3): local[index] exists, term differs ->
 *         on_truncate_from(index), then on_persist_entry(index, entry).
 *       append (raft.tla:383): log_len == prevLogIndex -> on_persist_entry(index).
 * entries_len is 0 (heartbeat) or 1 (this spec, like raft.tla).
 * Returns RAFT_OK; fills *out.
 */
int raft_on_append_entries_request(raft_state_t *s, int leader_idx,
                                   uint64_t term, uint64_t prev_log_index,
                                   uint64_t prev_log_term, uint64_t leader_commit,
                                   const void *entry, size_t entry_len,
                                   uint64_t entry_term, int entries_len,
                                   raft_append_result_t *out);
```
- Preconditions: none beyond a valid `s`.
- Postconditions: local log matches the leader through `match_index` on success; divergent suffix truncated on conflict; term adopted if higher.
- Invariants: `on_truncate_from` is always followed by re-`on_persist_entry` of the leader's entry; commit is never advanced here (`leader_commit` is opaque, passed for the downstream commit spec to consume — ignored now).

> **Note on `leader_commit`**: accepted into the signature now (so the wire/sim shape is stable for `raft-commit-safety`) but **not acted on** in this spec — R3.4/R5.4. This avoids a later signature change (a Revalidation Trigger).

> **Note on conflict branch (intentional, safe divergence from `raft.tla`)**: `raft.tla` (lines 375–388) models the conflict as a **truncate-only** step — `HandleAppendEntriesRequest` deletes the conflicting suffix and returns, and the leader's entry is appended on a *later* AppendEntries via the no-conflict branch (raft.tla:383). XCE deliberately **collapses** the truncate and the immediately-following no-conflict append into a single handler invocation: `on_truncate_from(index)` **then** `on_persist_entry(index, term, entry)`. This is a safe optimization — `raft.tla` itself notes implementations may batch these steps — and Log Matching is preserved because any entry kept after truncation still matches the leader's `(term, index)`, and the freshly-persisted entry is exactly the leader's. On this conflict-and-append path the returned `match_index` is `prevLogIndex + len(entries)` (= `index`), identical to the value the success/no-conflict branch returns, so the leader's `nextIndex`/`matchIndex` accounting is unchanged.

#### `raft_on_append_entries_reply` — HandleAppendEntriesResponse (raft.tla:393)

| Field | Detail |
|-------|--------|
| Intent | Advance per-peer progress on success; walk back `nextIndex` on failure |
| Requirements | 5.1, 5.2, 5.3, 5.4 |

##### Service Interface
```c
/*
 * raft_on_append_entries_reply: process a follower's AppendEntries response
 * (raft.tla:393). Applied only when reply_term == s->term (stale-term drop, 415).
 * success: next_index[peer]=match_index+1; match_index[peer]=match_index.
 * failure: next_index[peer]=Max(next_index[peer]-1, 1); match_index unchanged.
 * Does NOT advance commit (owned by raft-commit-safety).
 * Returns RAFT_OK.
 */
int raft_on_append_entries_reply(raft_state_t *s, int peer_idx, int success,
                                 uint64_t match_index, uint64_t reply_term);
```
- Preconditions: `s->role == RAFT_LEADER` for the update to matter (stale replies are ignored).
- Postconditions: `next_index`/`match_index` for `peer_idx` updated per branch.
- Invariants: `next_index[peer] ≥ 1` always (floor); commit untouched.

#### AppendEntries emit helper (raft.tla:204)

| Field | Detail |
|-------|--------|
| Intent | Compute `prevLogIndex`/`prevLogTerm` and emit ≤1 entry via `on_send_append_entries` |
| Requirements | 3.1, 3.2, 3.3, 3.4, 3.5 |

**Responsibilities & Constraints**
- For peer `j`: `prevLogIndex = next_index[j] - 1`; `prevLogTerm = (prevLogIndex > 0) ? on_get_entry_term(prevLogIndex) : 0`.
- `lastEntry = Min(log_len, next_index[j])`; send `SubSeq(log, next_index[j], lastEntry)` — at most one entry; a heartbeat when `next_index[j] > log_len`.
- Calls `on_send_append_entries(ctx, j, prevLogIndex, prevLogTerm, leaderCommit, /* entries handle */ ...)`. `leaderCommit` is supplied opaquely by the caller (the controller/sim), since commit lives outside this spec.
- Emitted only while `s->role == RAFT_LEADER` (R3.5).

> Exposed as `raft_send_append_entries(raft_state_t *s, int peer_idx, uint64_t leader_commit)` so the controller/simulator can drive replication per-peer (fan-out scheduling itself is an adapter concern).

### Seam — `raft_callbacks_t` additions (verbatim Authoritative shared contract)

These are appended to `raft_callbacks_t` and validated **required** in `raft_init`. Reproduced **verbatim** from the brief so downstream specs depend on the exact shapes:

```c
/* REQUIRED: durably append/overwrite the log entry at `index` with `term`. */
int      on_persist_entry(void *ctx, uint64_t index, uint64_t term,
                          const void *entry, size_t len);

/* REQUIRED: term of the local entry at `index` (0 if absent). */
uint64_t on_get_entry_term(void *ctx, uint64_t index);

/* REQUIRED: transmit one AppendEntries to peer_idx with prevLog{Index,Term},
 * leaderCommit, and n_entries (0 = heartbeat, 1 = one entry). The library holds
 * no entry bytes, so when n_entries==1 the adapter sources the single entry at
 * prev_log_index+1 from its own log. [Concretizes the original varargs "entries
 * handle" placeholder; impl 2026-06-21.] */
int      on_send_append_entries(void *ctx, int peer_idx,
                                uint64_t prev_log_index, uint64_t prev_log_term,
                                uint64_t leader_commit, int n_entries);

/* REQUIRED: delete local entries at index..end. */
int      on_truncate_from(void *ctx, uint64_t index);
```

**Contract rules (R7.5)**: the library decides *what / whether* to persist, send, or truncate; the callback only performs I/O. `on_get_entry_term` returning `0` is the canonical "absent" signal (used by `logOk` and by `prevLogTerm` computation). All four are NULL-checked in `raft_init` and required (return `-EINVAL` if NULL), matching how the election-required callbacks are validated today.

### Controller adapter (seam only — heavy wiring is `ha-term-safety`)

This spec describes the adapter **only at the seam**; the full controller integration is the downstream `ha-term-safety` consumer.

- `on_persist_entry` → controller WAL write that stamps `hdr.term` from `ha->term` (closing G1 at the byte level), at the LSN equal to `index`.
- `on_get_entry_term` → read the recorded `hdr.term` for a given LSN (from the in-memory WAL index / recovery read-back).
- `on_send_append_entries` → enqueue a repl envelope carrying `prevLogIndex`, `prevLogTerm`, the entry, and its term, onto the existing repl transport.
- `on_truncate_from` → truncate the WAL at/after the LSN, replacing the LSN-only reconcile in `ha_convergence.c` with the library's `(term, index)` decision.
- On recovery, the controller restores `log_len` (= recovered WAL length) and each entry's term, feeding `on_get_entry_term`; `initial_log_len` is passed into `raft_config_t`.

## Data Models

### Domain Model
- **Log entry (logical)**: `(index: uint64, term: uint64, value: bytes)`. The library holds only `index` accounting (`log_len`) and obtains `term` via callback; `value` bytes are owned by the adapter (WAL) / simulator (in-memory array). Index is 1-based, identical to the controller WAL LSN.
- **Per-peer progress**: `(next_index: uint64, match_index: uint64)` per peer, the TLA+ `leaderVars`.

### Logical Data Model
- `raft_state_t` gains `log_len`, `next_index[RAFT_MAX_PEERS]`, `match_index[RAFT_MAX_PEERS]`.
- `raft_config_t` gains `initial_log_len` (restored WAL length at boot; 0 on first boot — the library never reads storage, consistent with `initial_term`/`initial_voted_for`).
- `raft_append_result_t` is the request-handler reply payload.
- **No on-disk format change**: the WAL header already carries a `hdr.term` field (today always 0); this feature only requires the adapter to write the real term. The library imposes no new persisted layout (R7 in requirements / unchanged on-disk format per `ha-term-safety` R1.7).

### Data Contracts & Integration
- **Entries handle**: `on_send_append_entries` and `raft_on_append_entries_request` carry one entry as `(const void *entry, size_t len, uint64_t entry_term)` plus an `entries_len ∈ {0,1}`. This mirrors `raft.tla`'s "up to 1 entry" simplification and keeps the simulator and controller envelope shapes identical.
- **Term `0` sentinel**: `on_get_entry_term` returns `0` for an absent index; legacy pre-feature entries with a zero term are handled by the controller adapter (mapped to the lowest-known term) per `ha-term-safety` R1.8 — out of this library's scope, but the `0`-means-absent convention is honored consistently here.

## Error Handling

### Error Strategy
The library returns status codes only and **never aborts** (preserves "controller never self-terminates"). Callback failures propagate as negative returns; the caller decides liveness.

### Error Categories and Responses
- **Caller misuse**: `raft_log_append` while not leader → `-EPERM`, no log change. NULL required callback at `raft_init` → `-EINVAL`.
- **Persist failure**: `on_persist_entry` non-zero return → propagated; the library does **not** increment `log_len` on a failed append (no phantom entry), preserving the persist-before-act guarantee.
- **Truncate failure**: `on_truncate_from` non-zero return on the conflict branch → reported as a non-success request result so the leader retries (the follower has not corrupted its log; it simply did not converge this round).
- **Stale RPCs**: a request with `term < currentTerm` is rejected (success=0, matchIndex=0); a reply with `reply_term < currentTerm` is ignored (DropStaleResponse, tla:415).
- **Higher-term RPC**: always adopted first (UpdateTerm) — never rejected for a log mismatch (R6.4).

### Monitoring
Reuse the optional `on_trace` callback for new events: `"ae_recv"`, `"ae_reject_stale"`, `"ae_reject_logmismatch"`, `"ae_conflict_truncate"`, `"ae_append"`, `"term_adopt_ae"`, `"ae_reply_advance"`, `"ae_reply_backoff"`. NULL-checked exactly like existing trace calls.

## Testing Strategy

TDD-mandatory (`testing.md`): tests and simulator drivers are written **before** the `raft.c` implementation (RED → GREEN). All tests are `XCE_TESTING`-gated and live in `src/lib/testsuite/test_raft.c` driving `raft_sim.c`.

### Unit Tests (entry points, against in-memory sim log)
1. `raft_log_append` on a leader stamps `currentTerm`, assigns `Len(log)+1`, calls `on_persist_entry` once, returns the index (R2.1–2.4); on a follower returns `-EPERM` with no log change (R2.2).
2. `raft_on_append_entries_request` `logOk`: accepts when `prevLogIndex==0`; accepts when `on_get_entry_term(prevLogIndex)==prevLogTerm`; rejects (success=0, matchIndex=0) when terms differ (R4.1, R4.2 — G2).
3. Conflict branch: a follower with `(index=k, term=A)` receiving `(index=k, term=B)` calls `on_truncate_from(k)` then `on_persist_entry(k, B)` and reports success with `matchIndex=k` (R4.5, R4.6 — G3).
4. `raft_on_append_entries_reply`: success sets `next_index=matchIndex+1`, `match_index=reply`; failure decrements `next_index` flooring at 1; stale `reply_term` ignored (R5.1–5.3).
5. `UpdateTerm` on AppendEntries: a follower/leader at term T receiving an AE/heartbeat at term T+1 adopts T+1, clears vote, persists, and (if was leader) fires `on_become_follower` — applied before `logOk`, including a 0-entry heartbeat (R6.1–6.4 — G5).

### Integration / Simulator Tests (Log Matching invariant)
1. Two-phase replication: leader appends 3 entries, replicates to a quorum; assert every node's prefix is byte-identical (Log Matching, R8.4).
2. Partition + heal: isolate a follower during several appends, then heal; assert AppendEntries walks `nextIndex` back, the follower catches up, and Log Matching holds (R8.1, R8.3).
3. Divergent-suffix truncation: hand a follower a manufactured divergent tail (same indices, different term), replicate from a higher-term leader; assert the follower truncates and converges to the leader's prefix (R8.5 — G3).
4. Chaotic churn / G5 regression: a stale-term node (term lagging) rejoins under a higher-term leader; assert it adopts the higher term from the AppendEntries/heartbeat path and does **not** re-elect at a reused term (R8.6 — closes the bug-851 G5 mechanism in the deterministic harness).

### Message-fault Tests
1. Drop, duplicate, and reorder AppendEntries requests/replies in the sim queue; assert idempotent convergence and Log Matching after the faults clear (R8.3) — the already-have accept branch (R4.4) makes a duplicated AE a no-op.