myxce/.kiro/specs/raft-log-replication/design.md

Design Document

Overview

Purpose: This feature extends the election-only Raft library (src/lib/raft/raft.{c,h}) into a strict implementation of the log half of raft/raft.tla (Ongaro canonical), giving the library ownership of the replicated log. It closes the four log-half gaps of bug-851 (G1/G2/G3/G5): per-entry term, prevLogTerm log-matching, (term, index) conflict truncation, and term adoption on the AppendEntries/heartbeat path.

Users: The XCE controller (as a Raft node), the deterministic simulator (raft_sim.c), and three downstream specs that consume this seam verbatim — raft-commit-safety, raft-sim-conformance, and ha-term-safety.

Impact: Today the replicated log is hand-rolled across the controller (persist.c/repl.c/ha.c/ha_convergence.c) and matches by LSN contiguity only. After this feature, the library owns log growth, AppendEntries emission/handling, per-peer progress, and (term, index) truncation as pure, synchronous, deterministic state transitions; the controller becomes a thin adapter that performs I/O behind four new callbacks. The on-disk WAL format is unchanged (the term is a value fix, not a layout change).

Goals

• Add a library-owned log model where every entry carries a uint64_t term, index 1-based mapping 1:1 to the WAL LSN (G1).
• Implement ClientRequest, AppendEntries emission, HandleAppendEntriesRequest, and HandleAppendEntriesResponse exactly as raft.tla defines them (G2/G3).
• Apply UpdateTerm on every RPC including AppendEntries/heartbeat (G5).
• Define the four new callbacks verbatim per the brief's Authoritative shared contract.
• Extend raft_sim.c to assert the Log Matching invariant and divergent-suffix truncation under partitions, drop/dup/reorder, and restart.

Non-Goals

• No commit-index advance (AdvanceCommitIndex, G4) and no Figure-8 current-term commit rule — owned by raft-commit-safety.
• No (lastLogTerm, lastLogIndex) vote freshness (G6) — owned by raft-commit-safety.
• No config-change markers / upgrade snapshot — owned by ha-term-safety.
• No async I/O, fan-out scheduling, commit barrier, or snapshot transport — these live in controller-side callback adapters, not the pure core.
• No heavy controller WAL/repl/recovery wiring — this spec defines only the seam contract and a thin adapter description; the integration is ha-term-safety.

Boundary Commitments

This Spec Owns

• The library log model: per-entry uint64_t term, 1-based index, the in-library leader progress vectors (next_index[], match_index[]).
• New entry points: raft_log_append, raft_on_append_entries_request, raft_on_append_entries_reply.
• The four new callbacks in raft_callbacks_t: on_persist_entry, on_get_entry_term, on_send_append_entries, on_truncate_from.
• UpdateTerm-on-any-RPC applied on the AppendEntries/heartbeat path.
• The (term, index) reconcile seam: the library decides match / truncate / append; the controller routes the decision.
• Simulator extensions in raft_sim.c + tests in src/lib/testsuite/test_raft.c asserting Log Matching.

Out of Boundary

• Commit-index advance and current-term commit safety (raft-commit-safety, G4).
• Vote-freshness (term, index) comparison in RequestVote/PreVote (raft-commit-safety, G6).
• Configuration markers in the unified log + the change-boundary snapshot (ha-term-safety).
• The controller's WAL byte layout, repl envelope serialization, recovery loop, commit barrier, fsync/durability, and snapshot transport internals (ha-term-safety adapter). This spec specifies only what the adapter must route, not its internal mechanics.

Allowed Dependencies

• The implemented election core (raft-library): raft_state_t, raft_callbacks_t, raft_init, the vote/PreVote path. All unchanged.
• errno.h, stdint.h, string.h only (the library has no other dependencies and must stay free-standing for the simulator).
• The simulator harness (raft_sim.c) for verification.
• Must NOT depend on any controller header, WAL type, capnp type, or transport — the core stays pure.

Revalidation Triggers

Dependent specs (raft-commit-safety, raft-sim-conformance, ha-term-safety) must re-check integration if any of the following change:

• The signature or semantics of any of the four new callbacks (on_persist_entry, on_get_entry_term, on_send_append_entries, on_truncate_from).
• The signature or return semantics of raft_log_append, raft_on_append_entries_request, raft_on_append_entries_reply.
• The 1-based-index ↔ WAL-LSN identity, or the "term 0 means absent" convention.
• The per-peer next_index[] / match_index[] initialization or update rules.
• The placement of UpdateTerm before logOk on the AppendEntries path.

Architecture

Existing Architecture Analysis

The library is a pure, callback-driven state machine (raft.c, ~430 lines). raft_state_t holds only Raft invariant state; application policy is delegated to on_vote_eligible. Persistence is write-only via on_persist — the library never reads storage; the caller restores (term, voted_for) into raft_config_t before raft_init. This feature follows the same discipline for the log: the library never reads the WAL directly; it reads entry terms only through on_get_entry_term, and writes/truncates only through callbacks. The pattern preserved:

• Pure / synchronous / deterministic core — no I/O, no threads, no allocation in the hot path. The log model uses fixed bookkeeping (a Len(log) counter log_len plus per-peer vectors); the entry bytes live in the adapter (WAL) / simulator (in-memory array), reached via callbacks. The library never owns entry storage — consistent with how it already does not own (term, voted_for) storage.
• Persist-before-act ordering — on_persist is called before granting a vote / sending RPCs; on_persist_entry is called before reporting an append success, identically.
• Status returns, never self-terminate — the library returns codes; the controller owns liveness and fencing. No new code path aborts.

Architecture Pattern & Boundary Map

flowchart TB
    subgraph PureCore["src/lib/raft/raft.c — PURE CORE (this spec)"]
        APP["raft_log_append()<br/>ClientRequest (tla:246)"]
        SEND["AppendEntries emit<br/>(tla:204) next/match index"]
        REQ["raft_on_append_entries_request()<br/>HandleAppendEntriesRequest (tla:327)<br/>UpdateTerm (tla:406) first"]
        REPLY["raft_on_append_entries_reply()<br/>HandleAppendEntriesResponse (tla:393)"]
    end
    subgraph CBs["raft_callbacks_t — NEW seam (verbatim contract)"]
        C1["on_persist_entry(index,term,entry,len)"]
        C2["on_get_entry_term(index) -> term|0"]
        C3["on_send_append_entries(peer,prevIdx,prevTerm,leaderCommit,...)"]
        C4["on_truncate_from(index)"]
    end
    subgraph Consumers["ADAPTERS — implement callbacks (NOT this spec's heavy wiring)"]
        CTRL["controller adapter (ha-term-safety):<br/>WAL hdr.term=ha->term, repl envelope,<br/>recovery read-back, (term,idx) route"]
        SIM["raft_sim.c: in-memory log,<br/>synchronous routing (this spec)"]
    end
    APP --> C1
    SEND --> C3
    REQ --> C2 & C1 & C4
    REPLY -.updates.-> SEND
    C1 & C2 & C3 & C4 --> CTRL
    C1 & C2 & C3 & C4 --> SIM

Architecture Integration:

• Selected pattern: Pure state machine + callback ports (Hexagonal/Ports-and-Adapters), already established by the election core. New log actions are new ports; adapters (controller, simulator) implement them.
• Domain boundaries: protocol decisions (what to persist/send/truncate, accept/reject/conflict) stay in the library; I/O (WAL write, transport, recovery, in-memory backing) stays in adapters.
• Existing patterns preserved: raft_config_t extensibility, persist-before-act, NULL-checked optional callbacks, status-code returns, XCE_TESTING-gated simulator.
• Steering compliance: matches src/lib/CLAUDE.md (free-standing C, gnu17), ha-async-io.md (no blocking in the core; async lives in adapters), testing.md (TDD: simulator/test first), "controller never self-terminates" (library returns statuses only).

Technology Stack

Layer	Choice / Version	Role in Feature	Notes
Library core	C (gnu17), errno.h/stdint.h/string.h only	Pure log state machine	Free-standing; no controller/capnp deps
Callbacks	raft_callbacks_t function pointers	Seam for I/O	4 new required callbacks
Verification	raft_sim.c (XCE_TESTING) + test_raft.c	Deterministic Log Matching	In-memory log backing
Adapter (downstream)	controller persist.c/repl.c (ha-term-safety)	WAL/transport I/O	Not built here

File Structure Plan

Modified Files

• src/lib/raft/raft.h — Add log-model fields to raft_state_t (log_len, next_index[RAFT_MAX_PEERS], match_index[RAFT_MAX_PEERS]); add the four callbacks to raft_callbacks_t; declare raft_log_append, raft_on_append_entries_request, raft_on_append_entries_reply; add an raft_append_result_t reply struct; extend the simulator section (an in-memory log per raft_sim_node_t, an AppendEntries RPC buffer/route).
• src/lib/raft/raft.c — Implement the three new entry points + a private AppendEntries emit helper; add on_persist_entry ordering on raft_init validation; apply UpdateTerm on the AppendEntries path before logOk; initialize next_index/match_index in raft_start_election/on_become_leader path.
• src/lib/raft/raft_sim.c — In-memory per-node log; wire the four callbacks; add raft_sim_deliver_append_entries (synchronous request→reply routing honoring partitions); drop/dup/reorder + restart drivers; the Log Matching assertion helper.
• src/lib/testsuite/test_raft.c — New tests for each entry point and the simulator invariants (TDD-first).

Directory Structure (unchanged)

src/lib/raft/
├── raft.h          # + log fields, 4 callbacks, 3 entry points, sim log
├── raft.c          # + ClientRequest / AppendEntries / Handle* / UpdateTerm-on-AE
└── raft_sim.c      # + in-memory log, AE routing, Log Matching assert
src/lib/testsuite/
└── test_raft.c     # + log-replication + Log Matching tests

Each file keeps one responsibility: raft.c = pure protocol, raft_sim.c = deterministic harness + in-memory adapter, test_raft.c = assertions. No new file is introduced; the existing module pattern is extended.

System Flows

AppendEntries: leader emit → follower handle → leader reply (TLA+ actions in nodes)

sequenceDiagram
    participant L as Leader (raft.c)
    participant LC as Leader adapter (cb)
    participant FC as Follower adapter (cb)
    participant F as Follower (raft.c)
    Note over L: raft_log_append(v) — ClientRequest tla:246
    L->>LC: on_persist_entry(idx, term, v)
    Note over L: AppendEntries tla:204<br/>prevLogIndex=nextIndex[j]-1
    L->>LC: on_send_append_entries(j, prevIdx, prevTerm, leaderCommit, entry)
    LC-->>FC: (transport / sim route)
    Note over F: raft_on_append_entries_request()<br/>UpdateTerm tla:406 FIRST (G5)
    F->>FC: on_get_entry_term(prevIdx)  %% logOk (G2)
    alt term < currentTerm OR (follower AND not logOk)
        F-->>L: reply success=false, matchIndex=0
    else conflict: local[idx].term != entry.term (G3)
        F->>FC: on_truncate_from(idx)
        F->>FC: on_persist_entry(idx, term, entry)
        F-->>L: reply success=true, matchIndex=idx
    else no-conflict append OR already-have
        F->>FC: on_persist_entry(idx, term, entry)  %% append case
        F-->>L: reply success=true, matchIndex=prevIdx+len
    end
    Note over L: raft_on_append_entries_reply() tla:393
    alt success
        Note over L: nextIndex[j]=matchIndex+1; matchIndex[j]=reply.matchIndex
    else failure
        Note over L: nextIndex[j]=Max(nextIndex[j]-1, 1)  %% walk back
    end

Flow-level decisions:

• UpdateTerm fires first on the request path (before logOk), so a strictly-higher-term request is never spuriously rejected for a log mismatch (R6.4, closes G5).
• logOk is computed via on_get_entry_term(prevLogIndex); a return of 0 is the "absent" sentinel consistent with prevLogTerm == 0 when prevLogIndex == 0 (R1.6, R4.1).
• Conflict truncation deletes index..end via on_truncate_from then appends in the same handler invocation (R4.5/R4.6, closes G3) — an intentional, safe collapse of raft.tla's truncate-only conflict step plus the following no-conflict append; see Note on conflict branch below.
• Commit is opaque — leaderCommit is passed through untouched; this spec never advances commit (R3.4, R5.4).

Requirements Traceability

Requirement	Summary	Design Elements	raft.tla action
1.1–1.6	Library log model, per-entry term	raft_state_t.log_len; on_get_entry_term; on_persist_entry	ClientRequest (246), log[i].term
2.1–2.5	ClientRequest append	raft_log_append()	ClientRequest (246)
3.1–3.5	AppendEntries emit, next/match	next_index[]/match_index[]; on_send_append_entries	AppendEntries (204), BecomeLeader (229)
4.1–4.7	logOk / reject / step-down / accept / conflict	raft_on_append_entries_request(); on_get_entry_term; on_truncate_from; on_persist_entry	HandleAppendEntriesRequest (327)
5.1–5.4	reply advance/decrement	raft_on_append_entries_reply()	HandleAppendEntriesResponse (393)
6.1–6.5	UpdateTerm on any RPC	term-adoption block in request handler (before logOk); on_persist; on_become_follower	UpdateTerm (406)
7.1–7.7	Callback contract + pure split	raft_callbacks_t additions; raft_init validation; adapter description	(seam, not a TLA+ action)
8.1–8.6	Simulator Log Matching	raft_sim.c AE routing, in-memory log, assert helpers	Log Matching invariant

Components and Interfaces

Component	Layer	Intent	Req Coverage	Key Dependencies	Contracts
Log model (raft_state_t)	Library core	Per-entry term + per-peer progress	1, 3	election core (P0)	State
raft_log_append	Library core	ClientRequest append	2	on_persist_entry (P0)	Service
raft_on_append_entries_request	Library core	Handle+logOk+conflict+UpdateTerm	4, 6	on_get_entry_term, on_truncate_from, on_persist_entry, on_persist, on_become_follower (P0)	Service
raft_on_append_entries_reply	Library core	Advance/decrement progress	5	(internal state)	Service
New callbacks	Seam	I/O ports	7	adapters (P0)	Service
Simulator	Verification	Deterministic Log Matching	8	all of the above (P0)	State

Library Core

Log model additions to raft_state_t

Field	Detail
Intent	Hold the library's view of log length and per-peer replication progress (TLA+ logVars length + leaderVars)
Requirements	1.1, 1.2, 3.1

Responsibilities & Constraints

• uint64_t log_len; — Len(log); the highest 1-based index the library believes is persisted locally. Incremented on append, set by truncate.
• uint64_t next_index[RAFT_MAX_PEERS]; — per-peer nextIndex (1-based).
• uint64_t match_index[RAFT_MAX_PEERS]; — per-peer matchIndex (0 = nothing matched).
• The library stores no entry bytes and no per-entry term array — entry terms are obtained via on_get_entry_term; bytes live in the adapter/simulator. This keeps raft_state_t fixed-size and the core pure (consistent with not storing (term, voted_for) bytes).
• Invariant: match_index[j] ≤ next_index[j] - 1 for every peer; log_len is the 1:1 WAL LSN.

State Management

• log_len restored at init from the adapter's recovered WAL length (added to raft_config_t as initial_log_len, default 0; the library never reads storage).
• next_index[j] = log_len + 1, match_index[j] = 0 set when the node becomes leader (mirrors BecomeLeader, tla:229).

raft_log_append — ClientRequest (raft.tla:246)

Field	Detail
Intent	Append one entry stamped with currentTerm on the leader
Requirements	2.1, 2.2, 2.3, 2.4, 2.5

Service Interface

/*
 * raft_log_append: leader appends one client entry (raft.tla ClientRequest, 246).
 * Stamps the entry with s->term, assigns index Len(log)+1, persists it via
 * on_persist_entry BEFORE returning success (persist-before-act).
 * Does NOT advance any commit index (owned by raft-commit-safety).
 *
 * out_index: receives the assigned 1-based index (== WAL LSN) on success.
 * Returns RAFT_OK on success; -EPERM if role != RAFT_LEADER; the on_persist_entry
 * return is propagated on persist failure (no log mutation on failure).
 */
int raft_log_append(raft_state_t *s, const void *entry, size_t len,
                    uint64_t *out_index);

• Preconditions: s->role == RAFT_LEADER.
• Postconditions: log_len incremented by 1; *out_index == log_len; entry persisted at (log_len, s->term).
• Invariants: appended entry term == s->term; no commit-index change.

raft_on_append_entries_request — HandleAppendEntriesRequest (raft.tla:327) + UpdateTerm (406)

Field	Detail
Intent	Apply UpdateTerm first, then logOk + reject/step-down/accept (already-have / conflict-truncate / append)
Requirements	4.1–4.7, 6.1–6.5

Service Interface

/* Result of handling an AppendEntries request — the adapter sends this back. */
typedef struct {
    uint64_t reply_term;    /* current term to return to the leader */
    int      success;       /* 1 = accepted, 0 = rejected */
    uint64_t match_index;   /* on success: prevLogIndex + len(entries); else 0 */
} raft_append_result_t;

/*
 * raft_on_append_entries_request: handle an incoming AppendEntries / heartbeat.
 * Step order (must not be reordered):
 *  1. UpdateTerm (raft.tla:406, G5): if term > s->term, adopt term, clear vote,
 *     step down to FOLLOWER, on_persist(term,-1), fire on_become_follower if was
 *     leader. Applied on EVERY RPC including a 0-entry heartbeat, BEFORE logOk.
 *  2. logOk (raft.tla:328, G2): prevLogIndex==0 || (prevLogIndex<=log_len &&
 *     on_get_entry_term(prevLogIndex)==prevLogTerm).
 *  3. Reject (raft.tla:333): term<s->term || (term==s->term && FOLLOWER && !logOk)
 *     -> success=0, match_index=0.
 *  4. Candidate step-down (raft.tla:346): term==s->term && CANDIDATE -> FOLLOWER.
 *  5. Accept (raft.tla:351), index = prevLogIndex+1:
 *       already-have: entries empty, or local[index].term == entry.term -> success.
 *       conflict (raft.tla:375, G3): local[index] exists, term differs ->
 *         on_truncate_from(index), then on_persist_entry(index, entry).
 *       append (raft.tla:383): log_len == prevLogIndex -> on_persist_entry(index).
 * entries_len is 0 (heartbeat) or 1 (this spec, like raft.tla).
 * Returns RAFT_OK; fills *out.
 */
int raft_on_append_entries_request(raft_state_t *s, int leader_idx,
                                   uint64_t term, uint64_t prev_log_index,
                                   uint64_t prev_log_term, uint64_t leader_commit,
                                   const void *entry, size_t entry_len,
                                   uint64_t entry_term, int entries_len,
                                   raft_append_result_t *out);

• Preconditions: none beyond a valid s.
• Postconditions: local log matches the leader through match_index on success; divergent suffix truncated on conflict; term adopted if higher.
• Invariants: on_truncate_from is always followed by re-on_persist_entry of the leader's entry; commit is never advanced here (leader_commit is opaque, passed for the downstream commit spec to consume — ignored now).

Note on leader_commit: accepted into the signature now (so the wire/sim shape is stable for raft-commit-safety) but not acted on in this spec — R3.4/R5.4. This avoids a later signature change (a Revalidation Trigger).

Note on conflict branch (intentional, safe divergence from raft.tla): raft.tla (lines 375–388) models the conflict as a truncate-only step — HandleAppendEntriesRequest deletes the conflicting suffix and returns, and the leader's entry is appended on a later AppendEntries via the no-conflict branch (raft.tla:383). XCE deliberately collapses the truncate and the immediately-following no-conflict append into a single handler invocation: on_truncate_from(index) then on_persist_entry(index, term, entry). This is a safe optimization — raft.tla itself notes implementations may batch these steps — and Log Matching is preserved because any entry kept after truncation still matches the leader's (term, index), and the freshly-persisted entry is exactly the leader's. On this conflict-and-append path the returned match_index is prevLogIndex + len(entries) (= index), identical to the value the success/no-conflict branch returns, so the leader's nextIndex/matchIndex accounting is unchanged.

raft_on_append_entries_reply — HandleAppendEntriesResponse (raft.tla:393)

Field	Detail
Intent	Advance per-peer progress on success; walk back nextIndex on failure
Requirements	5.1, 5.2, 5.3, 5.4

Service Interface

/*
 * raft_on_append_entries_reply: process a follower's AppendEntries response
 * (raft.tla:393). Applied only when reply_term == s->term (stale-term drop, 415).
 * success: next_index[peer]=match_index+1; match_index[peer]=match_index.
 * failure: next_index[peer]=Max(next_index[peer]-1, 1); match_index unchanged.
 * Does NOT advance commit (owned by raft-commit-safety).
 * Returns RAFT_OK.
 */
int raft_on_append_entries_reply(raft_state_t *s, int peer_idx, int success,
                                 uint64_t match_index, uint64_t reply_term);

• Preconditions: s->role == RAFT_LEADER for the update to matter (stale replies are ignored).
• Postconditions: next_index/match_index for peer_idx updated per branch.
• Invariants: next_index[peer] ≥ 1 always (floor); commit untouched.

AppendEntries emit helper (raft.tla:204)

Field	Detail
Intent	Compute prevLogIndex/prevLogTerm and emit ≤1 entry via on_send_append_entries
Requirements	3.1, 3.2, 3.3, 3.4, 3.5

Responsibilities & Constraints

• For peer j: prevLogIndex = next_index[j] - 1; prevLogTerm = (prevLogIndex > 0) ? on_get_entry_term(prevLogIndex) : 0.
• lastEntry = Min(log_len, next_index[j]); send SubSeq(log, next_index[j], lastEntry) — at most one entry; a heartbeat when next_index[j] > log_len.
• Calls on_send_append_entries(ctx, j, prevLogIndex, prevLogTerm, leaderCommit, /* entries handle */ ...). leaderCommit is supplied opaquely by the caller (the controller/sim), since commit lives outside this spec.
• Emitted only while s->role == RAFT_LEADER (R3.5).

Exposed as raft_send_append_entries(raft_state_t *s, int peer_idx, uint64_t leader_commit) so the controller/simulator can drive replication per-peer (fan-out scheduling itself is an adapter concern).

Seam — raft_callbacks_t additions (verbatim Authoritative shared contract)

These are appended to raft_callbacks_t and validated required in raft_init. Reproduced verbatim from the brief so downstream specs depend on the exact shapes:

/* REQUIRED: durably append/overwrite the log entry at `index` with `term`. */
int      on_persist_entry(void *ctx, uint64_t index, uint64_t term,
                          const void *entry, size_t len);

/* REQUIRED: term of the local entry at `index` (0 if absent). */
uint64_t on_get_entry_term(void *ctx, uint64_t index);

/* REQUIRED: transmit one AppendEntries to peer_idx with prevLog{Index,Term},
 * leaderCommit, and n_entries (0 = heartbeat, 1 = one entry). The library holds
 * no entry bytes, so when n_entries==1 the adapter sources the single entry at
 * prev_log_index+1 from its own log. [Concretizes the original varargs "entries
 * handle" placeholder; impl 2026-06-21.] */
int      on_send_append_entries(void *ctx, int peer_idx,
                                uint64_t prev_log_index, uint64_t prev_log_term,
                                uint64_t leader_commit, int n_entries);

/* REQUIRED: delete local entries at index..end. */
int      on_truncate_from(void *ctx, uint64_t index);

Contract rules (R7.5): the library decides what / whether to persist, send, or truncate; the callback only performs I/O. on_get_entry_term returning 0 is the canonical "absent" signal (used by logOk and by prevLogTerm computation). All four are NULL-checked in raft_init and required (return -EINVAL if NULL), matching how the election-required callbacks are validated today.

Controller adapter (seam only — heavy wiring is ha-term-safety)

This spec describes the adapter only at the seam; the full controller integration is the downstream ha-term-safety consumer.

• on_persist_entry → controller WAL write that stamps hdr.term from ha->term (closing G1 at the byte level), at the LSN equal to index.
• on_get_entry_term → read the recorded hdr.term for a given LSN (from the in-memory WAL index / recovery read-back).
• on_send_append_entries → enqueue a repl envelope carrying prevLogIndex, prevLogTerm, the entry, and its term, onto the existing repl transport.
• on_truncate_from → truncate the WAL at/after the LSN, replacing the LSN-only reconcile in ha_convergence.c with the library's (term, index) decision.
• On recovery, the controller restores log_len (= recovered WAL length) and each entry's term, feeding on_get_entry_term; initial_log_len is passed into raft_config_t.

Data Models

Domain Model

• Log entry (logical): (index: uint64, term: uint64, value: bytes). The library holds only index accounting (log_len) and obtains term via callback; value bytes are owned by the adapter (WAL) / simulator (in-memory array). Index is 1-based, identical to the controller WAL LSN.
• Per-peer progress: (next_index: uint64, match_index: uint64) per peer, the TLA+ leaderVars.

Logical Data Model

• raft_state_t gains log_len, next_index[RAFT_MAX_PEERS], match_index[RAFT_MAX_PEERS].
• raft_config_t gains initial_log_len (restored WAL length at boot; 0 on first boot — the library never reads storage, consistent with initial_term/initial_voted_for).
• raft_append_result_t is the request-handler reply payload.
• No on-disk format change: the WAL header already carries a hdr.term field (today always 0); this feature only requires the adapter to write the real term. The library imposes no new persisted layout (R7 in requirements / unchanged on-disk format per ha-term-safety R1.7).

Data Contracts & Integration

• Entries handle: on_send_append_entries and raft_on_append_entries_request carry one entry as (const void *entry, size_t len, uint64_t entry_term) plus an entries_len ∈ {0,1}. This mirrors raft.tla's "up to 1 entry" simplification and keeps the simulator and controller envelope shapes identical.
• Term 0 sentinel: on_get_entry_term returns 0 for an absent index; legacy pre-feature entries with a zero term are handled by the controller adapter (mapped to the lowest-known term) per ha-term-safety R1.8 — out of this library's scope, but the 0-means-absent convention is honored consistently here.

Error Handling

Error Strategy

The library returns status codes only and never aborts (preserves "controller never self-terminates"). Callback failures propagate as negative returns; the caller decides liveness.

Error Categories and Responses

• Caller misuse: raft_log_append while not leader → -EPERM, no log change. NULL required callback at raft_init → -EINVAL.
• Persist failure: on_persist_entry non-zero return → propagated; the library does not increment log_len on a failed append (no phantom entry), preserving the persist-before-act guarantee.
• Truncate failure: on_truncate_from non-zero return on the conflict branch → reported as a non-success request result so the leader retries (the follower has not corrupted its log; it simply did not converge this round).
• Stale RPCs: a request with term < currentTerm is rejected (success=0, matchIndex=0); a reply with reply_term < currentTerm is ignored (DropStaleResponse, tla:415).
• Higher-term RPC: always adopted first (UpdateTerm) — never rejected for a log mismatch (R6.4).

Monitoring

Reuse the optional on_trace callback for new events: "ae_recv", "ae_reject_stale", "ae_reject_logmismatch", "ae_conflict_truncate", "ae_append", "term_adopt_ae", "ae_reply_advance", "ae_reply_backoff". NULL-checked exactly like existing trace calls.

Testing Strategy

TDD-mandatory (testing.md): tests and simulator drivers are written before the raft.c implementation (RED → GREEN). All tests are XCE_TESTING-gated and live in src/lib/testsuite/test_raft.c driving raft_sim.c.

Unit Tests (entry points, against in-memory sim log)

1. raft_log_append on a leader stamps currentTerm, assigns Len(log)+1, calls on_persist_entry once, returns the index (R2.1–2.4); on a follower returns -EPERM with no log change (R2.2).
2. raft_on_append_entries_request logOk: accepts when prevLogIndex==0; accepts when on_get_entry_term(prevLogIndex)==prevLogTerm; rejects (success=0, matchIndex=0) when terms differ (R4.1, R4.2 — G2).
3. Conflict branch: a follower with (index=k, term=A) receiving (index=k, term=B) calls on_truncate_from(k) then on_persist_entry(k, B) and reports success with matchIndex=k (R4.5, R4.6 — G3).
4. raft_on_append_entries_reply: success sets next_index=matchIndex+1, match_index=reply; failure decrements next_index flooring at 1; stale reply_term ignored (R5.1–5.3).
5. UpdateTerm on AppendEntries: a follower/leader at term T receiving an AE/heartbeat at term T+1 adopts T+1, clears vote, persists, and (if was leader) fires on_become_follower — applied before logOk, including a 0-entry heartbeat (R6.1–6.4 — G5).

Integration / Simulator Tests (Log Matching invariant)

1. Two-phase replication: leader appends 3 entries, replicates to a quorum; assert every node's prefix is byte-identical (Log Matching, R8.4).
2. Partition + heal: isolate a follower during several appends, then heal; assert AppendEntries walks nextIndex back, the follower catches up, and Log Matching holds (R8.1, R8.3).
3. Divergent-suffix truncation: hand a follower a manufactured divergent tail (same indices, different term), replicate from a higher-term leader; assert the follower truncates and converges to the leader's prefix (R8.5 — G3).
4. Chaotic churn / G5 regression: a stale-term node (term lagging) rejoins under a higher-term leader; assert it adopts the higher term from the AppendEntries/heartbeat path and does not re-elect at a reused term (R8.6 — closes the bug-851 G5 mechanism in the deterministic harness).

Message-fault Tests

1. Drop, duplicate, and reorder AppendEntries requests/replies in the sim queue; assert idempotent convergence and Log Matching after the faults clear (R8.3) — the already-have accept branch (R4.4) makes a duplicated AE a no-op.