myxce/.kiro/specs/raft-commit-safety/design.md

Design Document

Overview

Purpose: This feature adds the commit-safety and election-freshness rules of raft/raft.tla (Ongaro canonical) to the pure Raft library (src/lib/raft/raft.{c,h}), on top of the log half delivered by raft-log-replication. It closes the two remaining safety gaps of bug-851: G4 (term-blind commit advance) and G6 (LSN-only vote freshness).

Users: The XCE controller (as a Raft node, which will route its commit barrier through the new entry point in the downstream ha-term-safety integration), the deterministic simulator (raft_sim.c), and the downstream raft-sim-conformance spec (Leader Completeness oracle).

Impact: Today the commit position advances term-blind in the controller (ha_compute_safe_committed_lsn + the commit barrier) — it can mark a previous-term entry committed by replica-count alone (G4) — and vote freshness compares LSNs only, never the last-log term (G6). After this feature, the library owns commit_index and advances it exactly as raft.tla AdvanceCommitIndex does (quorum agreement and current-term entry), notifying the controller via on_commit_advanced to trigger the existing synchronous commit barrier (the barrier itself is unchanged and remains the controller's). Vote and PreVote freshness become the raft.tla (lastLogTerm, lastLogIndex) lexicographic comparison. The change is a pure, synchronous, deterministic addition to the core; all durability/I/O stays in adapters.

Goals

• Add a library-owned commit_index and raft_advance_commit(...) implementing raft.tla AdvanceCommitIndex (line 259) with the current-term rule log[Max(agreeIndexes)].term == currentTerm (line 270) — the Figure-8 safety rule (G4).
• Add the required callback on_commit_advanced(ctx, commit_index) that triggers the existing synchronous commit barrier; the library never performs the barrier itself.
• Upgrade vote freshness to the raft.tla HandleRequestVoteRequest logOk (lastLogTerm, lastLogIndex) comparison (line 285) on both the binding vote path and the PreVote eligibility path (G6).
• Extend raft_sim.c to assert the Figure-8 refusal and Leader Completeness across forced leader changes.

Non-Goals

• No log model / AppendEntries / per-peer progress — owned by raft-log-replication and reused verbatim (log_len, next_index[], match_index[], on_get_entry_term, on_persist_entry, on_send_append_entries, on_truncate_from, raft_log_append, raft_on_append_entries_request, raft_on_append_entries_reply).
• No commit barrier / durability mechanism — the synchronous barrier, fsync/group-commit, persist-before-ACK, reply gate, and snapshot transport stay in the controller (ha-term-safety adapter). This spec only triggers the barrier.
• No config-change markers / upgrade snapshot — owned by ha-term-safety.
• No conformance / TLC trace-check harness — owned by raft-sim-conformance.
• No controller wiring — replacing ha_compute_safe_committed_lsn's term-blind advance with the library's commit_index is ha-term-safety; this spec defines only the library entry point + callback it will route through.

Boundary Commitments

This Spec Owns

• The library commit_index field in raft_state_t and its restore field in raft_config_t (initial_commit_index, default 0).
• The new entry point raft_advance_commit(raft_state_t *s) implementing AdvanceCommitIndex with the current-term rule.
• The new required callback on_commit_advanced in raft_callbacks_t (fired only on strict advance) + its raft_init validation.
• The (lastLogTerm, lastLogIndex) freshness upgrade on the binding vote path (raft_on_vote_request) and the PreVote eligibility path (raft_on_pre_vote_request), including carrying last_log_term alongside last_log_index.
• Simulator extensions in raft_sim.c + tests in src/lib/testsuite/test_raft.c asserting Figure-8 refusal and Leader Completeness.

Out of Boundary

• The log model, AppendEntries, and per-peer match_index[] themselves (raft-log-replication) — consumed verbatim, never redefined.
• The synchronous commit barrier, fsync/group-commit, persist-before-ACK durability, reply gate, snapshot transport (controller / ha-term-safety).
• Config markers in the unified log + the change-boundary snapshot (ha-term-safety).
• The conformance / TLC trace-check harness (raft-sim-conformance).
• The controller's replacement of ha_compute_safe_committed_lsn with commit_index and the controller's freshness-floor plumbing of last_log_term onto the wire (ha-term-safety). This spec specifies only what the library entry point/callback route, not the controller mechanics.

Allowed Dependencies

• The implemented election core (raft-library) and the log half (raft-log-replication): raft_state_t, raft_callbacks_t, raft_init, the vote/PreVote path, log_len, next_index[], match_index[], on_get_entry_term. All reused; the vote path is extended additively.
• errno.h, stdint.h, string.h only — the core stays free-standing for the simulator.
• The simulator harness (raft_sim.c) for verification.
• Must NOT depend on any controller header, WAL type, capnp type, or transport — the core stays pure.

Revalidation Triggers

Dependent specs (raft-sim-conformance, ha-term-safety) must re-check integration if any of the following change:

• The signature or semantics of raft_advance_commit or the on_commit_advanced callback (especially the strict-advance / fire-once rule and "trigger only, never performs the barrier").
• The placement of the current-term rule (on_get_entry_term(Max(agreeIndexes)) == currentTerm) or the agreeIndexes/quorum definition.
• The freshness signature change — the addition of last_log_term to raft_on_vote_request, the PreVote eligibility path, and any vote-request payload / on_vote_eligible extension.
• The commit_index restore convention (initial_commit_index, default 0) or its monotonicity guarantee.

Architecture

Existing Architecture Analysis

After raft-log-replication, the library owns the log as pure, synchronous, deterministic state: per-entry term via on_get_entry_term, per-peer next_index[] / match_index[], and leader_commit carried opaquely (never acted on). The election core delegates freshness to on_vote_eligible, which today receives only (from_idx, term) — the caller checks config_tag / applied-LSN floor but has no last-log term. This feature follows the same discipline:

• Pure / synchronous / deterministic core — raft_advance_commit reads entry terms via on_get_entry_term and emits a single on_commit_advanced notification; it owns no entry bytes and performs no I/O. The freshness upgrade is a predicate computed from values already in the library log model plus the candidate's advertised (last_log_term, last_log_index).
• Persist-before-act ordering — unchanged on the vote path: a vote that passes the new (lastLogTerm, lastLogIndex) predicate is still persisted via on_persist before being granted. commit_index advance is not itself persisted by the library — durability of the committed position remains the controller barrier's job, triggered by on_commit_advanced.
• Status returns, never self-terminate — raft_advance_commit returns a status; no new path aborts.
• Separate actions — mirroring raft.tla, AdvanceCommitIndex is a distinct action from HandleAppendEntriesResponse. raft_on_append_entries_reply (owned upstream) only updates match_index[]; commit advance is the separate raft_advance_commit call. This keeps atomic regions small and lets a single-node leader commit (the {self} term in agreeIndexes).

Architecture Pattern & Boundary Map

flowchart TB
    subgraph PureCore["src/lib/raft/raft.c — PURE CORE (this spec)"]
        ADV["raft_advance_commit()<br/>AdvanceCommitIndex (tla:259)<br/>+ current-term rule (tla:270)"]
        VOTE["raft_on_vote_request()<br/>logOk (lastLogTerm,lastLogIndex) (tla:285) G6"]
        PVOTE["raft_on_pre_vote_request()<br/>same (lastLogTerm,lastLogIndex) freshness"]
    end
    subgraph Upstream["raft-log-replication — REUSED VERBATIM (not this spec)"]
        MI["match_index[] / log_len"]
        GET["on_get_entry_term(index) -> term|0"]
        REPLY["raft_on_append_entries_reply()<br/>updates match_index[] only"]
    end
    subgraph CBs["raft_callbacks_t — NEW seam (this spec)"]
        C5["on_commit_advanced(ctx, commit_index)<br/>fired ONLY on strict advance"]
    end
    subgraph Consumers["ADAPTERS — implement callback (NOT this spec's heavy wiring)"]
        CTRL["controller adapter (ha-term-safety):<br/>on_commit_advanced -> EXISTING sync commit barrier<br/>(persist-before-ACK preserved, NOT replaced)"]
        SIM["raft_sim.c: drive raft_advance_commit,<br/>assert Figure-8 + Leader Completeness (this spec)"]
    end
    REPLY -.updates.-> MI
    ADV --> MI & GET
    ADV --> C5
    VOTE --> GET
    PVOTE --> GET
    C5 --> CTRL
    C5 --> SIM
    ADV --> SIM

Architecture Integration:

• Selected pattern: Pure state machine + callback ports (Hexagonal/Ports-and-Adapters), already established by the election and log cores. Commit advance is a new pure action; on_commit_advanced is a new port adapters implement.
• Domain boundaries: the commit-safety decision (which index is safe to commit, given quorum + current-term) stays in the library; the durable act (the synchronous commit barrier, fsync, ACK) stays in the controller adapter, triggered by the port.
• Existing patterns preserved: raft_config_t extensibility, persist-before-act on the vote path, NULL-checked required callbacks, status-code returns, XCE_TESTING-gated simulator.
• Steering compliance: matches src/lib/CLAUDE.md (free-standing C, gnu17), ha-async-io.md (no blocking in the core; the barrier/async lives in the adapter), testing.md (TDD: simulator/test first), ha-async-io.md durability policy ("persist-before-ACK + synchronous commit barrier" preserved — the callback triggers it), "controller never self-terminates" (library returns statuses only).

Technology Stack

Layer	Choice / Version	Role in Feature	Notes
Library core	C (gnu17), errno.h/stdint.h/string.h only	Pure commit-safety + freshness	Free-standing; no controller/capnp deps
Callbacks	raft_callbacks_t function pointers	Commit-advance trigger seam	1 new required callback (on_commit_advanced)
Verification	raft_sim.c (XCE_TESTING) + test_raft.c	Figure-8 + Leader Completeness	Reuses the in-memory log backing from raft-log-replication
Adapter (downstream)	controller commit barrier (ha-term-safety)	Durable commit / ACK	Not built here; only triggered

File Structure Plan

Modified Files

• src/lib/raft/raft.h — Add uint64_t commit_index; to raft_state_t; add uint64_t initial_commit_index; to raft_config_t; add the on_commit_advanced callback to raft_callbacks_t; declare raft_advance_commit; extend the vote-path signatures to carry last_log_term (alongside the existing last_log_index) per the chosen freshness-plumbing option below; add a Figure-8 / Leader-Completeness driver + assertion helper to the simulator section.
• src/lib/raft/raft.c — Implement raft_advance_commit (agreeIndexes + current-term rule + monotonic strict-advance firing on_commit_advanced); add the on_commit_advanced NULL check + initial_commit_index restore in raft_init; upgrade the freshness predicate in raft_on_vote_request and raft_on_pre_vote_request to the (lastLogTerm, lastLogIndex) comparison.
• src/lib/raft/raft_sim.c — Drive raft_advance_commit after delivery; the Figure-8 construction; the Leader-Completeness assertion across forced leader changes; the freshness-gate exercise.
• src/lib/testsuite/test_raft.c — New tests for raft_advance_commit (current-term hold/advance, monotonicity, non-leader), on_commit_advanced fire-once/never semantics, the (lastLogTerm, lastLogIndex) vote/PreVote freshness, and the simulator invariants (TDD-first).

Directory Structure (unchanged)

src/lib/raft/
├── raft.h          # + commit_index, on_commit_advanced, raft_advance_commit, last_log_term on vote path
├── raft.c          # + AdvanceCommitIndex(current-term rule), (lastLogTerm,lastLogIndex) freshness
└── raft_sim.c      # + commit-advance driver, Figure-8, Leader Completeness assert
src/lib/testsuite/
└── test_raft.c     # + commit-safety + freshness + Figure-8/Leader-Completeness tests

Each file keeps one responsibility: raft.c = pure protocol, raft_sim.c = deterministic harness, test_raft.c = assertions. No new file is introduced; the existing module pattern is extended.

System Flows

Commit advance: separate action after replication (raft.tla AdvanceCommitIndex)

sequenceDiagram
    participant L as Leader (raft.c)
    participant LC as Leader adapter (cb)
    Note over L: replication already updated match_index[]<br/>via raft_on_append_entries_reply (raft-log-replication)
    Note over L: raft_advance_commit() — separate action (tla:259)
    L->>L: agreeIndexes = { i in 1..log_len :<br/>{self} ∪ {p: match_index[p]>=i} ∈ Quorum }
    alt agreeIndexes empty
        Note over L: hold commit_index (else-branch)
    else Max(agreeIndexes) from a PREVIOUS term
        L->>LC: on_get_entry_term(Max) != currentTerm
        Note over L: hold commit_index (Figure-8 refusal, G4)
    else Max(agreeIndexes) from CURRENT term
        L->>LC: on_get_entry_term(Max) == currentTerm
        Note over L: commit_index = Max(agreeIndexes) (strict advance)
        L->>LC: on_commit_advanced(ctx, commit_index)
        Note over LC: trigger EXISTING sync commit barrier<br/>(persist-before-ACK, NOT replaced)
    end

Vote freshness: (lastLogTerm, lastLogIndex) comparison (raft.tla HandleRequestVoteRequest logOk)

sequenceDiagram
    participant V as Voter (raft.c)
    participant VC as Voter adapter (cb)
    Note over V: raft_on_vote_request(..., cand_last_log_term, cand_last_log_index)
    Note over V: UpdateTerm / stale-term / one-vote-per-term (unchanged)
    V->>VC: on_get_entry_term(log_len) -> local_last_log_term (0 if empty)
    Note over V: logOk = cand_last_log_term > local_last_log_term<br/>∨ (== ∧ cand_last_log_index >= log_len)
    alt !logOk
        Note over V: DENY (even if all other gates would grant) — G6
    else logOk AND application gate AND invariants pass
        V->>VC: on_persist(term, from_idx)  %% persist-before-grant unchanged
        Note over V: GRANT
    end

Flow-level decisions:

• Commit is a separate action — match_index[] is updated by raft_on_append_entries_reply (upstream); the commit index advances only when raft_advance_commit is driven, exactly as raft.tla separates the two actions (R2.6). This lets a single-node leader commit ({self} ∈ agreeIndexes).
• Current-term rule is the Figure-8 gate — commit_index advances to Max(agreeIndexes) only when that entry is from currentTerm; a previous-term entry replicated to a quorum is held until a current-term entry above it commits, then committed indirectly (R1.3/R1.4, closes G4).
• on_commit_advanced fires only on strict advance — never on a held commit index, never from raft_on_append_entries_reply (R2.2/R2.3/R2.6).
• Freshness denies before any grant — a stale (lastLogTerm, lastLogIndex) denies even when the term/vote invariants and the application gate would grant (R3.4, closes G6); persist-before-grant is unchanged (R3.6).

Freshness Plumbing — design decision

raft.tla logOk needs the candidate's (lastLogTerm, lastLogIndex) and the voter's own. Two options were considered for carrying the candidate values into the library:

• Option A — extend the on_vote_eligible callback signature to on_vote_eligible(ctx, from_idx, term, last_log_term, last_log_index) and have the library compute logOk against on_get_entry_term(log_len)/log_len before calling the gate. Rejected as the sole mechanism: it pushes the protocol-defining logOk either into the adapter (violating "library owns protocol decisions") or splits it.
• Option B (chosen) — the library owns logOk. The candidate's last_log_term and last_log_index are accepted as explicit arguments on raft_on_vote_request (and on the PreVote eligibility path), the library computes logOk against its own on_get_entry_term(log_len) / log_len, and only then consults the unchanged application gate on_vote_eligible for XCE-specific policy (primary-deny / heartbeat stickiness / catch-up). This keeps the raft.tla logOk predicate inside the library (R3.1, R5.5) while leaving on_vote_eligible for application policy that is genuinely outside Raft.

The candidate advertises its (last_log_term, last_log_index) in the RequestVote / PreVote wire RPC; the controller adapter decodes them and passes them to the library entry point. The on-wire encoding of last_log_term (and the controller's freshness-floor plumbing) is the ha-term-safety adapter's job — out of this spec's scope; this spec fixes only the library signatures and the predicate.

Note — freshness-decision locus moves from caller/dispatcher into the library (both paths). Today the freshness decision lives outside the library: raft.h (lines 246–248) documents that PreVote freshness "is applied by the dispatcher before this call", and the binding vote path delegates the decision to on_vote_eligible. Option B moves that decision locus into the library for both the binding (raft_on_vote_request) and the PreVote (raft_on_pre_vote_request) paths: raft.c now computes the (lastLogTerm, lastLogIndex) logOk itself rather than relying on the caller/dispatcher. This supersedes the existing raft.h lines 246–248 doc contract, and updating those raft.h doc comments (so the header no longer states the dispatcher owns PreVote freshness) is part of the header-contract task.

Requirements Traceability

Requirement	Summary	Design Elements	raft.tla action
1.1–1.7	Library commit_index, agreeIndexes, current-term rule, monotonic, leader-only	raft_state_t.commit_index; raft_advance_commit(); on_get_entry_term; match_index[] (upstream)	AdvanceCommitIndex (259), current-term rule (270)
2.1–2.6	on_commit_advanced trigger, fire-once, separate from reply	raft_callbacks_t.on_commit_advanced; strict-advance firing; raft_init validation	(seam; AdvanceCommitIndex distinct from HandleAppendEntriesResponse)
3.1–3.6	(lastLogTerm,lastLogIndex) binding-vote freshness	raft_on_vote_request logOk; on_get_entry_term(log_len)/log_len; persist-before-grant	HandleRequestVoteRequest logOk (285)
4.1–4.5	Same freshness on PreVote, side-effect-free	raft_on_pre_vote_request logOk; no persist/term/role mutation	HandleRequestVoteRequest logOk (285), PreVote extension
5.1–5.4	Barrier/durability preserved as policy; PreVote/priority unchanged; no abort	on_commit_advanced as trigger only; additive-only changes	(policy; not a TLA+ action)
6.1–6.6	Simulator Figure-8 + Leader Completeness + freshness gate	raft_sim.c commit driver, Figure-8 construction, Leader-Completeness assert, freshness exercise	Figure-8 / Leader Completeness invariant

Components and Interfaces

Component	Layer	Intent	Req Coverage	Key Dependencies	Contracts
commit_index (raft_state_t)	Library core	Library-owned commit position	1	log core (P0)	State
raft_advance_commit	Library core	AdvanceCommitIndex + current-term rule	1, 2	on_get_entry_term, match_index[], on_commit_advanced (P0)	Service
on_commit_advanced callback	Seam	Commit-advance trigger	2, 5	controller barrier / sim (P0)	Service
raft_on_vote_request freshness	Library core	(lastLogTerm,lastLogIndex) logOk	3, 5	on_get_entry_term, log_len, on_persist, on_vote_eligible (P0)	Service
raft_on_pre_vote_request freshness	Library core	Same logOk, side-effect-free	4, 5	on_get_entry_term, log_len, on_vote_eligible (P0)	Service
Simulator	Verification	Figure-8 + Leader Completeness	6	all of the above (P0)	State

Library Core

commit_index addition to raft_state_t

Field	Detail
Intent	Hold the library's view of the safe commit position (TLA+ commitIndex)
Requirements	1.1, 1.5

Responsibilities & Constraints

• uint64_t commit_index; — the highest 1-based index (== WAL LSN) the library considers safely committed. Monotonic non-decreasing.
• Restored at init from the adapter (raft_config_t.initial_commit_index, default 0; the library never reads storage — consistent with initial_term / initial_log_len).
• The library never advances commit_index except through raft_advance_commit; raft_on_append_entries_reply (upstream) leaves it untouched (R2.6).

raft_advance_commit — AdvanceCommitIndex (raft.tla:259) with the current-term rule (line 270)

Field	Detail
Intent	Advance commit_index to Max(agreeIndexes) only when that entry is from currentTerm
Requirements	1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 2.2, 2.3

Service Interface

/*
 * raft_advance_commit: leader recomputes its commit index (raft.tla
 * AdvanceCommitIndex, 259). A SEPARATE action from append-entries reply
 * handling (mirrors raft.tla, so a single-node leader can commit and to
 * keep atomic regions small).
 *
 * Computes agreeIndexes = { index in 1..log_len :
 *   ({self} ∪ {peer : match_index[peer] >= index}) is a quorum }.
 * If agreeIndexes is non-empty AND on_get_entry_term(Max(agreeIndexes))
 *   == s->term, sets commit_index = Max(agreeIndexes) (Figure-8 current-term
 *   rule) and, on a STRICT advance, fires on_commit_advanced(ctx, commit_index)
 *   exactly once.
 * Otherwise leaves commit_index unchanged and does NOT fire on_commit_advanced.
 *
 * The library performs NO durability work here — on_commit_advanced only
 * TRIGGERS the controller's existing synchronous commit barrier.
 *
 * Returns RAFT_OK on success (whether or not the index advanced);
 * -EPERM if role != RAFT_LEADER (commit_index left unchanged).
 */
int raft_advance_commit(raft_state_t *s);

• Preconditions: s->role == RAFT_LEADER (else -EPERM, no change — R1.6).
• Postconditions: commit_index either unchanged or advanced to Max(agreeIndexes) when that entry is current-term; on_commit_advanced fired iff commit_index strictly increased.
• Invariants: commit_index never decreases (R1.5); the library reads entry terms only via on_get_entry_term (R1.7); commit_index ≤ log_len.

Quorum: (n_peers / 2) + 1, identical to the election-core quorum; match_index[self] is implicitly log_len (self always agrees up to its own log length), realized by seeding the agree-count with self for each candidate index. agreeIndexes is computed by scanning candidate indexes from log_len downward and taking the first that reaches quorum (equivalent to Max(agreeIndexes) without materializing the set).

Freshness upgrade — raft_on_vote_request (raft.tla:285, closes G6)

Field	Detail
Intent	Add the (lastLogTerm, lastLogIndex) lexicographic logOk before the application gate
Requirements	3.1, 3.2, 3.3, 3.4, 3.5, 3.6

Service Interface

/*
 * raft_on_vote_request: process an incoming RequestVote RPC (extended).
 * Adds the raft.tla HandleRequestVoteRequest logOk (285): the library
 * computes
 *   logOk = (cand_last_log_term > LastTerm(local))
 *           || (cand_last_log_term == LastTerm(local)
 *               && cand_last_log_index >= log_len)
 * where LastTerm(local) = (log_len > 0) ? on_get_entry_term(log_len) : 0.
 * Step order (unchanged except the new logOk gate):
 *   1. stale-term deny / UpdateTerm (unchanged)
 *   2. role gate, one-vote-per-term (unchanged)
 *   3. logOk gate (NEW, G6): deny if !logOk, before on_vote_eligible
 *   4. application gate on_vote_eligible (unchanged)
 *   5. on_persist before grant (unchanged)
 * Returns RAFT_GRANT / RAFT_DENY; sets *reply_term.
 */
int raft_on_vote_request(raft_state_t *s, int from_idx, uint64_t term,
                         uint64_t cand_last_log_term,
                         uint64_t cand_last_log_index,
                         uint64_t *reply_term);

• Preconditions: none beyond a valid s.
• Postconditions: a vote is granted only when the existing invariants, the new (lastLogTerm, lastLogIndex) logOk, and the application gate all pass; persisted before grant.
• Invariants: logOk is evaluated before on_vote_eligible; a !logOk denies regardless of application policy (R3.4); no weakening of stale-term / one-vote-per-term (R3.5).

Signature change is a Revalidation Trigger. raft_on_vote_request gains cand_last_log_term and cand_last_log_index. The controller and the simulator (the two callers) must pass the candidate's advertised last-log (term, index). The PreVote path mirrors this.

Freshness upgrade — raft_on_pre_vote_request (side-effect-free)

Field	Detail
Intent	Apply the identical (lastLogTerm, lastLogIndex) logOk on the straw poll, mutating nothing
Requirements	4.1, 4.2, 4.3, 4.4, 4.5

Service Interface

/*
 * raft_on_pre_vote_request: answer a PreVote probe (extended).
 * Applies the SAME (lastLogTerm, lastLogIndex) logOk as the binding vote
 * (Requirement 3.1), in addition to the existing stale-term deny and the
 * on_vote_eligible gate. Mutates NOTHING: no term/voted_for/role/commit_index
 * change, no on_persist (PreVote invariant preserved).
 * Returns RAFT_GRANT / RAFT_DENY.
 */
int raft_on_pre_vote_request(raft_state_t *s, int from_idx, uint64_t cand_term,
                             uint64_t cand_last_log_term,
                             uint64_t cand_last_log_index);

• Preconditions: none beyond a valid s.
• Postconditions: grant only when stale-term, logOk, and the application gate all pass; no persistent state touched.
• Invariants: identical freshness rule to the binding path (R4.1, R4.2); side-effect-free (R4.4).

Seam — raft_callbacks_t addition (verbatim Authoritative shared contract)

Appended to raft_callbacks_t and validated required in raft_init. Reproduced verbatim from the brief so downstream specs depend on the exact shape:

/* REQUIRED: fired AFTER commit_index strictly advances. The controller adapter
 * feeds this into the EXISTING synchronous commit barrier / persist-before-ACK
 * durability — this callback TRIGGERS commit-eval; it does NOT replace the
 * barrier. */
void on_commit_advanced(void *ctx, uint64_t commit_index);

Contract rules (R2.4, R5.1): the library decides whether the commit index advanced; the callback only triggers the adapter's commit work. It is fired exactly once per strict advance (R2.2), never on a held commit index (R2.3), and never from raft_on_append_entries_reply (R2.6). It is NULL-checked and required in raft_init (-EINVAL if NULL — R2.5), matching how the other required callbacks are validated.

Controller adapter (seam only — heavy wiring is ha-term-safety)

This spec describes the adapter only at the seam; the full controller integration is the downstream ha-term-safety consumer.

• on_commit_advanced → the controller routes the new safe commit_index into its existing synchronous commit barrier (commit-eval sweep, persist-before-ACK, reply gate) — replacing the term-blind ha_compute_safe_committed_lsn advance with the library's current-term-gated value. The barrier mechanism is unchanged; only its input (the safe commit position) now comes from the library.
• The candidate's advertised (last_log_term, last_log_index) is decoded from the RequestVote / PreVote wire RPC and passed to raft_on_vote_request / raft_on_pre_vote_request; last_log_term is read back from the WAL term (on_get_entry_term-backed index) on recovery. The on-wire encoding and the freshness-floor plumbing are ha-term-safety.
• On recovery, the controller restores commit_index (= recovered committed LSN) into raft_config_t.initial_commit_index.

Data Models

Domain Model

• Commit position (logical): commit_index : uint64 — the highest 1-based index (== WAL LSN) safely committed, owned by the library, monotonic.
  • Convention: callers pass s->commit_index as the leader_commit argument to raft_send_append_entries — the leader's committed value rides AppendEntries from commit_index, which this spec owns (upstream raft-log-replication carries leader_commit opaquely; the value it carries is the commit_index defined here).
• Freshness tuple: (last_log_term : uint64, last_log_index : uint64) per candidate, compared lexicographically against the voter's own (on_get_entry_term(log_len), log_len).

Logical Data Model

• raft_state_t gains commit_index.
• raft_config_t gains initial_commit_index (restored committed LSN at boot; 0 on first boot — the library never reads storage, consistent with initial_term / initial_log_len).
• No new persisted on-disk format: commit_index durability is the controller barrier's existing responsibility; the library imposes no new persisted layout. last_log_term reuses the WAL hdr.term already written by raft-log-replication's adapter.

Data Contracts & Integration

• Freshness tuple: raft_on_vote_request and raft_on_pre_vote_request accept (cand_last_log_term, cand_last_log_index) as explicit arguments. The voter's own values come from on_get_entry_term(log_len) (term 0 when log_len == 0, consistent with raft.tla LastTerm) and log_len.
• Commit trigger: on_commit_advanced(ctx, commit_index) carries only the new committed index; no entry bytes, no term — the adapter looks up whatever it needs through its own state.

Error Handling

Error Strategy

The library returns status codes only and never aborts (preserves "controller never self-terminates"). Callbacks for commit-advance are trigger-only and return void; the freshness path keeps the existing persist-failure handling.

Error Categories and Responses

• Caller misuse: raft_advance_commit while not leader → -EPERM, commit_index unchanged, no on_commit_advanced. NULL on_commit_advanced at raft_init → -EINVAL.
• Held commit (Figure-8): when Max(agreeIndexes) is a previous-term entry, commit_index is held — this is the correct safety behavior, not an error; no callback fires.
• Empty agreeIndexes: no quorum agrees on any index → hold, no callback.
• Stale freshness: !logOk → vote/PreVote DENY (not an error; the standard freshness rejection).
• Persist failure on grant: unchanged from the existing vote path — a failed on_persist denies the vote without recording it.

Monitoring

Reuse the optional on_trace callback for new events: "commit_advance", "commit_hold_prevterm", "commit_hold_noquorum", "vote_denied_log_term" (stale lastLogTerm), "vote_denied_log_index" (tie term, stale index), "pre_vote_denied_freshness". NULL-checked exactly like existing trace calls.

Testing Strategy

TDD-mandatory (testing.md): tests and simulator drivers are written before the raft.c implementation (RED → GREEN). All tests are XCE_TESTING-gated and live in src/lib/testsuite/test_raft.c driving raft_sim.c. They reuse the in-memory per-node log backing added by raft-log-replication.

Unit Tests (entry points, against the in-memory sim log)

1. raft_advance_commit current-term advance: a leader whose match_index[] shows a quorum agreeing through index k, with the entry at k from currentTerm, advances commit_index to k and fires on_commit_advanced(k) once (R1.2, R1.3, R2.2).
2. raft_advance_commit Figure-8 hold: a leader with a quorum agreeing through a previous-term entry at k (no current-term entry above it) holds commit_index and fires nothing (R1.4, R2.3 — G4).
3. raft_advance_commit monotonicity + non-leader: never decreases commit_index; returns -EPERM with no change and no callback when not leader (R1.5, R1.6).
4. on_commit_advanced separation: raft_on_append_entries_reply updating match_index[] does not by itself advance commit_index or fire on_commit_advanced; only a subsequent raft_advance_commit does (R2.6).
5. raft_init requires on_commit_advanced: -EINVAL when NULL; restores initial_commit_index (R2.5, R1.1).
6. (lastLogTerm, lastLogIndex) vote freshness: a candidate with a stale last_log_term is denied even with a greater last_log_index; a candidate with a fresher last_log_term is granted with a smaller last_log_index; a tie on term grants only when last_log_index >= log_len (R3.1, R3.4). PreVote applies the identical rule with no persist/term/role mutation (R4.1, R4.4).

Integration / Simulator Tests (Figure-8, Leader Completeness)

1. Figure-8 scenario: construct a previous-term entry replicated to a quorum but not yet covered by a committed current-term entry; drive raft_advance_commit and assert commit_index does not move over it; then replicate a current-term entry above it to a quorum, drive raft_advance_commit, and assert commit_index advances over both (indirect commit) (R6.2, R6.3 — G4).
2. Leader Completeness: across forced leader changes (timeout / partition-driven), assert that any entry committed (≤ commit_index on the committing leader) is present in the log of every node that subsequently becomes leader (R6.4) — the (lastLogTerm, lastLogIndex) freshness gate (R6.5) is what makes this hold, so the two tests are complementary.
3. Freshness gate exercise: drive votes with mismatched (last_log_term, last_log_index) tuples and assert the lexicographic outcome (stale-term-but-higher-index denied; fresher-term-but-lower-index granted) (R6.5).
4. Trigger semantics: assert on_commit_advanced fires exactly on strict advance and not while the current-term rule holds the commit index (R6.6).