# Design Document ## Overview **Purpose**: This feature adds the **commit-safety and election-freshness** rules of `raft/raft.tla` (Ongaro canonical) to the pure Raft library (`src/lib/raft/raft.{c,h}`), on top of the log half delivered by `raft-log-replication`. It closes the two remaining safety gaps of bug-851: **G4** (term-blind commit advance) and **G6** (LSN-only vote freshness). **Users**: The XCE controller (as a Raft node, which will route its commit barrier through the new entry point in the downstream `ha-term-safety` integration), the deterministic simulator (`raft_sim.c`), and the downstream `raft-sim-conformance` spec (Leader Completeness oracle). **Impact**: Today the commit position advances **term-blind** in the controller (`ha_compute_safe_committed_lsn` + the commit barrier) — it can mark a previous-term entry committed by replica-count alone (G4) — and vote freshness compares LSNs only, never the last-log term (G6). After this feature, the library owns `commit_index` and advances it exactly as `raft.tla` `AdvanceCommitIndex` does (quorum agreement **and** current-term entry), notifying the controller via `on_commit_advanced` to **trigger** the existing synchronous commit barrier (the barrier itself is unchanged and remains the controller's). Vote and PreVote freshness become the `raft.tla` `(lastLogTerm, lastLogIndex)` lexicographic comparison. The change is a pure, synchronous, deterministic addition to the core; all durability/I/O stays in adapters. ### Goals - Add a library-owned `commit_index` and `raft_advance_commit(...)` implementing `raft.tla` `AdvanceCommitIndex` (line 259) **with** the current-term rule `log[Max(agreeIndexes)].term == currentTerm` (line 270) — the Figure-8 safety rule (G4). - Add the required callback `on_commit_advanced(ctx, commit_index)` that **triggers** the existing synchronous commit barrier; the library never performs the barrier itself. - Upgrade vote freshness to the `raft.tla` `HandleRequestVoteRequest` `logOk` `(lastLogTerm, lastLogIndex)` comparison (line 285) on both the binding vote path and the PreVote eligibility path (G6). - Extend `raft_sim.c` to assert the **Figure-8** refusal and **Leader Completeness** across forced leader changes. ### Non-Goals - **No log model / AppendEntries / per-peer progress** — owned by `raft-log-replication` and reused verbatim (`log_len`, `next_index[]`, `match_index[]`, `on_get_entry_term`, `on_persist_entry`, `on_send_append_entries`, `on_truncate_from`, `raft_log_append`, `raft_on_append_entries_request`, `raft_on_append_entries_reply`). - **No commit barrier / durability mechanism** — the synchronous barrier, fsync/group-commit, persist-before-ACK, reply gate, and snapshot transport stay in the controller (`ha-term-safety` adapter). This spec only triggers the barrier. - **No config-change markers / upgrade snapshot** — owned by `ha-term-safety`. - **No conformance / TLC trace-check harness** — owned by `raft-sim-conformance`. - **No controller wiring** — replacing `ha_compute_safe_committed_lsn`'s term-blind advance with the library's `commit_index` is `ha-term-safety`; this spec defines only the library entry point + callback it will route through. ## Boundary Commitments ### This Spec Owns - The library `commit_index` field in `raft_state_t` and its restore field in `raft_config_t` (`initial_commit_index`, default 0). - The new entry point `raft_advance_commit(raft_state_t *s)` implementing `AdvanceCommitIndex` with the current-term rule. - The new required callback `on_commit_advanced` in `raft_callbacks_t` (fired only on strict advance) + its `raft_init` validation. - The `(lastLogTerm, lastLogIndex)` freshness upgrade on the binding vote path (`raft_on_vote_request`) and the PreVote eligibility path (`raft_on_pre_vote_request`), including carrying `last_log_term` alongside `last_log_index`. - Simulator extensions in `raft_sim.c` + tests in `src/lib/testsuite/test_raft.c` asserting Figure-8 refusal and Leader Completeness. ### Out of Boundary - The log model, AppendEntries, and per-peer `match_index[]` themselves (`raft-log-replication`) — consumed verbatim, never redefined. - The synchronous commit barrier, fsync/group-commit, persist-before-ACK durability, reply gate, snapshot transport (controller / `ha-term-safety`). - Config markers in the unified log + the change-boundary snapshot (`ha-term-safety`). - The conformance / TLC trace-check harness (`raft-sim-conformance`). - The controller's replacement of `ha_compute_safe_committed_lsn` with `commit_index` and the controller's freshness-floor plumbing of `last_log_term` onto the wire (`ha-term-safety`). This spec specifies only **what the library entry point/callback route**, not the controller mechanics. ### Allowed Dependencies - The implemented election core (`raft-library`) and the log half (`raft-log-replication`): `raft_state_t`, `raft_callbacks_t`, `raft_init`, the vote/PreVote path, `log_len`, `next_index[]`, `match_index[]`, `on_get_entry_term`. All reused; the vote path is extended additively. - `errno.h`, `stdint.h`, `string.h` only — the core stays free-standing for the simulator. - The simulator harness (`raft_sim.c`) for verification. - Must NOT depend on any controller header, WAL type, capnp type, or transport — the core stays pure. ### Revalidation Triggers Dependent specs (`raft-sim-conformance`, `ha-term-safety`) must re-check integration if any of the following change: - The signature or semantics of `raft_advance_commit` or the `on_commit_advanced` callback (especially the strict-advance / fire-once rule and "trigger only, never performs the barrier"). - The placement of the current-term rule (`on_get_entry_term(Max(agreeIndexes)) == currentTerm`) or the `agreeIndexes`/quorum definition. - The freshness signature change — the addition of `last_log_term` to `raft_on_vote_request`, the PreVote eligibility path, and any vote-request payload / `on_vote_eligible` extension. - The `commit_index` restore convention (`initial_commit_index`, default 0) or its monotonicity guarantee. ## Architecture ### Existing Architecture Analysis After `raft-log-replication`, the library owns the log as pure, synchronous, deterministic state: per-entry term via `on_get_entry_term`, per-peer `next_index[]` / `match_index[]`, and `leader_commit` carried **opaquely** (never acted on). The election core delegates freshness to `on_vote_eligible`, which today receives only `(from_idx, term)` — the caller checks `config_tag` / applied-LSN floor but has no last-log **term**. This feature follows the same discipline: - **Pure / synchronous / deterministic core** — `raft_advance_commit` reads entry terms via `on_get_entry_term` and emits a single `on_commit_advanced` notification; it owns no entry bytes and performs no I/O. The freshness upgrade is a predicate computed from values already in the library log model plus the candidate's advertised `(last_log_term, last_log_index)`. - **Persist-before-act ordering** — unchanged on the vote path: a vote that passes the new `(lastLogTerm, lastLogIndex)` predicate is still persisted via `on_persist` before being granted. `commit_index` advance is not itself persisted by the library — durability of the committed position remains the controller barrier's job, triggered by `on_commit_advanced`. - **Status returns, never self-terminate** — `raft_advance_commit` returns a status; no new path aborts. - **Separate actions** — mirroring `raft.tla`, `AdvanceCommitIndex` is a **distinct** action from `HandleAppendEntriesResponse`. `raft_on_append_entries_reply` (owned upstream) only updates `match_index[]`; commit advance is the separate `raft_advance_commit` call. This keeps atomic regions small and lets a single-node leader commit (the `{self}` term in `agreeIndexes`). ### Architecture Pattern & Boundary Map ```mermaid flowchart TB subgraph PureCore["src/lib/raft/raft.c — PURE CORE (this spec)"] ADV["raft_advance_commit()
AdvanceCommitIndex (tla:259)
+ current-term rule (tla:270)"] VOTE["raft_on_vote_request()
logOk (lastLogTerm,lastLogIndex) (tla:285) G6"] PVOTE["raft_on_pre_vote_request()
same (lastLogTerm,lastLogIndex) freshness"] end subgraph Upstream["raft-log-replication — REUSED VERBATIM (not this spec)"] MI["match_index[] / log_len"] GET["on_get_entry_term(index) -> term|0"] REPLY["raft_on_append_entries_reply()
updates match_index[] only"] end subgraph CBs["raft_callbacks_t — NEW seam (this spec)"] C5["on_commit_advanced(ctx, commit_index)
fired ONLY on strict advance"] end subgraph Consumers["ADAPTERS — implement callback (NOT this spec's heavy wiring)"] CTRL["controller adapter (ha-term-safety):
on_commit_advanced -> EXISTING sync commit barrier
(persist-before-ACK preserved, NOT replaced)"] SIM["raft_sim.c: drive raft_advance_commit,
assert Figure-8 + Leader Completeness (this spec)"] end REPLY -.updates.-> MI ADV --> MI & GET ADV --> C5 VOTE --> GET PVOTE --> GET C5 --> CTRL C5 --> SIM ADV --> SIM ``` **Architecture Integration**: - **Selected pattern**: Pure state machine + callback ports (Hexagonal/Ports-and-Adapters), already established by the election and log cores. Commit advance is a new pure action; `on_commit_advanced` is a new port adapters implement. - **Domain boundaries**: the commit-safety **decision** (which index is safe to commit, given quorum + current-term) stays in the library; the durable **act** (the synchronous commit barrier, fsync, ACK) stays in the controller adapter, triggered by the port. - **Existing patterns preserved**: `raft_config_t` extensibility, persist-before-act on the vote path, NULL-checked required callbacks, status-code returns, `XCE_TESTING`-gated simulator. - **Steering compliance**: matches `src/lib/CLAUDE.md` (free-standing C, gnu17), `ha-async-io.md` (no blocking in the core; the barrier/async lives in the adapter), `testing.md` (TDD: simulator/test first), `ha-async-io.md` durability policy ("persist-before-ACK + synchronous commit barrier" preserved — the callback triggers it), "controller never self-terminates" (library returns statuses only). ### Technology Stack | Layer | Choice / Version | Role in Feature | Notes | |-------|------------------|-----------------|-------| | Library core | C (gnu17), `errno.h`/`stdint.h`/`string.h` only | Pure commit-safety + freshness | Free-standing; no controller/capnp deps | | Callbacks | `raft_callbacks_t` function pointers | Commit-advance trigger seam | 1 new required callback (`on_commit_advanced`) | | Verification | `raft_sim.c` (XCE_TESTING) + `test_raft.c` | Figure-8 + Leader Completeness | Reuses the in-memory log backing from `raft-log-replication` | | Adapter (downstream) | controller commit barrier (`ha-term-safety`) | Durable commit / ACK | Not built here; only triggered | ## File Structure Plan ### Modified Files - `src/lib/raft/raft.h` — Add `uint64_t commit_index;` to `raft_state_t`; add `uint64_t initial_commit_index;` to `raft_config_t`; add the `on_commit_advanced` callback to `raft_callbacks_t`; declare `raft_advance_commit`; extend the vote-path signatures to carry `last_log_term` (alongside the existing `last_log_index`) per the chosen freshness-plumbing option below; add a Figure-8 / Leader-Completeness driver + assertion helper to the simulator section. - `src/lib/raft/raft.c` — Implement `raft_advance_commit` (agreeIndexes + current-term rule + monotonic strict-advance firing `on_commit_advanced`); add the `on_commit_advanced` NULL check + `initial_commit_index` restore in `raft_init`; upgrade the freshness predicate in `raft_on_vote_request` and `raft_on_pre_vote_request` to the `(lastLogTerm, lastLogIndex)` comparison. - `src/lib/raft/raft_sim.c` — Drive `raft_advance_commit` after delivery; the Figure-8 construction; the Leader-Completeness assertion across forced leader changes; the freshness-gate exercise. - `src/lib/testsuite/test_raft.c` — New tests for `raft_advance_commit` (current-term hold/advance, monotonicity, non-leader), `on_commit_advanced` fire-once/never semantics, the `(lastLogTerm, lastLogIndex)` vote/PreVote freshness, and the simulator invariants (TDD-first). ### Directory Structure (unchanged) ``` src/lib/raft/ ├── raft.h # + commit_index, on_commit_advanced, raft_advance_commit, last_log_term on vote path ├── raft.c # + AdvanceCommitIndex(current-term rule), (lastLogTerm,lastLogIndex) freshness └── raft_sim.c # + commit-advance driver, Figure-8, Leader Completeness assert src/lib/testsuite/ └── test_raft.c # + commit-safety + freshness + Figure-8/Leader-Completeness tests ``` > Each file keeps one responsibility: `raft.c` = pure protocol, `raft_sim.c` = deterministic harness, `test_raft.c` = assertions. No new file is introduced; the existing module pattern is extended. ## System Flows ### Commit advance: separate action after replication (raft.tla AdvanceCommitIndex) ```mermaid sequenceDiagram participant L as Leader (raft.c) participant LC as Leader adapter (cb) Note over L: replication already updated match_index[]
via raft_on_append_entries_reply (raft-log-replication) Note over L: raft_advance_commit() — separate action (tla:259) L->>L: agreeIndexes = { i in 1..log_len :
{self} ∪ {p: match_index[p]>=i} ∈ Quorum } alt agreeIndexes empty Note over L: hold commit_index (else-branch) else Max(agreeIndexes) from a PREVIOUS term L->>LC: on_get_entry_term(Max) != currentTerm Note over L: hold commit_index (Figure-8 refusal, G4) else Max(agreeIndexes) from CURRENT term L->>LC: on_get_entry_term(Max) == currentTerm Note over L: commit_index = Max(agreeIndexes) (strict advance) L->>LC: on_commit_advanced(ctx, commit_index) Note over LC: trigger EXISTING sync commit barrier
(persist-before-ACK, NOT replaced) end ``` ### Vote freshness: (lastLogTerm, lastLogIndex) comparison (raft.tla HandleRequestVoteRequest logOk) ```mermaid sequenceDiagram participant V as Voter (raft.c) participant VC as Voter adapter (cb) Note over V: raft_on_vote_request(..., cand_last_log_term, cand_last_log_index) Note over V: UpdateTerm / stale-term / one-vote-per-term (unchanged) V->>VC: on_get_entry_term(log_len) -> local_last_log_term (0 if empty) Note over V: logOk = cand_last_log_term > local_last_log_term
∨ (== ∧ cand_last_log_index >= log_len) alt !logOk Note over V: DENY (even if all other gates would grant) — G6 else logOk AND application gate AND invariants pass V->>VC: on_persist(term, from_idx) %% persist-before-grant unchanged Note over V: GRANT end ``` Flow-level decisions: - **Commit is a separate action** — `match_index[]` is updated by `raft_on_append_entries_reply` (upstream); the commit index advances only when `raft_advance_commit` is driven, exactly as `raft.tla` separates the two actions (R2.6). This lets a single-node leader commit (`{self}` ∈ `agreeIndexes`). - **Current-term rule is the Figure-8 gate** — `commit_index` advances to `Max(agreeIndexes)` only when that entry is from `currentTerm`; a previous-term entry replicated to a quorum is **held** until a current-term entry above it commits, then committed indirectly (R1.3/R1.4, closes G4). - **`on_commit_advanced` fires only on strict advance** — never on a held commit index, never from `raft_on_append_entries_reply` (R2.2/R2.3/R2.6). - **Freshness denies before any grant** — a stale `(lastLogTerm, lastLogIndex)` denies even when the term/vote invariants and the application gate would grant (R3.4, closes G6); persist-before-grant is unchanged (R3.6). ## Freshness Plumbing — design decision `raft.tla` `logOk` needs the candidate's `(lastLogTerm, lastLogIndex)` and the voter's own. Two options were considered for carrying the candidate values into the library: - **Option A — extend the `on_vote_eligible` callback signature** to `on_vote_eligible(ctx, from_idx, term, last_log_term, last_log_index)` and have the library compute `logOk` against `on_get_entry_term(log_len)`/`log_len` before calling the gate. **Rejected as the sole mechanism**: it pushes the protocol-defining `logOk` either into the adapter (violating "library owns protocol decisions") or splits it. - **Option B (chosen) — the library owns `logOk`.** The candidate's `last_log_term` and `last_log_index` are accepted as explicit arguments on `raft_on_vote_request` (and on the PreVote eligibility path), the library computes `logOk` against its own `on_get_entry_term(log_len)` / `log_len`, and only then consults the unchanged application gate `on_vote_eligible` for XCE-specific policy (primary-deny / heartbeat stickiness / catch-up). This keeps the `raft.tla` `logOk` predicate **inside** the library (R3.1, R5.5) while leaving `on_vote_eligible` for application policy that is genuinely outside Raft. The candidate advertises its `(last_log_term, last_log_index)` in the `RequestVote` / PreVote wire RPC; the controller adapter decodes them and passes them to the library entry point. The on-wire encoding of `last_log_term` (and the controller's freshness-floor plumbing) is the `ha-term-safety` adapter's job — out of this spec's scope; this spec fixes only the library signatures and the predicate. > **Note — freshness-decision locus moves from caller/dispatcher into the library (both paths).** Today the freshness decision lives **outside** the library: `raft.h` (lines 246–248) documents that PreVote freshness "is applied by the dispatcher before this call", and the binding vote path delegates the decision to `on_vote_eligible`. Option B **moves that decision locus into the library** for **both** the binding (`raft_on_vote_request`) and the PreVote (`raft_on_pre_vote_request`) paths: `raft.c` now computes the `(lastLogTerm, lastLogIndex)` `logOk` itself rather than relying on the caller/dispatcher. This **supersedes the existing `raft.h` lines 246–248 doc contract**, and updating those `raft.h` doc comments (so the header no longer states the dispatcher owns PreVote freshness) is part of the header-contract task. ## Requirements Traceability | Requirement | Summary | Design Elements | raft.tla action | |-------------|---------|-----------------|-----------------| | 1.1–1.7 | Library `commit_index`, agreeIndexes, current-term rule, monotonic, leader-only | `raft_state_t.commit_index`; `raft_advance_commit()`; `on_get_entry_term`; `match_index[]` (upstream) | `AdvanceCommitIndex` (259), current-term rule (270) | | 2.1–2.6 | `on_commit_advanced` trigger, fire-once, separate from reply | `raft_callbacks_t.on_commit_advanced`; strict-advance firing; `raft_init` validation | (seam; `AdvanceCommitIndex` distinct from `HandleAppendEntriesResponse`) | | 3.1–3.6 | `(lastLogTerm,lastLogIndex)` binding-vote freshness | `raft_on_vote_request` logOk; `on_get_entry_term(log_len)`/`log_len`; persist-before-grant | `HandleRequestVoteRequest` logOk (285) | | 4.1–4.5 | Same freshness on PreVote, side-effect-free | `raft_on_pre_vote_request` logOk; no persist/term/role mutation | `HandleRequestVoteRequest` logOk (285), PreVote extension | | 5.1–5.4 | Barrier/durability preserved as policy; PreVote/priority unchanged; no abort | `on_commit_advanced` as trigger only; additive-only changes | (policy; not a TLA+ action) | | 6.1–6.6 | Simulator Figure-8 + Leader Completeness + freshness gate | `raft_sim.c` commit driver, Figure-8 construction, Leader-Completeness assert, freshness exercise | Figure-8 / Leader Completeness invariant | ## Components and Interfaces | Component | Layer | Intent | Req Coverage | Key Dependencies | Contracts | |-----------|-------|--------|--------------|------------------|-----------| | `commit_index` (`raft_state_t`) | Library core | Library-owned commit position | 1 | log core (P0) | State | | `raft_advance_commit` | Library core | AdvanceCommitIndex + current-term rule | 1, 2 | `on_get_entry_term`, `match_index[]`, `on_commit_advanced` (P0) | Service | | `on_commit_advanced` callback | Seam | Commit-advance trigger | 2, 5 | controller barrier / sim (P0) | Service | | `raft_on_vote_request` freshness | Library core | `(lastLogTerm,lastLogIndex)` logOk | 3, 5 | `on_get_entry_term`, `log_len`, `on_persist`, `on_vote_eligible` (P0) | Service | | `raft_on_pre_vote_request` freshness | Library core | Same logOk, side-effect-free | 4, 5 | `on_get_entry_term`, `log_len`, `on_vote_eligible` (P0) | Service | | Simulator | Verification | Figure-8 + Leader Completeness | 6 | all of the above (P0) | State | ### Library Core #### `commit_index` addition to `raft_state_t` | Field | Detail | |-------|--------| | Intent | Hold the library's view of the safe commit position (TLA+ `commitIndex`) | | Requirements | 1.1, 1.5 | **Responsibilities & Constraints** - `uint64_t commit_index;` — the highest 1-based index (== WAL LSN) the library considers safely committed. Monotonic non-decreasing. - Restored at init from the adapter (`raft_config_t.initial_commit_index`, default 0; the library never reads storage — consistent with `initial_term` / `initial_log_len`). - The library never advances `commit_index` except through `raft_advance_commit`; `raft_on_append_entries_reply` (upstream) leaves it untouched (R2.6). #### `raft_advance_commit` — AdvanceCommitIndex (raft.tla:259) with the current-term rule (line 270) | Field | Detail | |-------|--------| | Intent | Advance `commit_index` to `Max(agreeIndexes)` only when that entry is from `currentTerm` | | Requirements | 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 2.2, 2.3 | ##### Service Interface ```c /* * raft_advance_commit: leader recomputes its commit index (raft.tla * AdvanceCommitIndex, 259). A SEPARATE action from append-entries reply * handling (mirrors raft.tla, so a single-node leader can commit and to * keep atomic regions small). * * Computes agreeIndexes = { index in 1..log_len : * ({self} ∪ {peer : match_index[peer] >= index}) is a quorum }. * If agreeIndexes is non-empty AND on_get_entry_term(Max(agreeIndexes)) * == s->term, sets commit_index = Max(agreeIndexes) (Figure-8 current-term * rule) and, on a STRICT advance, fires on_commit_advanced(ctx, commit_index) * exactly once. * Otherwise leaves commit_index unchanged and does NOT fire on_commit_advanced. * * The library performs NO durability work here — on_commit_advanced only * TRIGGERS the controller's existing synchronous commit barrier. * * Returns RAFT_OK on success (whether or not the index advanced); * -EPERM if role != RAFT_LEADER (commit_index left unchanged). */ int raft_advance_commit(raft_state_t *s); ``` - Preconditions: `s->role == RAFT_LEADER` (else `-EPERM`, no change — R1.6). - Postconditions: `commit_index` either unchanged or advanced to `Max(agreeIndexes)` when that entry is current-term; `on_commit_advanced` fired iff `commit_index` strictly increased. - Invariants: `commit_index` never decreases (R1.5); the library reads entry terms only via `on_get_entry_term` (R1.7); `commit_index ≤ log_len`. > **Quorum**: `(n_peers / 2) + 1`, identical to the election-core quorum; `match_index[self]` is implicitly `log_len` (self always agrees up to its own log length), realized by seeding the agree-count with self for each candidate index. `agreeIndexes` is computed by scanning candidate indexes from `log_len` downward and taking the first that reaches quorum (equivalent to `Max(agreeIndexes)` without materializing the set). #### Freshness upgrade — `raft_on_vote_request` (raft.tla:285, closes G6) | Field | Detail | |-------|--------| | Intent | Add the `(lastLogTerm, lastLogIndex)` lexicographic logOk before the application gate | | Requirements | 3.1, 3.2, 3.3, 3.4, 3.5, 3.6 | ##### Service Interface ```c /* * raft_on_vote_request: process an incoming RequestVote RPC (extended). * Adds the raft.tla HandleRequestVoteRequest logOk (285): the library * computes * logOk = (cand_last_log_term > LastTerm(local)) * || (cand_last_log_term == LastTerm(local) * && cand_last_log_index >= log_len) * where LastTerm(local) = (log_len > 0) ? on_get_entry_term(log_len) : 0. * Step order (unchanged except the new logOk gate): * 1. stale-term deny / UpdateTerm (unchanged) * 2. role gate, one-vote-per-term (unchanged) * 3. logOk gate (NEW, G6): deny if !logOk, before on_vote_eligible * 4. application gate on_vote_eligible (unchanged) * 5. on_persist before grant (unchanged) * Returns RAFT_GRANT / RAFT_DENY; sets *reply_term. */ int raft_on_vote_request(raft_state_t *s, int from_idx, uint64_t term, uint64_t cand_last_log_term, uint64_t cand_last_log_index, uint64_t *reply_term); ``` - Preconditions: none beyond a valid `s`. - Postconditions: a vote is granted only when the existing invariants, the new `(lastLogTerm, lastLogIndex)` logOk, and the application gate all pass; persisted before grant. - Invariants: `logOk` is evaluated **before** `on_vote_eligible`; a `!logOk` denies regardless of application policy (R3.4); no weakening of stale-term / one-vote-per-term (R3.5). > **Signature change is a Revalidation Trigger.** `raft_on_vote_request` gains `cand_last_log_term` and `cand_last_log_index`. The controller and the simulator (the two callers) must pass the candidate's advertised last-log `(term, index)`. The PreVote path mirrors this. #### Freshness upgrade — `raft_on_pre_vote_request` (side-effect-free) | Field | Detail | |-------|--------| | Intent | Apply the identical `(lastLogTerm, lastLogIndex)` logOk on the straw poll, mutating nothing | | Requirements | 4.1, 4.2, 4.3, 4.4, 4.5 | ##### Service Interface ```c /* * raft_on_pre_vote_request: answer a PreVote probe (extended). * Applies the SAME (lastLogTerm, lastLogIndex) logOk as the binding vote * (Requirement 3.1), in addition to the existing stale-term deny and the * on_vote_eligible gate. Mutates NOTHING: no term/voted_for/role/commit_index * change, no on_persist (PreVote invariant preserved). * Returns RAFT_GRANT / RAFT_DENY. */ int raft_on_pre_vote_request(raft_state_t *s, int from_idx, uint64_t cand_term, uint64_t cand_last_log_term, uint64_t cand_last_log_index); ``` - Preconditions: none beyond a valid `s`. - Postconditions: grant only when stale-term, logOk, and the application gate all pass; no persistent state touched. - Invariants: identical freshness rule to the binding path (R4.1, R4.2); side-effect-free (R4.4). ### Seam — `raft_callbacks_t` addition (verbatim Authoritative shared contract) Appended to `raft_callbacks_t` and validated **required** in `raft_init`. Reproduced verbatim from the brief so downstream specs depend on the exact shape: ```c /* REQUIRED: fired AFTER commit_index strictly advances. The controller adapter * feeds this into the EXISTING synchronous commit barrier / persist-before-ACK * durability — this callback TRIGGERS commit-eval; it does NOT replace the * barrier. */ void on_commit_advanced(void *ctx, uint64_t commit_index); ``` **Contract rules (R2.4, R5.1)**: the library decides *whether* the commit index advanced; the callback only triggers the adapter's commit work. It is fired exactly once per strict advance (R2.2), never on a held commit index (R2.3), and never from `raft_on_append_entries_reply` (R2.6). It is NULL-checked and required in `raft_init` (`-EINVAL` if NULL — R2.5), matching how the other required callbacks are validated. ### Controller adapter (seam only — heavy wiring is `ha-term-safety`) This spec describes the adapter **only at the seam**; the full controller integration is the downstream `ha-term-safety` consumer. - `on_commit_advanced` → the controller routes the new safe `commit_index` into its **existing** synchronous commit barrier (commit-eval sweep, persist-before-ACK, reply gate) — replacing the term-blind `ha_compute_safe_committed_lsn` advance with the library's current-term-gated value. The barrier mechanism is unchanged; only its **input** (the safe commit position) now comes from the library. - The candidate's advertised `(last_log_term, last_log_index)` is decoded from the `RequestVote` / PreVote wire RPC and passed to `raft_on_vote_request` / `raft_on_pre_vote_request`; `last_log_term` is read back from the WAL term (`on_get_entry_term`-backed index) on recovery. The on-wire encoding and the freshness-floor plumbing are `ha-term-safety`. - On recovery, the controller restores `commit_index` (= recovered committed LSN) into `raft_config_t.initial_commit_index`. ## Data Models ### Domain Model - **Commit position (logical)**: `commit_index : uint64` — the highest 1-based index (== WAL LSN) safely committed, owned by the library, monotonic. - **Convention**: callers pass `s->commit_index` as the `leader_commit` argument to `raft_send_append_entries` — the leader's committed value rides AppendEntries from `commit_index`, which this spec owns (upstream `raft-log-replication` carries `leader_commit` opaquely; the value it carries is the `commit_index` defined here). - **Freshness tuple**: `(last_log_term : uint64, last_log_index : uint64)` per candidate, compared lexicographically against the voter's own `(on_get_entry_term(log_len), log_len)`. ### Logical Data Model - `raft_state_t` gains `commit_index`. - `raft_config_t` gains `initial_commit_index` (restored committed LSN at boot; 0 on first boot — the library never reads storage, consistent with `initial_term` / `initial_log_len`). - No new persisted on-disk format: `commit_index` durability is the controller barrier's existing responsibility; the library imposes no new persisted layout. `last_log_term` reuses the WAL `hdr.term` already written by `raft-log-replication`'s adapter. ### Data Contracts & Integration - **Freshness tuple**: `raft_on_vote_request` and `raft_on_pre_vote_request` accept `(cand_last_log_term, cand_last_log_index)` as explicit arguments. The voter's own values come from `on_get_entry_term(log_len)` (term `0` when `log_len == 0`, consistent with `raft.tla` `LastTerm`) and `log_len`. - **Commit trigger**: `on_commit_advanced(ctx, commit_index)` carries only the new committed index; no entry bytes, no term — the adapter looks up whatever it needs through its own state. ## Error Handling ### Error Strategy The library returns status codes only and **never aborts** (preserves "controller never self-terminates"). Callbacks for commit-advance are trigger-only and return void; the freshness path keeps the existing persist-failure handling. ### Error Categories and Responses - **Caller misuse**: `raft_advance_commit` while not leader → `-EPERM`, `commit_index` unchanged, no `on_commit_advanced`. NULL `on_commit_advanced` at `raft_init` → `-EINVAL`. - **Held commit (Figure-8)**: when `Max(agreeIndexes)` is a previous-term entry, `commit_index` is held — this is the correct safety behavior, not an error; no callback fires. - **Empty agreeIndexes**: no quorum agrees on any index → hold, no callback. - **Stale freshness**: `!logOk` → vote/PreVote DENY (not an error; the standard freshness rejection). - **Persist failure on grant**: unchanged from the existing vote path — a failed `on_persist` denies the vote without recording it. ### Monitoring Reuse the optional `on_trace` callback for new events: `"commit_advance"`, `"commit_hold_prevterm"`, `"commit_hold_noquorum"`, `"vote_denied_log_term"` (stale `lastLogTerm`), `"vote_denied_log_index"` (tie term, stale index), `"pre_vote_denied_freshness"`. NULL-checked exactly like existing trace calls. ## Testing Strategy TDD-mandatory (`testing.md`): tests and simulator drivers are written **before** the `raft.c` implementation (RED → GREEN). All tests are `XCE_TESTING`-gated and live in `src/lib/testsuite/test_raft.c` driving `raft_sim.c`. They reuse the in-memory per-node log backing added by `raft-log-replication`. ### Unit Tests (entry points, against the in-memory sim log) 1. `raft_advance_commit` current-term advance: a leader whose `match_index[]` shows a quorum agreeing through index `k`, with the entry at `k` from `currentTerm`, advances `commit_index` to `k` and fires `on_commit_advanced(k)` once (R1.2, R1.3, R2.2). 2. `raft_advance_commit` Figure-8 hold: a leader with a quorum agreeing through a **previous-term** entry at `k` (no current-term entry above it) holds `commit_index` and fires nothing (R1.4, R2.3 — G4). 3. `raft_advance_commit` monotonicity + non-leader: never decreases `commit_index`; returns `-EPERM` with no change and no callback when not leader (R1.5, R1.6). 4. `on_commit_advanced` separation: `raft_on_append_entries_reply` updating `match_index[]` does **not** by itself advance `commit_index` or fire `on_commit_advanced`; only a subsequent `raft_advance_commit` does (R2.6). 5. `raft_init` requires `on_commit_advanced`: `-EINVAL` when NULL; restores `initial_commit_index` (R2.5, R1.1). 6. `(lastLogTerm, lastLogIndex)` vote freshness: a candidate with a stale `last_log_term` is denied even with a greater `last_log_index`; a candidate with a fresher `last_log_term` is granted with a smaller `last_log_index`; a tie on term grants only when `last_log_index >= log_len` (R3.1, R3.4). PreVote applies the identical rule with no persist/term/role mutation (R4.1, R4.4). ### Integration / Simulator Tests (Figure-8, Leader Completeness) 1. **Figure-8 scenario**: construct a previous-term entry replicated to a quorum but not yet covered by a committed current-term entry; drive `raft_advance_commit` and assert `commit_index` does **not** move over it; then replicate a current-term entry above it to a quorum, drive `raft_advance_commit`, and assert `commit_index` advances over **both** (indirect commit) (R6.2, R6.3 — G4). 2. **Leader Completeness**: across forced leader changes (timeout / partition-driven), assert that any entry committed (≤ `commit_index` on the committing leader) is present in the log of every node that subsequently becomes leader (R6.4) — the `(lastLogTerm, lastLogIndex)` freshness gate (R6.5) is what makes this hold, so the two tests are complementary. 3. **Freshness gate exercise**: drive votes with mismatched `(last_log_term, last_log_index)` tuples and assert the lexicographic outcome (stale-term-but-higher-index denied; fresher-term-but-lower-index granted) (R6.5). 4. **Trigger semantics**: assert `on_commit_advanced` fires exactly on strict advance and not while the current-term rule holds the commit index (R6.6).