Adds the Raft Apply-Ordering & Hook Decoupling roadmap phase: canonical
pre-hooks -> raft commit -> apply -> notify ordering, with the raft protocol
step lifted out of the notification hook fan-out (root of the bug-795/828/
851/1060 divergence family). Decision: apply-after-commit via a single
shared apply_committed_entry() seam reused by online apply and restart
replay.
- raft-drive-async: full spec docs (brief/requirements/research/design)
committed alongside its already-tracked tasks.md; async-drive prerequisite
for the apply-decoupling work, sequenced to start first (threatens
failover liveness today).
- raft-apply-decoupling: new spec (brief + facts-in-raft.md) covering the
apply seam and the facts (WAL_MODE_RETAIN) apply-timing question (open
decision D1).
- job-array-single-wal-record: promoted from backlog to a tracked spec,
sequenced after raft-apply-decoupling.
- ha-consistency-invariants: add criterion 2.9 (Facts-Apply invariant),
documenting the current on-receipt fact-reaction deviation this family
will reconcile.
- raft-commit-safety / raft-log-replication: reconcile spec.json phase to
"implemented" (both were mislabeled tasks-generated despite full 18/18
and 24/24 task completion + raft_sim GO).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SDWVNQLKxqTVHh42cH7rV3
Canonical clean build (tools/xce-build.sh lib: rm -rf + fresh meson ASAN setup) passes
from scratch: libxce test OK, Fail 0, ASAN-clean. All election + log-replication +
commit-safety (G4 Figure-8 commit advance) + freshness (G6) + simulator invariant tests
green. Closes the raft-commit-safety library half of bug-851 (G4/G6).
KNOWN: the controller (cluster.c) vote callers still use the pre-extension signatures and
will not compile until the ha-term-safety adapter wires last_log_term/index — the accepted
revalidation trigger, addressed next.
7.1 Figure-8 commit refusal (G4): prior-term entry at a genuine quorum HELD (only the
current-term gate keeps commit_index at 0), then a current-term entry advances over
both (indirect commit) with one fire. Closes the Req 6.3 two-phase coverage.
7.2 Leader Completeness: committed prefix on {0,1}; node 0 fails; the stale node 2 is
DENIED by the freshness gate so only the up-to-date node 1 can win — and it holds the
full committed prefix (term+bytes).
7.3 freshness lexicographic (vote + prevote): stale-term-higher-index DENIED,
fresher-term-lower-index GRANTED, tie decided by index; prevote side-effect-free.
7.4 on_commit_advanced trigger: fire-count tracks STRICT advances — held(0)/advance(1)/
no-op-redrive(1)/advance(2)/non-leader(2).
Test-only (sim_set_log_entry local helper); reviewed in-parent (transient API 529 on the
review subagent). ASAN green (243 tests).
logOk (raft.tla:285, closes G6): local_last_log_term = log_len>0 ? on_get_entry_term(log_len):0;
logOk = cand_last_log_term > local || (== && cand_last_log_index >= log_len). Lexicographic:
term dominates, tie decided by index. Shared raft_log_ok helper.
Vote (4.2): logOk deny inserted after stale-term/UpdateTerm/role-gate/one-vote-per-term and
BEFORE on_vote_eligible (denies even when the app gate would grant); persist-before-grant intact.
PreVote (5.2): identical logOk, fully side-effect-free (no term/vote/role/commit mutation, no
on_persist). Signatures extended (1.3) on raft_on_vote_request/pre_vote_request; all 39 lib call
sites + sim routing updated; sim captures the candidate's send-time last-log tuple (raft_sim_rpc_t).
Empty-log call sites pass (0,0) → logOk true → existing outcomes preserved. 9 freshness tests.
NOTE: controller cluster.c vote callers still use the old signatures and will NOT compile — an
accepted, design-documented revalidation trigger; the controller adapter plumbing is owned by the
downstream ha-term-safety spec (to be implemented next). Library suite ASAN green (239 tests).
AdvanceCommitIndex (raft.tla:259/270, closes G4): leader-only (-EPERM); Max(agreeIndexes)
= highest N with a quorum ((n_peers/2)+1) of match_index>=N, self counted as log_len;
scan log_len downward. CURRENT-TERM RULE: advance commit_index to N only when
on_get_entry_term(N)==s->term, else HOLD (Figure-8 safety — never commit a previous-term
entry on replication count alone; a later current-term commit indirectly commits it).
Monotonic; fires on_commit_advanced once per strict advance; commit_index<=log_len.
3.3: raft_on_append_entries_reply left untouched — verified it never advances commit
(separation, Req 2.6). 6.1: sim commit-driver helper raft_sim_advance_commit added.
Req 6.3 two-phase indirect-commit transition covered end-to-end by task 7.1 (sim).
ASAN green (230 tests).
1.1: raft_state_t.commit_index (monotonic, ==WAL LSN, <=log_len, advanced only via
raft_advance_commit, untouched by AE reply) + raft_config_t.initial_commit_index.
1.2: on_commit_advanced(ctx, commit_index) REQUIRED callback — fired only on strict
advance, a TRIGGER for the controller commit barrier (never does durability itself).
2.1: raft_init requires it (-EINVAL if NULL) + restores commit_index from
initial_commit_index. raft_advance_commit prototype declared (impl in 3.2).
Coupled callback wiring to keep the required-callback ripple green: sim wires
sim_on_commit_advanced (+per-node commit_index mirror & fire-count), all test cbs
builders set it, raft_sim_restart_node restores the durable commit position. Vote
signatures unchanged; no commit-advance logic yet. ASAN green (224 tests).
