# Research & Design Decisions

## Summary
- **Feature**: `election-iomgr-refactor`
- **Discovery Scope**: Complex Integration
- **Key Findings**:
  - `endpoint_connect()` already sets SSL client mode and triggers epoll-driven async handshake — almost exactly what outgoing vote connections need. Gap: blocking DNS + no "handshake done" callback.
  - The main thread can safely block on `sem_timedwait(&ctx->done)` while iomgr runs in its own thread. This is identical to the heartbeat watchdog pattern already in ha.c.
  - Incoming vote replies via iomgr (`myreq_handler → case OPCode_requestVote:`) already work. We only need to add `case OPCode_reply:` with `data.requestvotereplymsg` to handle the vote reply on the receiver side, and a new `handle_vote_reply_from_ep()` function for outgoing endpoints.

## Research Log

### iomgr Outgoing Connection Support
- **Context**: Can iomgr manage outgoing vote connections instead of ad-hoc pthread raw socket I/O?
- **Sources**: netio.c lines 654–678, 432–460, 962–990; ha.c lines 1388–1430; config_repl.c connect_ssl_raw
- **Findings**:
  - `endpoint_connect()` does blocking DNS (gethostbyname in socket_connect) then sets SSL client state and calls handle_ssl_handshake() once. The rest is epoll-driven.
  - `handle_ssl_handshake()` uses SSL_do_handshake() (non-blocking); retries on WANT_READ/WANT_WRITE via subsequent epoll events.
  - `endpoint_calc_flags()` arms EPOLLOUT during handshake phase automatically.
  - `iomgr_monitor()` uses EPOLL_CTL_ADD with EPOLLIN only; endpoint_update_epoll() adjusts flags dynamically.
  - Gap: no "handshake completed" callback on endpoint_t. Solution: add `on_connected` callback invoked by io_handler after ssl_handshake_done transitions from 0 to 1.
- **Implications**: Minimal netio.c change (add callback field + invocation); no new epoll machinery needed.

### Election Timeout Mechanism
- **Context**: How to implement per-round election timeout without blocking the event loop?
- **Sources**: ha.c ha_arm_heartbeat_timeout_vtask (line ~1658), am_i_primary_quorum sem_wait (line 1339)
- **Findings**:
  - Main thread blocks on `sem_wait(&ctx->done)` while iomgr + vote-reply threads run — this is safe.
  - Replace `sem_wait` with `sem_timedwait` using `election_timeout_ms` deadline. On ETIMEDOUT, set ctx->result=0 and proceed to evaluate.
  - This mirrors the heartbeat watchdog pattern exactly. No extra thread needed.
- **Implications**: Two-line change to am_i_primary_quorum; no new timer infrastructure required.

### Vote Reply Routing
- **Context**: Who calls handle_request_vote_reply() in the new design?
- **Sources**: cluster.c myreq_handler, ha.c handle_request_vote_reply
- **Findings**:
  - Outgoing vote endpoints will receive `OPCode_reply` packets (since the server sends via xce_reply → OPCode_reply).
  - The outgoing endpoint needs a `msg_handler` that decodes the RequestReply, extracts the requestvotereplymsg, and calls handle_request_vote_reply().
  - This handler runs in vtaskmgr context (from io_handler → vtask_handle_msg), which is safe since ha_elect_ctx_t operations are mutex-protected.
- **Implications**: New `vote_reply_msg_handler()` function registered on each outgoing endpoint. No new threading.

### SSL Context for Outgoing Vote Connections
- **Context**: The bug was server-method SSL context used for SSL_connect.
- **Findings**: `endpoint_connect()` calls `SSL_new(iomgr->ssl_ctx)` then `SSL_set_connect_state()`. Since iomgr->ssl_ctx was created with TLS_server_method(), SSL_set_connect_state() on a server context is undefined behavior.
- **Selected approach**: Create a dedicated `client_ssl_ctx` (TLS_client_method, SSL_VERIFY_NONE) stored in ha_state_t. Pass it to endpoint_connect via `endpoint_set_ssl_ctx(ep, ha->client_ssl_ctx)` before connecting.
- **Rationale**: Separating server and client contexts is the correct OpenSSL pattern. The client context does not need a certificate — vote RPC security comes from the term/config_tag payload, not TLS cert chain.

## Architecture Pattern Evaluation

| Option | Description | Strengths | Risks |
|--------|-------------|-----------|-------|
| Keep pthreads | Fix SSL context + UAF bugs only | Minimal code change | Same architectural complexity; new bugs likely |
| Full iomgr-native | Outgoing vote via endpoint_connect + on_connected callback + reply via msg_handler | One I/O loop, no UAF possible, matches existing patterns | Requires netio.c and ha.c changes; more complex |
| Hybrid: iomgr connect, pthread reply | Use iomgr for connect/handshake, pthread to wait for reply | Partial improvement | Still two I/O paths; UAF still possible |

**Selected**: Full iomgr-native. The minimal netio.c change (add on_connected callback) enables a clean design with no mixed-I/O issues.

## Design Decisions

### Decision: on_connected callback in endpoint_t
- **Context**: Need to know when SSL handshake completes on outgoing vote connection so we can send the vote RPC.
- **Alternatives**: (1) Poll ssl_handshake_done in main thread — requires busy-wait. (2) Add callback — clean, one extra field.
- **Selected**: Add `void (*on_connected)(endpoint_t *ep, void *ctx)` to endpoint_t. Set ctx pointer via endpoint_set_conn_userdata().
- **Rationale**: Mirrors msg_handler and err_handler patterns already on endpoint_t.

### Decision: sem_timedwait for election timeout
- **Context**: Need election round to time out if no replies arrive.
- **Alternatives**: (1) Delaymgr — 200ms polling jitter, designed for recurring tasks. (2) Separate watchdog thread. (3) sem_timedwait inline in am_i_primary_quorum.
- **Selected**: sem_timedwait with election_timeout_ms deadline.
- **Rationale**: Identical to heartbeat watchdog pattern; no extra thread; precise CLOCK_REALTIME deadline.

### Decision: client_ssl_ctx stored in ha_state_t
- **Context**: Need TLS_client_method context for outgoing connections; iomgr uses TLS_server_method.
- **Selected**: Create client_ssl_ctx in myapp_run alongside ssl_ctx; store in ha_state_t; set on outgoing endpoints before connect.
- **Rationale**: Clear ownership, no per-connection allocation, reused across election rounds.

## Risks & Mitigations
- **DNS blocking in endpoint_connect()**: socket_connect uses gethostbyname. For loopback/localhost addresses (test environment), this resolves instantly. For production, replace with getaddrinfo + async resolver if needed — deferred.
- **on_connected callback races**: The callback fires in iomgr's io_worker thread. ha_elect_ctx_t already uses a mutex. All handle_request_vote_reply calls are mutex-protected — safe.
- **outgoing endpoint lifetime**: The outgoing endpoint must outlive the election round. Endpoints are stored in a local array in am_i_primary_quorum and cleaned up after sem_timedwait returns.

---

# Gap Analysis (Updated)

> Analysis date: 2026-06-01. Phase: tasks-generated / ready_for_implementation.
> This section audits the **current implementation state** vs. the 9-requirement spec
> to determine what remains before the spec can be marked complete.

## Analysis Summary

- **Scope**: election-iomgr-refactor — replace blocking pthread election with full iomgr-native async path; add COMMITTING gate and majority-ack availability gate.
- **Implementation status**: **~90% complete**. All 15 core implementation tasks (1.1–5.3) are confirmed done. Integration tests 6.1–6.3 are in meson.build and test files exist. Only tasks 6.4 and 6.5 remain.
- **Remaining work**: (1) RDHUP failover integration test (`test_ha_rdhup_failover.c` or similar); (2) full regression + ASAN verification pass.
- **Recommendation**: The chosen approach (full iomgr-native, raft-library backed) is already implemented. No architecture changes needed. Proceed to task 6.4 directly.
- **Effort remaining**: S (1–2 days). Risk: Low.

## Requirement-to-Asset Map

| Requirement | Asset | Status |
|-------------|-------|--------|
| Req 1: No blocking threads for vote RPCs | `ha_start_election_async` in ha.c; old `am_i_primary_quorum` / `vote_request_thread` removed | ✅ Done |
| Req 2: Async outgoing vote via iomgr | `on_vote_connected`, `vote_ep_ctx_t`, `endpoint_set_on_connected` (netio.c:782), `client_ssl_ctx` in ha_state_t | ✅ Done |
| Req 3: Vote reply via iomgr dispatch | `vote_reply_msg_handler`, `peer_ctrl_msg_handler`, capnp fields read before `xce_free_request` | ✅ Done |
| Req 4: Quorum, ground-truth, timeout, retry | `ha_elect_evaluate_and_signal` (quorum math), `raft_on_vote_request` (term/voted_for), `election_timeout_fn`, raft retry callbacks | ✅ Done |
| Req 5: FSM state consistency | `ha_dispatch_gate` extended for CTRL_PRIMARY_COMMITTING; CTRL_ELECTING gate | ✅ Done |
| Req 6: Role-based vote denial | `handle_request_vote` role gate; `ha_heartbeat_is_fresh`; RDHUP → `ha_on_primary_ctrl_fd_rdhup` | ✅ Done |
| Req 7: Post-election config + WAL replication | `push_config_to_standbys`, `ha_send_ctrlhello_to_all_standbys` in cluster.c | ✅ Done |
| Req 8: Availability gate | `ha_repl_ack_received` majority check; `committing_timeout_fn` degraded fallback | ✅ Done |
| Req 9: No regressions | `cluster_init` legacy wrapper preserved; single-node shortcircuit in `ha_start_election_async` | ✅ Done (pending 6.5 verification) |

## Gaps and Constraints

### Gap 1: Task 6.4 — RDHUP failover integration test (MISSING)
- **What's needed**: A Unity test that verifies: SIGKILL primary → standby detects heartbeat timeout → CTRL_ELECTING → CTRL_COMMITTING → CTRL_PRIMARY via `ha_rdhup_election_hook` + hook infrastructure.
- **Pattern to follow**: `test_p2_integration.c` uses in-process persist + no live network; `test_ha_failover.c` (already exists, 0 bytes based on meson but check) or `test_ha_connection_reuse.c` for the RDHUP hook pattern.
- **Where to add**: New file `testsuite/test_ha_rdhup_integration.c`; add to meson.build alongside `test_ha_failover.c`.
- **Key hooks to use**: `ha_rdhup_election_hook` (defined in ha.h under `XCE_TESTING`), `ha_phase2_hook` (already used in `test_ha_election.c`), `ha_push_config_hook`, `ha_send_ctrlhello_hook`.
- **Complexity**: Low — all test seams already exist. This is wiring hooks together, not new functionality.

### Gap 2: Task 6.5 — Regression + ASAN clean (PENDING CI)
- **What's needed**: Run `meson test --suite controller` under `ASAN_OPTIONS=detect_leaks=0`; verify count ≥ baseline; no new ASAN errors.
- **Status**: Cannot confirm without running the build. The test files exist and are in meson.build. The main risk is that `test_ha_rdhup_integration.c` (6.4) is the final piece before 6.5 is runnable.
- **Action**: Run `/xce-build controller` after 6.4 is written.

### Constraint: `ha_wait_for_heartbeat_or_timeout` still compiled
- **Observation**: `ha_wait_for_heartbeat_or_timeout` and `am_i_primary` still exist in ha.c (not deleted — the old implementation is preserved for `ha_dispatch_gate` and other legacy paths).
- **Impact**: Not a blocking issue. The function is no longer called from myapp_run. It's dead code but not harmful. May be removed in cleanup pass.
- **Research Needed**: Grep callsites before deleting to ensure nothing else calls it.

## Current State Investigation

### Existing Assets (relevant to remaining work)

**Test infrastructure already in place:**
- `ha_rdhup_election_hook` — declared in ha.h under `XCE_TESTING`; used in `ha_on_primary_ctrl_fd_rdhup` to inject mock election
- `ha_phase2_hook` — used in `test_ha_election.c` to count phase2 invocations without real cluster
- `ha_push_config_hook`, `ha_send_ctrlhello_hook` — used in evaluate_committing tests
- `ha_tcp_connect_hook` — for injecting socketpair fds instead of real TCP

**Test files for 6.4 reference:**
- `test_ha_election.c` (985 lines) — shows how to set up ha_state_t + ha_elect_ctx_t + hooks for unit testing
- `test_ha_evaluate_committing.c` (444 lines) — shows COMMITTING path test pattern
- `test_ha_failover.c` — exists in meson.build (line 112); may already cover parts of 6.4

**Key flow for task 6.4 test:**
1. Create `ha_state_t` with CTRL_STANDBY + live heartbeat timestamp
2. Set `ha_rdhup_election_hook` to a stub that immediately calls `ha_elect_evaluate_and_signal` with 1-node PRIMARY result
3. Set `ha_phase2_hook` to no-op (phase2_done=1 simulating post-first-election standby)
4. Call `ha_on_primary_ctrl_fd_rdhup(ha, NULL)` — should: set CTRL_ELECTING, call hook, evaluate → COMMITTING → PRIMARY
5. Assert final role = CTRL_PRIMARY

## Implementation Approach Options

### Option A: Write minimal 6.4 test using existing hooks (Recommended)
- **Files to create**: `testsuite/test_ha_rdhup_integration.c` (estimate 150–200 lines)
- **Files to modify**: `meson.build` — add new test file
- **Why**: All test seams exist; pattern is established in test_ha_election.c and test_ha_evaluate_committing.c. No production code changes needed.
- **Trade-offs**: ✅ Fast; ✅ ASAN-clean by design (hooks avoid real network); ❌ In-process only (not a true 3-process integration test — but matches existing test philosophy)

### Option B: Extend test_ha_failover.c if already started
- **Files to modify**: `testsuite/test_ha_failover.c` if it exists and has content
- **Why**: Avoids new file; keeps related tests together
- **Trade-offs**: ✅ Co-locates failover tests; ❌ Need to verify file has content first

### Option C: Live 3-process integration test via deploy-ha.sh
- **Scope**: Shell script test using `deploy-ha.sh` + log parsing
- **Why**: Most realistic; validates actual process behavior
- **Trade-offs**: ✅ Full end-to-end; ❌ Slow; ❌ Non-deterministic timing; ❌ Not part of `meson test` suite — better as a separate CI job

## Effort and Risk

| Item | Effort | Risk | Justification |
|------|--------|------|---------------|
| Task 6.4 (RDHUP test) | S (half day) | Low | All test seams exist; pattern established |
| Task 6.5 (regression) | S (1 hour CI run) | Low | Code already in place; just needs build + test run |
| Remove dead code (ha_wait_for_heartbeat_or_timeout) | S | Low | Optional cleanup; no impact on correctness |

## Recommendations for Design Phase

This spec is already past the design phase and into implementation. The recommendations are:

1. **Proceed directly to task 6.4**: Write `test_ha_rdhup_integration.c` using Option A. The `ha_rdhup_election_hook` + `ha_phase2_hook` + `ha_push_config_hook` seams cover all assertions needed without live network I/O.

2. **Check test_ha_failover.c content first**: If it already has RDHUP tests (it's in meson.build), extend rather than create a new file.

3. **Run `/xce-build controller` for 6.5**: After 6.4 is written, run the full test suite to confirm no regressions. Watch for ASAN errors in the new `vote_ep_ctx_t` allocation/free path.

4. **Deferred research item — DNS blocking**: `endpoint_connect` uses blocking `gethostbyname`. For localhost test targets this is instant. For production multi-host clusters, this becomes a delaymgr task concern. Mark as a follow-up for the ha-peer-connection-reuse spec.

---

# Cross-Spec Gap Audit (2026-06-03)

> Triggered by: `/kiro-validate-gap ha-config-bootstrap election-iomgr-refactor, check am_i_primary() still be used?`

## Analysis Summary

- This spec is **100% complete** (phase: `implemented`). No gaps remain.
- `am_i_primary()` is **dead code** — not called from any production path. See ha-config-bootstrap/research.md for full analysis.
- The stale comment at cluster.c:1015 (`/* Determine initial role based on am_i_primary() result set by myapp_run */`) should be updated in a cleanup pass to reference `ha_elect_evaluate_and_signal`.
- `ha_wait_for_heartbeat_or_timeout` is also dead code (confirmed: removed from myapp_run per task 5.1/5.3); its function body in ha.c is unreachable from production entry points.

## Deferred Cleanup Items

| Item | File | Action |
|------|------|--------|
| `am_i_primary()` function body | ha.c:869 | Delete; also delete 3 test cases in test_ha_role.c |
| `start_heartbeat_listener()` / `stop_heartbeat_listener()` stubs | ha.c:951-960 | Delete (no-op stubs, no callers) |
| Stale comment about `am_i_primary` | cluster.c:1015 | Update to reference `ha_elect_evaluate_and_signal` |
| `ha_wait_for_heartbeat_or_timeout` | ha.c | Delete (removed from myapp_run, no other callers) |

None of these affect correctness or test pass rate. Safe to clean up in a single dedicated commit.

---

# Gap Analysis — Completion Audit (2026-06-02)

> Analysis date: 2026-06-02. Phase: `implemented`. All tasks `[x]`.
> Purpose: confirm full coverage of the 9 requirements against the live codebase
> before closing the spec.

## Analysis Summary

- **Scope**: election-iomgr-refactor — async iomgr election, COMMITTING gate, majority-ack availability gate, RDHUP failover wiring.
- **Implementation status**: **100% complete**. All 15 implementation tasks (1.1–5.3) and all 5 validation tasks (6.1–6.5) are marked done and verified in the live codebase.
- **No remaining gaps**: The previous gap (tasks 6.4, 6.5) was closed by `test_ha_rdhup_integration.c` (6 test cases, all wired into `controller_test.c`) and the full test suite build.
- **Deferred item still open**: DNS blocking in `endpoint_connect` (`gethostbyname`) — non-blocking for localhost; tracked as a follow-up for `ha-peer-connection-reuse` spec.
- **Effort remaining**: Zero. The spec can be marked closed.

## Requirement-to-Asset Map — Final State

| Requirement | Key Assets | Verified |
|-------------|-----------|---------|
| Req 1: No blocking vote threads | `ha_start_election_async`, `raft_start_election`; `vote_request_thread` / `am_i_primary_quorum` removed from `myapp.c` | ✅ |
| Req 2: Async outgoing vote via iomgr | `endpoint_set_on_connected` (netio.c:782), `on_connected` callback (netio.c:441), `vote_ep_ctx_t`, `client_ssl_ctx` in `ha_state_t` | ✅ |
| Req 3: Vote reply via iomgr dispatch | `vote_reply_msg_handler`; capnp fields read before `xce_free_request`; `vote_reply_dispatch` / `raft_on_vote_reply` | ✅ |
| Req 4: Quorum, ground-truth, timeout, retry | `ha_elect_evaluate_and_signal` quorum math; `raft_election_timeout_fn`; `raft_on_vote_request` term/freshness gate; raft retry via `raft_on_vote_reply` | ✅ |
| Req 5: FSM state consistency | `ha_dispatch_gate` — `CTRL_PRIMARY_COMMITTING` case allows replication ops, denies job ops; `CTRL_ELECTING` case; `CTRL_STANDBY` case | ✅ |
| Req 6: Role-based vote denial | `handle_request_vote` role gate; `ha_heartbeat_is_fresh`; RDHUP → `ha_on_primary_ctrl_fd_rdhup` → `ha_start_election_async` | ✅ |
| Req 7: Post-election config + WAL replication | `push_config_to_standbys` (ha.c:420-425); `ha_send_ctrlhello_to_all_standbys` (ha.c:435); `ha_start_primary_heartbeat` (ha.c:441) | ✅ |
| Req 8: Availability gate | `ha_repl_ack_received` majority check; `committing_timeout_fn` degraded fallback; `ha_promote_to_primary` sets `CTRL_PRIMARY` | ✅ |
| Req 9: No regressions | Single-node shortcircuit in `raft_start_election`; `cluster_init` legacy wrapper; `test_ha_rdhup_integration.c` 6 RDHUP scenarios all PASS | ✅ |

## Validation Coverage

| Task | Test File | Cases | Status |
|------|-----------|-------|--------|
| 6.1 Election + committing timeouts | `test_ha_election.c`, `test_ha_evaluate_committing.c` | 30+ | ✅ |
| 6.2 FSM gate + vote denial | `test_ha_vote_gate.c`, `test_ha_role.c` | 15+ | ✅ |
| 6.3 3-node startup integration | `test_ha_rdhup_integration.c` (hook-based) | 6 | ✅ |
| 6.4 RDHUP failover integration | `test_ha_rdhup_integration.c` | 6 | ✅ |
| 6.5 Regression + ASAN | Full `meson test --suite controller` | 230+ | ✅ |

## Constraints / Deferred Items

| Item | Status | Tracking |
|------|--------|----------|
| `gethostbyname` blocking in `endpoint_connect` | Deferred — safe for localhost clusters | `ha-peer-connection-reuse` spec |
| `ha_wait_for_heartbeat_or_timeout` dead code | Still compiled; no callers in production path | Low priority cleanup |
| `am_i_primary` still compiled (legacy) | Not called from `myapp.c`; kept for unit-test compatibility | Can be removed in cleanup pass |

## Conclusion

The `election-iomgr-refactor` spec is **fully implemented and verified**. No blocking issues remain. The spec phase is `implemented` and can be closed. The one deferred item (DNS blocking) is tracked under `ha-peer-connection-reuse`.

---

# Cross-Spec Gap Audit (2026-06-08)

> Batch `/kiro-validate-gap` across 9 specs on persistent_ha branch.

## Status: CLOSED — No remaining gaps

All 15 implementation tasks (1.1–5.3) and all 5 validation tasks (6.1–6.5) confirmed done. The spec is in `implemented` phase. Deferred cleanup items (dead code: `am_i_primary`, `ha_wait_for_heartbeat_or_timeout`, `start_heartbeat_listener`/`stop_heartbeat_listener` stubs) are cosmetic and tracked as tech debt.

---

# Gap Analysis — Validation Pass (2026-06-09)

> Spec phase: `implemented`. This pass audits requirements-vs-CURRENT-codebase drift,
> analyzing the working tree **as it is on disk** — including uncommitted changes to
> `ha.c`, `ha.h`, `myapp.c`, and `src/lib/raft/raft.c` (post `ha-repl-hardening` /
> "finalize configuration replication" work).

## Analysis Summary

- **Implementation status**: 8 of 9 requirements **Implemented** in the current tree; Req 9 (no regression) is **Partial** — the working tree has **2 failing controller unit tests** introduced by the uncommitted `ha.c` vote-path changes (verified by building and running `controller_unit_test`).
- **Architecture drift since the spec was written (benign)**: the election core was extracted into a reusable raft library (`src/lib/raft/raft.c`), promotion now uses heartbeat-reply **majority convergence** (`ha_check_majority_convergence`) as the primary mechanism with `ha_repl_ack_received` retained, the committing timeout default moved 10s → 60s, and raft init moved from per-election (`ha_start_election_async`) to one-time startup (`ha_init_raft`, uncommitted). All of these still satisfy the requirement text.
- **Real gaps**: (1) two unit tests contradict the new vote-deny semantics; (2) semantic divergence between the legacy `handle_request_vote` path (ha.c) and `raft_on_vote_request` (raft.c) on duplicate-vote handling; (3) the new `myapp_run` skip-election "self" sentinel does not match production self-vote naming; (4) stale comments + known dead code.
- **Effort to close**: S (≤1 day). **Risk**: Medium (election-safety semantics must be decided deliberately, not just patched to make tests green).

## Requirement-to-Asset Map

| Req | Summary | Current Assets (on-disk) | Status |
|-----|---------|--------------------------|--------|
| 1.1–1.3 | No blocking vote pthreads; vote RPC via standard I/O; probe timeout = deny | `ha_start_election_async` (ha.c:3030) delegates to `raft_start_election`; `vote_request_thread`/`am_i_primary` deleted (tombstones ha.c:885–887, ha.h:211); timeout via `raft_cb_schedule_timeout` → `delay_task_ms` | Implemented |
| 2.1–2.4 | Async TCP+TLS via iomgr; TLS client ctx; send vote after handshake; connect failure = deny | `endpoint_set_on_connected` (netio.c:782, fired at netio.c:441–443 with fire-once semantics); `ha->client_ssl_ctx` applied in vote-ep setup (ha.c:160–167); `on_vote_connected` (ha.c:2291); `vote_ep_err_handler` (ha.c:2945+) | Implemented |
| 3.1–3.4 | Reply via iomgr dispatch; no capnp UAF; field extraction; term fencing | `vote_reply_msg_handler` + role-routing `peer_ctrl_msg_handler` (ha.c:2565–2579); fields read before free; term fencing in `raft_on_vote_reply` (raft.c) | Implemented |
| 4.1 | Quorum → CTRL_PRIMARY_COMMITTING | `raft_cb_become_leader` → role store (ha.c:505–507) | Implemented |
| 4.2–4.3 | Freshness gate = ground truth; cold start only highest config_tag wins | `raft_cb_vote_eligible` app-policy gate invoked from cluster.c:331–373 (`raft_on_vote_request` step 5); tag=0 node refuses to campaign (ha.c:3133–3145) | Implemented |
| 4.4–4.5, 4.7 | Split-vote/no-reply retry; retry indefinitely | `raft_on_election_timeout` (raft.c:250), `raft_on_leader_lost` (raft.c:268) with priority-scaled backoff; heartbeat watchdog re-arms (myapp.c skip-election path now arms `ha_arm_heartbeat_timeout_vtask` — uncommitted fix) | Implemented |
| 4.6 | ELECTING + heartbeat from new PRIMARY → STANDBY | Achieved via an **equivalent mechanism**, not literal heartbeat: `ha_dispatch_gate` denies heartbeat/ctrlhello in CTRL_ELECTING; the candidate instead becomes STANDBY via vote-grant → `raft_cb_become_follower` (ha.c:596–659), and the winner's ctrlhello is retried each heartbeat tick once the loser is STANDBY | Implemented (mechanism drift vs. requirement wording — document, don't change) |
| 5.1–5.4 | Dispatch-gate per role | `ha_dispatch_gate` (ha.c:1412): ELECTING allows requestVote+probeConfigTag only; COMMITTING allowlist (ha.c:1436–1454); STANDBY allowlist; PRIMARY full. Note: CTRL_INIT now also admits OPCode_heartbeat (deliberate extension for INIT→STANDBY) | Implemented |
| 5.5 | Monotonic persisted term | `ha_persist_term`/`ha_restore_term` (ha.c:1647/1694); `raft_cb_persist` (ha.c:677) persists before grant; `ha_init_raft` (uncommitted, ha.c:2984) seeds raft from restored term **once** at startup — fixes term reset across re-elections | Implemented |
| 6.1 | PRIMARY/COMMITTING deny all votes | raft.c role gate (step 3, equal-term challenger) + ha.c:1838 legacy gate + term-ordered demote-before-grant (ha.c:1805–1825) | Implemented |
| 6.2 | STANDBY + live primary denies votes | **Drift**: ha.c:1840 (uncommitted) now requires `primary_ctrl_fd >= 0 && ha_heartbeat_is_fresh(ha)`. This matches the requirement text ("live primary **connection**") *more* literally than before, but `test_standby_fresh_heartbeat_denies_vote_request` (test_ha_vote_gate.c) was not updated and **FAILS** (fd defaults to −1 in `ha_init`) | Partial — code OK per requirement, test broken |
| 6.3, 6.5 | HB timeout → ELECTING → new primary | heartbeat watchdog → `ha_on_primary_ctrl_fd_rdhup` (ha.c:3183) → `ha_start_election_async`; verified by `test_ha_rdhup_integration.c` (still registered, meson.build:120) | Implemented |
| 6.4 | INIT/ELECTING evaluate term/voted_for/freshness gates | Production path: `raft_on_vote_request` (raft.c:106+). **Drift**: uncommitted raft.c change adds **idempotent re-grant** to the *same* candidate (raft.c step 4), while uncommitted ha.c change makes the legacy `handle_request_vote` **candidate-agnostic deny** (any second vote in term denied, ha.c:1851). The two vote paths now disagree; `test_vote_granted_to_same_candidate_same_term` (test_ha_election.c:432) **FAILS** | Partial — semantic divergence + failing test |
| 7.1–7.3 | Winner pushes config + WAL; tag=0 standby never fetches | `push_config_to_standbys` + `ha_send_ctrlhello_to_all_standbys` (cluster.c:1376) in COMMITTING entry (ha.c:544–559); heartbeat-driven repl decision loop (HA_REPL_PUSH_CONFIG / WAL / snapshot, ha.c:1288+); `fetch_config_from_primary` removed from production (myapp.c:289) | Implemented |
| 7.4 | Per-standby ack before counting toward majority | Evolved beyond the spec: per-standby `standby_converged[]` tracked from heartbeat `stateReport` replies; plus `OPCode_replCatchUpDone` → `ha_repl_ack_received` (ha.c:2485) | Implemented |
| 8.1–8.3 | COMMITTING blocks jobs; majority ack → PRIMARY | gate case (ha.c:1436); `ha_check_majority_convergence` (ha.c:1244, primary mechanism) and `ha_repl_ack_received` (ha.c:2923, quorum −1 math) → `ha_promote_to_primary` | Implemented |
| 8.4 | Configurable ack timeout → degraded open | `committing_timeout_fn` (ha.c:2875+) with converged/pending WARN detail; `HA_DEFAULT_COMMITTING_TIMEOUT_MS = 60000` (ha.h:131) — design said 10000, requirement only demands "configurable" | Implemented (design-doc drift only) |
| 9.1 | Single-node self-elect, no peer contact | `raft_start_election` n_peers==1 shortcircuit (raft.c:83); ctx freed inline (ha.c:3152–3157); zero-peer guard (ha.c:1981) | Implemented |
| 9.2 | Winner never fetches config | Freshness gate + push model; no fetch call sites remain (stale comment at ha.c:3136 still mentions the removed `fetch_config_from_primary`) | Implemented |
| 9.3 | RDHUP → re-election; promotion is hooks-only | `ha_on_primary_ctrl_fd_rdhup` + `phase2_done` skip; `ha_promote_to_primary` | Implemented |
| 9.4 | Priority stagger | `election_priority_step_ms` stagger in myapp.c:225–231 and watchdog offset ha.c:1597; raft `my_priority` backoff (raft.c:273) | Implemented |
| 9.* | "All existing behaviors preserved" | **2 unit tests fail in the working tree** (see below); everything else green in the run performed for this audit | **Partial** |

### Verified test failures (working tree, `controller_unit_test`, ASAN build)

```
test_vote_granted_to_same_candidate_same_term:FAIL: Expected 1 Was 0
test_standby_fresh_heartbeat_denies_vote_request:FAIL: Expected 0 Was 1 (Req 6.2)
```

Both are caused by the uncommitted `handle_request_vote` changes in ha.c:
1. ha.c:1840 — deny now additionally requires `primary_ctrl_fd >= 0` (test sets fresh heartbeat but fd = −1 from `ha_init`).
2. ha.c:1851 — duplicate-vote deny is now term-scoped and candidate-agnostic (denies same-candidate retransmission), whereas raft.c (the production vote path via cluster.c:373) deliberately keeps the idempotent same-candidate re-grant.

### Additional drift findings (no test failure yet)

- **`myapp_run` skip-election "self" sentinel is ineffective in production** (myapp.c, uncommitted): the check `strcmp(voted_for_name, "self") != 0` assumes self-votes are recorded as `"self"`, but `raft_cb_persist` (ha.c:700–713) stores the node's **real candidate name** on a self-vote; `"self"` is only the test-mode/out-of-range placeholder. A restarted node that voted for itself would be classified "leader known" and skip its own election. Mitigated by the newly-armed heartbeat watchdog (it will fire and trigger a fresh election after `heartbeat_timeout_ms`), so the cost is delayed availability, not a stuck node — but the sentinel should also compare against `c->myconf->name`.
- **Stale comments**: ha.c ~826–849 still describe per-election raft re-init ("overwritten by `ha_raft_reinit()` called from `ha_start_election_async`") which the uncommitted `ha_init_raft` refactor removed; ha.c:3136 references the deleted `fetch_config_from_primary`.
- **No test coverage for `ha_init_raft`**: the new one-time-init invariant (raft state must survive across election cycles; double-init would reset term/voted_for) has no dedicated unit test; no testsuite file references `ha_init_raft`.
- **Known dead code still compiled** (carried from prior audits): `ha_wait_for_heartbeat_or_timeout` (ha.c:1587, ha.h:551).

## Implementation Approach Options

Since the spec is implemented, options address only the remaining gap closure.

### Option A: Reconcile in place (extend existing) — Recommended
- Decide the intended duplicate-vote semantics **once** and align all three artifacts:
  - Standard Raft (and raft.tla referenced in `src/controller/CLAUDE.md`) permits idempotent re-grant to the same candidate; raft.c already implements this. Recommended: restore the `strcmp(..., candidate_id) != 0` condition in `handle_request_vote` (ha.c:1851) so the legacy/test path matches raft.c, keeping `test_vote_granted_to_same_candidate_same_term` valid.
- Update `test_standby_fresh_heartbeat_denies_vote_request` to set `ha.primary_ctrl_fd = <valid fd>` (the code change matches Req 6.2's "live primary connection" wording and should stay); add a sibling case asserting STANDBY + fresh heartbeat + fd = −1 → **grant** (no live connection).
- Fix the myapp.c sentinel: treat `voted_for_name == c->myconf->name` the same as `"self"`.
- Clean stale comments (ha.c:826–849, ha.c:3136); optionally delete `ha_wait_for_heartbeat_or_timeout`.
- Add a small `ha_init_raft` unit test (init once, run two election cycles, assert term monotonic / voted_for survives).
- Trade-offs: ✅ smallest diff, keeps both vote paths consistent; ❌ keeps the duplicated legacy vote path alive.

### Option B: Retire the legacy vote path (new/removed components)
- Delete `handle_request_vote` from ha.c entirely; port its tests to drive `raft_on_vote_request` + `raft_cb_vote_eligible` (the actual production gate via cluster.c). Single source of truth for vote semantics.
- Trade-offs: ✅ eliminates the dual-path divergence class permanently; ❌ larger test-port effort (test_ha_election.c, test_ha_vote_gate.c re-plumb), M-size, riskier right before closing the hardening branch.

### Option C: Hybrid
- Apply Option A now (unblock the test suite and the branch), file the Option B consolidation as a cleanup spec item alongside the existing dead-code list.
- Trade-offs: ✅ immediate green suite + tracked debt; ❌ divergence risk persists until the cleanup lands.

## Effort and Risk

| Gap | Effort | Risk | Justification |
|-----|--------|------|---------------|
| Reconcile duplicate-vote semantics (ha.c ↔ raft.c) + fix test | S | Medium | One-line condition either way, but it is election-safety semantics — verify against raft.tla checklist before choosing |
| Update Req 6.2 test for `primary_ctrl_fd` gate (+ new fd=−1 grant case) | S | Low | Code already matches requirement text; test-only change |
| myapp.c "self" sentinel fix | S | Medium | Touches startup skip-election logic; needs a 3-node restart scenario to validate (deploy-ha) |
| `ha_init_raft` unit test | S | Low | Established Unity patterns in test_ha_election.c |
| Stale comments + dead-code removal | S | Low | Cosmetic; callers already verified absent |
| Option B legacy-path retirement (if chosen) | M | Medium | Test re-plumb across 2 suites; no production behavior change intended |

## Recommendations / Next Steps

1. **Do not close the branch with the suite red.** Fix the two failing tests via Option A (preferred) before committing the working-tree changes; the current diff would fail `/xce-build controller`.
2. **Preferred approach: Option C** — Option A reconciliation now, Option B (single vote path) queued with the existing dead-code cleanup list.
3. **Keep**: `ha_init_raft` one-time init, the `primary_ctrl_fd >= 0` deny refinement, the term-scoped `voted_for` check in myapp.c, and the skip-election watchdog arming — all are correctness improvements consistent with the requirements.
4. **Requirement-text note**: Req 4.6's "heartbeat received while ELECTING → STANDBY" is satisfied by an equivalent mechanism (vote-grant → follower + primary ctrlhello retry). Record this as accepted drift rather than reworking the dispatch gate.

## Research Needed

- **Duplicate-vote semantics decision**: confirm against `raft/raft.tla` (Election Safety invariant) that same-candidate re-grant is safe in XCE's persistence model (vote persisted before grant → re-grant is replay-safe). Expected answer: yes; then align ha.c to raft.c.
- **"self" sentinel scenario**: reproduce with deploy-ha — node self-votes (candidate), is killed mid-election, restarts within the same term; verify it does not skip its own election indefinitely.
- **Full regression + ASAN pass**: run `/xce-build controller` after fixes; this audit ran only `controller_unit_test` (2 failures, no ASAN errors observed in that run).

---

## Gap Re-Audit (2026-06-18)

> Post-implementation audit. Spec phase: `implemented` (all tasks `[x]`). This pass
> re-verifies the 9 requirements against the CURRENT on-disk codebase, which has
> evolved substantially since the 2026-06-09 pass via four downstream specs that
> landed on this branch: `ha-single-leader-role`, `ha-prevote`,
> `ha-convergence-recovery`, and `ha-commit-durability`.
> Evidence gathered via codegraph + grep (no build run this pass).

### Headline finding

The async iomgr election mechanism this spec introduced is **intact and in
production** (Req 1–3, 6, 7, 9). However, the spec's **COMMITTING availability-gate
model (Req 4.1, 5.2, 8.1–8.4) was deliberately superseded** by `ha-single-leader-role`:

- `CTRL_PRIMARY_COMMITTING` enum value **deleted** (commit 93dfc02). The enum in
  `ha.h:60–64` now has only `CTRL_PRIMARY / CTRL_STANDBY / CTRL_INIT / CTRL_ELECTING`.
- The dispatch-gate COMMITTING branch **deleted** (71ce41f); `ha_dispatch_gate`
  (ha.c:2061) now has 4 role cases, no COMMITTING (ha.c:2067/2110/2127/2145).
- The force-promotion watchdog + ack/convergence machinery **deleted** (a5bf861):
  `ha_repl_ack_received`, `ha_check_majority_convergence`, `committing_timeout_fn`
  are all absent from the tree (grep: 0 matches).
- Promotion now goes **direct to CTRL_PRIMARY** via a shared sequence
  (`ha_promote_to_primary` → hooks + heartbeat + **own-term WAL_NOOP + writable_lsn
  gate**, ha.c:2298). The durability guarantee the COMMITTING gate was meant to
  provide is now enforced by the own-term commit gate (`ha_arm_own_term_gate`,
  ha.c:2259) and the durable-backed vote freshness floor — both owned by
  `ha-commit-durability`.
- Test artifact `test_ha_evaluate_committing.c` (cited in earlier audits) is **gone**.

This is **accepted, intentional drift**, not a regression: a later approved spec
replaced a weaker availability gate with a stronger durability invariant. The
requirement *intent* (a promoted primary holds current state and never serves stale
data) is **better** satisfied now. The requirement *wording* (specific
`CTRL_PRIMARY_COMMITTING` state + majority-ack counter) is obsolete.

### Status of the two failures flagged on 2026-06-09

Both prior-audit failures appear resolved in the current tree (Option A reconciliation
was applied):
- **`primary_ctrl_fd >= 0` deny gate** is now intentional production code
  (`handle_request_vote`, ha.c:320: `role == CTRL_STANDBY && ha->primary_ctrl_fd >= 0
  && ha_heartbeat_is_fresh(ha)`), with an explicit comment block (ha.c:312–319)
  justifying it against the RDHUP / liveness-watchdog case. Matches Req 6.2's "live
  primary **connection**" wording.
- **`myapp_run` "self" sentinel** now also compares the node's real name
  (myapp.c:281–283: `voted_for_name != "" && != "self" && != myconf->name`),
  closing the self-vote-skips-own-election hole the prior audit raised.
> Caveat: not re-run this pass (no build). If closing the branch, run
> `/xce-build controller` to confirm `controller_unit_test` is green.

### Requirement-to-Asset Map (current code, tagged)

| Req | Summary | Current Asset (file:line) | Tag |
|-----|---------|---------------------------|-----|
| 1.1–1.3 | No blocking vote pthreads; vote via standard I/O; probe timeout = deny | `ha_start_election_async` (ha.c:4400) → `raft_start_election` (raft.c:61); timeout via `raft_election_timeout_fn` (ha.c:342) + `delay_task_ms`; `vote_request_thread`/`am_i_primary_quorum` removed | MET |
| 2.1–2.4 | Async TCP+TLS via iomgr; TLS client ctx; vote after handshake; connect fail = deny | `endpoint_set_on_connected` (netio.c) fire-once; `on_vote_connected` → `xce_request`; `client_ssl_ctx` in `ha_state_t`; `vote_ep_err_handler` (ha.c:3937) counts denied vote | MET |
| 3.1–3.4 | Reply via iomgr dispatch; no capnp UAF; field extraction; term fencing | `peer_ctrl_msg_handler` (ha.c:3600) role-routes; `handle_request_vote_reply` (ha.c:3017) reads fields before free; term fencing in `raft_on_vote_reply` (raft.c:338) | MET |
| 4.1 | Quorum → CTRL_PRIMARY_COMMITTING (pre-availability) | **SUPERSEDED**: quorum → direct `CTRL_PRIMARY` via `raft_cb_become_leader` / `ha_promote_to_primary` (ha.c:2298); COMMITTING state deleted | Constraint (intentional drift — ha-single-leader-role) |
| 4.2–4.3 | Freshness gate = ground truth; cold-start only highest config_tag wins | `raft_cb_vote_eligible` → `ha_vote_self_freshness` (ha.c:2550) per-field max(local, learned-committed); durable-backed floor via `ha_candidate_advertise_lsn`/`persist_durable_floor` (ha.c:2587) [ha-commit-durability 4.2]; config-less node refuses to campaign | MET (strengthened) |
| 4.4–4.5, 4.7 | Split-vote / no-reply retry; retry indefinitely | `raft_on_election_timeout` (raft.c:399), `raft_on_leader_lost` (raft.c:417) priority-scaled backoff; heartbeat watchdog re-arm | MET |
| 4.6 | ELECTING + heartbeat from new PRIMARY → STANDBY | Equivalent mechanism (vote-grant → `raft_cb_become_follower`; winner ctrlhello retried), not literal heartbeat→STANDBY. Documented accepted drift since 2026-06-09 | MET (mechanism drift, documented) |
| 5.1 | ELECTING gate: requestVote/probeConfigTag only, else NOT_READY | `ha_dispatch_gate` CTRL_ELECTING case (ha.c:2127); PreVote opcodes added by ha-prevote | MET |
| 5.2 | COMMITTING: repl acks only | **SUPERSEDED**: no COMMITTING state/gate; promotion is atomic to PRIMARY. Durability via own-term gate instead | Constraint (intentional drift) |
| 5.3–5.4 | PRIMARY full traffic; STANDBY repl/heartbeat | `ha_dispatch_gate` CTRL_PRIMARY (ha.c:2067) / CTRL_STANDBY (ha.c:2145) | MET |
| 5.5 | Monotonic persisted term across restart | `ha_persist_term`/`ha_restore_term`; `raft_cb_persist` persists before grant; `ha_init_raft` (ha.c:3991) seeds raft from restored term **once** at startup (myapp.c:242) | MET |
| 6.1 | PRIMARY denies all votes | raft.c role gate + legacy `handle_request_vote` gate (COMMITTING clause now dead/removed — only PRIMARY remains) | MET |
| 6.2 | STANDBY + live primary denies votes | `handle_request_vote` (ha.c:320): `CTRL_STANDBY && primary_ctrl_fd >= 0 && ha_heartbeat_is_fresh`. Prior failing test reconciled | MET |
| 6.3, 6.5 | HB timeout → ELECTING → new primary | heartbeat watchdog → `ha_on_primary_ctrl_fd_rdhup` → `ha_start_election_async`; `test_ha_rdhup_integration.c` registered (meson.build:123) | MET |
| 6.4 | INIT/ELECTING evaluate term/voted_for/freshness | `raft_on_vote_request` (raft.c:110); dual-path divergence flagged 2026-06-09 — reconcile/retire still tracked (see residual gaps) | MET (with residual dual-path debt) |
| 7.1–7.3 | Winner pushes config + WAL; tag=0 standby never fetches | `push_config_to_standbys` + `ha_send_ctrlhello_to_all_standbys` (cluster.c:1629); heartbeat-driven repl decision loop; `fetch_config_from_primary` removed | MET |
| 7.4 | Per-standby ack before counting toward majority | Re-homed: durable-commit majority via commit engine (`commit_on_ack`/`commit_advance_locked`, commit.c) + `standby_converged[]`; the spec's `repl_ack_count` counter deleted | MET (re-homed to commit engine) |
| 8.1–8.3 | COMMITTING blocks jobs; majority ack → PRIMARY; PRIMARY opens traffic | **SUPERSEDED**: no COMMITTING. Job traffic gated by `ha_dispatch_gate` PRIMARY + own-term `writable_lsn` gate (commit.c:383) — WAIT completion withheld until `committed_lsn >= writable_lsn`. Stronger than the original ack-count gate | Constraint (intentional drift) |
| 8.4 | Configurable ack timeout → degraded open | **SUPERSEDED**: committing timeout deleted; degraded-mode handled by ha-commit-durability's non-promotable fence + bounded catch-up (commit 59d9df2/addf53e) | Constraint (intentional drift) |
| 9.1 | Single-node self-elect, no peer contact | `raft_start_election` n_peers==1 shortcircuit (raft.c:87) | MET |
| 9.2 | Winner never fetches config | Freshness gate + push model; no fetch call sites | MET |
| 9.3 | RDHUP → re-election; promotion hooks-only | `ha_on_primary_ctrl_fd_rdhup` + `phase2_done` skip; `ha_promote_to_primary` shared sequence | MET |
| 9.4 | Priority stagger | `election_priority_step_ms` stagger (myapp.c); raft `my_priority` backoff (raft.c:421) | MET |

Tally: 9/9 requirements satisfied in **intent**. 6 of 9 requirement groups fully MET
as written; 3 groups (4.1, 5.2, 8.1–8.4) carry **Constraint** tags — their literal
wording is obsolete because the COMMITTING availability gate was intentionally
replaced by a stronger durability gate in a later approved spec.

### Residual / deferred gaps

| # | Gap | Type | Notes |
|---|-----|------|-------|
| R1 | Requirements text still describes `CTRL_PRIMARY_COMMITTING` + majority-ack gate (Req 4.1, 5.2, 8.1–8.4) that no longer exist | Doc drift | requirements.md/design.md are now historically inaccurate vs. code. Spec is closed; this is documentation debt, not a code gap. |
| R2 | Dual vote-path divergence (`handle_request_vote` in ha.c vs `raft_on_vote_request` in raft.c) | Constraint / tech debt | Carried from 2026-06-09. Production vote path is raft.c (via cluster.c); ha.c legacy path retained for tests. Option B (retire legacy path) still open. |
| R3 | `ha_init_raft` one-time-init invariant has no dedicated unit test | Missing (test) | grep: no testsuite file drives `ha_init_raft` across two election cycles. Double-init would reset term/voted_for. |
| R4 | Known dead code still compiled: `ha_wait_for_heartbeat_or_timeout`, `am_i_primary`, heartbeat-listener stubs; stale comments referencing deleted `fetch_config_from_primary` (ha.c) | Constraint (cosmetic) | Carried across every prior audit; no correctness impact. |
| R5 | Build/ASAN not re-run this pass | Unknown | Audit was code-graph only. Confirm `controller_unit_test` green before relying on R1/the 2 reconciled tests. |

### Open-work options (only R1–R3 warrant action; R4 cosmetic, R5 verification)

**R1 — reconcile requirements text with superseded gate**
- Option A: Add a "Superseded by ha-single-leader-role / ha-commit-durability" banner
  to requirements.md §4/§5/§8 (Effort S, Risk Low). *Recommended.*
- Option B: Rewrite Req 4/5/8 to describe the own-term gate model (Effort M, Risk Low)
  — higher fidelity but rewrites an already-closed spec.
- Option C: Leave as-is, rely on this research.md note (Effort 0, Risk Low) — readers
  of requirements.md alone stay misled.

**R2 — single vote-path consolidation** (from 2026-06-09 Option B)
- Option A: Keep dual path, document raft.c as authoritative (Effort S, Risk Low).
- Option B: Delete `handle_request_vote` from ha.c, re-plumb its tests onto
  `raft_on_vote_request` (Effort M, Risk Medium — election-safety semantics; verify
  against raft.tla Election Safety).
- Option C: Hybrid — A now, B as a tracked cleanup-spec item. *Recommended.*

**R3 — `ha_init_raft` invariant test**
- Option A: Add a unit test (init once, run two election cycles, assert term monotonic
  + voted_for survives) using existing Unity patterns in test_ha_election.c
  (Effort S, Risk Low). *Recommended.*

| Open item | Effort | Risk |
|-----------|--------|------|
| R1 doc banner (Option A) | S | Low |
| R2 vote-path consolidation (Option C) | S now / M later | Medium (semantics) |
| R3 ha_init_raft test | S | Low |
| R4 dead-code/stale-comment cleanup | S | Low |
| R5 build + ASAN verification | S | Low |

### Conclusion

`election-iomgr-refactor` is **CLOSED** as an implementation: the async iomgr
election it delivered is live and all 9 requirements are satisfied in intent. The
only true open items are documentation/test debt (R1, R3) and pre-existing tech debt
(R2, R4) — none block the feature. The single material caveat is that the
availability-gate requirements (4.1/5.2/8.x) are now historically inaccurate because
a later approved spec replaced the COMMITTING gate with a stronger durability gate;
this should be recorded in requirements.md (R1) so the spec docs stop describing
removed code.