Rongsong Shen
06135954ff
Pre-existing uncommitted work: ha-repl-hardening payload data plane (7.x), heartbeat liveness fix (Req 1.3), connection-reuse ASAN fixes, doc cleanup. New: ha-consistency-invariants umbrella (requirements/design/research) and its four implementation sub-specs (specs generated via batch, cross-spec review applied), steering roadmap + product.md Raft+CP decision.
38 lines
5.3 KiB
Markdown
38 lines
5.3 KiB
Markdown
# Research Log: ha-commit-engine
|
||
|
||
Date: 2026-06-11. Discovery status: **complete via umbrella** — this sub-spec inherits the ha-consistency-invariants discovery (`.kiro/specs/ha-consistency-invariants/research.md`: gap analysis, Option C selection, deep-dives §A–§C, locked decisions). No decisions are re-derived here. This log records only the slice-scoped code verification performed for this spec's design.
|
||
|
||
## Inherited Locked Decisions (umbrella — do not re-litigate)
|
||
- Commit engine as single authority; WAIT/RETAIN completion modes (research synthesis)
|
||
- Admission cap relative to `n_vcpus` (never absolute); `COMMIT_WAIT_RESERVED` default 2 (research item 3(d–e))
|
||
- Blocking condvar on vtask threads accepted; async continuation deferred; no liveness risk (ACKs on iomgr) (research item 3(b))
|
||
- Catch-up standbys excluded from quorum accounting and commit counting until `repl_complete_catchup` (umbrella §Quorum Accounting + commit_on_membership_change contract)
|
||
- Commit state volatile (no persisted commitIndex); convergence/rejoin owned by ha-convergence-recovery (research §C)
|
||
- Own-term gate via WAL_NOOP at promotion — API (`commit_set_writable_lsn`) ships in this spec, consumers in ha-single-leader-role (research item 4)
|
||
|
||
## Slice Verification (2026-06-11, this spec)
|
||
|
||
| Fact | Evidence | Implication |
|
||
|------|----------|-------------|
|
||
| `standby_conn_t.acked_lsn` atomic exists, updated by `repl_on_ack` | src/include/repl.h:24; repl.c:176, store at repl.c:187 | matchIndex bookkeeping ready; engine consumes read-only |
|
||
| Dishonest counting: `acked_lsn > 0` | repl.c:419 | replaced by per-LSN engine evaluation |
|
||
| `n_active == 0 → success` short-circuit | repl.c:432-435 | removed for n_peers>1 via engine (engine knows n_peers; repl does not) |
|
||
| Quorum denominator is `n_active`, not `n_peers` | repl.c:437 | engine uses configured `n_peers` per umbrella commit rule |
|
||
| `ha_has_write_quorum` counts `standby_endpoints->count` (includes catch-up) | ha.c:1507-1526; gate call at ha.c:1537 | fix via new `repl_active_standby_count` (ep && !in_catch_up); `in_catch_up` at repl.h:26 |
|
||
| Sync barrier: `persist_wal_sync_append` | persist.c:1003-1049; committed_lsn store at 1035; `has_standbys → -ENODEV` degraded-success at ~1027-1031; truncation store at persist.c:1438 | adapter passes `written_lsn`; 1035 store moves into engine; -ENODEV case replaced by engine decision; 1438 store stays |
|
||
| ACKs processed on iomgr | `peer_ctrl_msg_handler` → `repl_ack_decode_and_dispatch` (ha.c:2632) → `repl_on_ack` call at ha.c:2748 | `commit_on_ack` feed point; must never block |
|
||
| Membership mutation points | repl.h:50-51 (`repl_add_standby`/`repl_remove_standby`), repl.h:76/79 (`repl_start_catchup`/`repl_complete_catchup`); removal call sites ha.c:3176, 3866 | `commit_on_membership_change` call sites |
|
||
| vtask pool default 8, `--vcpus` override | cluster.c:1076-1083 | `n_vcpus` input to `commit_engine_init`; init point after vtaskmgr sizing |
|
||
| `replicator_t` embedded in `persist_t` (`p->repl`); cluster reaches via `c->persist.repl` | persist.c:1032; ha.c:538, 2748 | engine reaches standby registry via its `persist_t*`; `commit_engine_init(ce, p, ...)` signature suffices |
|
||
| `ha_state_t` has `n_peers` and `raft_c`, but no repl pointer | ha.h:180-185 | add explicit `replicator_t *repl` back-pointer (raft_c only set by election path — not reliable for the gate) |
|
||
| repl.h/persist.h live in src/include (lib-visible); controller-local headers in src/controller/include | find results; cluster.h, worker.h | `commit.h` goes to src/controller/include; repl.h uses forward-declared `struct commit_engine *` to avoid dependency inversion |
|
||
| `repl_wait_majority_ack` callers: persist.c:1032 (walAck), config_repl.c:916 (configAck) | grep | signature gains `entry_lsn`; configAck branch untouched (out of boundary) |
|
||
| Existing tests encode old semantics | testsuite/test_repl_majority.c (walAck cases at :85-:177); meson controller_unit_test_src list at meson.build:74-110 | walAck cases must be updated; new test file registered in meson |
|
||
| `XCE_ERR_NOT_READY` defined | ha.c:35 (`= E_XCE_NOT_READY`) | reuse for admission-cap rejection |
|
||
|
||
## Design Synthesis Outcomes (this slice)
|
||
- **Adapter placement** (brief-mandated): `repl_wait_majority_ack` stays as the shell; gains `entry_lsn` param because the replicator has no LSN knowledge; engine-less fallback retained ONLY for unit-test contexts and explicitly retired by ha-commit-barrier-rollout. Production walAck always goes through the engine.
|
||
- **Engine reachability**: forward-declared `struct commit_engine *ce` on `replicator_t` (keeps src/include/repl.h free of controller headers); engine instance embedded in `cluster_t`; init after vtask sizing + peer count known.
|
||
- **Standalone behavior change (intentional)**: with the engine wired, `n_peers <= 1` sync appends now advance `committed_lsn` (previously the -ENODEV path left it unmoved). Raft-correct (self-quorum), satisfies spec req 6.3; flagged in design.
|
||
- **Membership-shrink semantics**: re-evaluation = wake + re-check; no forced early-fail (umbrella 4.3 requires fail-not-confirm, satisfied by timeout; reconnection within the deadline may still legitimately commit).
|
||
- Review gate: design verified against the umbrella contract — interface verbatim, no contradiction with locked decisions, all 20 slice requirement IDs traced.
|