forked from lei.chen/myxce
Rongsong Shen
06135954ff
Pre-existing uncommitted work: ha-repl-hardening payload data plane (7.x), heartbeat liveness fix (Req 1.3), connection-reuse ASAN fixes, doc cleanup. New: ha-consistency-invariants umbrella (requirements/design/research) and its four implementation sub-specs (specs generated via batch, cross-spec review applied), steering roadmap + product.md Raft+CP decision.
24 lines
2.3 KiB
Markdown
24 lines
2.3 KiB
Markdown
# Brief: ha-commit-engine
|
|
|
|
> Sub-spec 1 of 4 implementing the **ha-consistency-invariants** umbrella contract.
|
|
> Authoritative requirements/design/research: `.kiro/specs/ha-consistency-invariants/` (requirements.md, design.md §Commit Engine + §Quorum Accounting, research.md). This brief scopes the slice; do not re-derive decisions.
|
|
|
|
## Problem
|
|
The majority-ACK check is dishonest today: `repl_wait_majority_ack` counts standbys with `acked_lsn > 0` ("ACKed anything ever"), not `acked_lsn >= entry_lsn`; quorum is checked only at the dispatch gate (TOCTOU); catch-up standbys count toward write quorum while excluded from broadcast; zero active standbys short-circuits to success. Even the one existing barrier (job submit) provides no real per-entry guarantee.
|
|
|
|
## Desired Outcome
|
|
A commit engine (`commit.c`/`include/commit.h`) is the single authority for commit-index advancement: per-LSN majority evaluation (`acked_lsn >= N`, self counted), contiguous + monotonic `committed_lsn`, WAIT/RETAIN completion modes, `n_vcpus`-relative admission cap, membership-change re-evaluation. Wired behind the existing job-submit sync path only (adapter in `repl_wait_majority_ack`). `ha_has_write_quorum` excludes catch-up standbys.
|
|
|
|
## Scope
|
|
- **In**: `commit.c` + `include/commit.h` per design §Commit Engine service interface; quorum-accounting fix in `ha.c`; `repl_on_ack` → `commit_on_ack` feed; removal of the `n_active == 0 → success` short-circuit for `n_peers > 1`; `n_peers <= 1` self-quorum preserved; `testsuite/test_commit_engine.c`.
|
|
- **Out**: classification/rollout to other WAL types (ha-commit-barrier-rollout); own-term gate *consumers* and COMMITTING removal (ha-single-leader-role — but `commit_set_writable_lsn` API ships here, unset = no gate); demotion/egress/catch-up (ha-convergence-recovery).
|
|
|
|
## Requirements (umbrella IDs)
|
|
2.1 (job-submit path), 2.3, 2.6, 4.1, 4.2, 4.3, 4.4, 6.5, 7.1, 7.4, 7.5.
|
|
|
|
## Upstream / Downstream
|
|
- **Upstream**: ha-wal-replication (acked_lsn, WalAckReply — complete).
|
|
- **Downstream**: ha-commit-barrier-rollout, ha-single-leader-role, ha-convergence-recovery (all consume the engine).
|
|
|
|
## Constraints
|
|
Per umbrella design: `commit_on_ack` callable from iomgr (never blocks); blocking waits on vtask threads only; admission cap expressed relative to `n_vcpus` (never absolute); TDD in testsuite/ first.
|