Root cause (found via live 3-node E2E + a verification test): lsn was 0-based
(hdr.lsn = current_lsn++ from current_lsn=0), and 0 was overloaded as a
sentinel, causing three cold-start defects on a freshly-promoted leader:
- the first WAL entry (own-term promotion NOOP at lsn 0) was structurally
uncommittable: commit_advance_locked only stores when cand>cur and
committed_lsn starts at 0, so lsn 0 can never advance it;
- commit_set_writable_lsn(noop_lsn=0) set writable_lsn=0, which wait_done
reads as 'gate off' — the own-term safety gate was a no-op on the first leader;
- bug-543 (WAL hole): ha_stream_wal_to_standby filters hdr.lsn>standby_lsn;
a fresh empty standby reports standby_lsn=0 (ambiguous 'empty' vs 'has lsn 0'),
so lsn 0 failed 0>0 and was dropped -> non-contiguous standby WAL.
Fix: a fresh persist store initializes current_lsn=1, reserving lsn 0 as the
none/empty sentinel. The first real entry is lsn 1 -> committable (0->1),
writable_lsn=1 -> gate effective, and standby_lsn=0 unambiguously means 'empty'
so catch-up streams from the first entry with no hole. persist_recover is
data-driven (current_lsn=max(hdr.lsn+1)), so existing on-disk WALs and non-fresh
recovery are unaffected; the change only sets the base for brand-new stores.
No wire/capnp change.
Verification: test_coldstart_lsn0 drives the real commit engine + real streamer
and proves H1 (first entry commits 0->1), H3 (gate effective + blocks when
committed<writable), H4 (empty-standby catch-up is contiguous [1,2], control
standby_lsn=1 gets only [2]). Full controller suite green under ASAN. Tests that
relied on the lsn-0 base were shifted (several strengthened); commit.c /
config_repl.c / ha.c / wire format untouched.
Owning specs: ha-commit-engine / ha-wal-replication / persist. Follow-up:
sweep stale lsn-0 comments (repl.c:429-436, persist.c:1055-1075/1240-1243/1296).
repl_active_standby_count: registry truth (ep != NULL && !in_catch_up)
under r->lock. ha_state_t gains repl back-pointer (wired at init by
task 5.3); ha_has_write_quorum uses registry truth when wired, exact
legacy fallback when NULL; formula unchanged. Test 29 (wired gate,
catch-up-only standby → NOT_READY, admits after clear) captured RED
pre-fix. 29/29 group tests; suite at the 3-failure baseline.
Tests 21-28: repl_active_standby_count contract (5 RED vs inert -1
stub) + 3 gate-level pins (n_peers<=1 always-true, zero-connected
reject, stateless re-accept) that 4.2 must preserve. Declaration in
repl.h; ha.c/ha.h untouched (4.2 boundary). 28 tests / 5 RED; suite
at the 3-failure baseline with unchanged signatures.
Tests 17-20: catch-up ACK exclusion vs pending waiter, membership-change
wake of covered waiter, removal-below-majority fails at own deadline
(no forced early-fail), non-blocking bound. Landed GREEN: task 1.2's
commit_on_membership_change already implements the full 3.2 contract,
so RED was unachievable; reviewer confirmed mutation-killing power and
zero production changes. 20/20 group tests; suite at baseline.
9 new cases (8-16): block-until-majority, deadline timeout, standalone
fast path, zero-ACK regression, n_vcpus-relative cap with floor, ACK
progress at saturation, gate as blocking predicate (NOT_READY is
admission-cap-only per corrected contract). Main-thread-only assertions
after joins; 16 tests / 9 RED failures; baseline unchanged.
