# Brief: ha-wal-replication ## Problem WAL entries are written to disk on the primary but never transmitted to standbys. `repl_broadcast()` is a no-send stub (casts payload to void, only increments counters). `repl_wait_majority_ack()` returns immediately because the replicator's `standbys[]` array is never populated from the HA layer's `standby_endpoints`. The result: job submits succeed locally but are invisible on standbys and not durable across failover. ## Current State - `repl.c`: `repl_add_standby()`, `repl_remove_standby()`, `repl_broadcast()`, `repl_wait_majority_ack()` all exist with correct signatures. `repl_broadcast` is a stub (line 196-197: `(void)payload; (void)payload_len`). - `persist.c`: `persist_wal_sync_append()` calls `repl_broadcast` → `repl_wait_majority_ack` → advances `committed_lsn`. The call chain is wired; the send is what's missing. - `ha.c` / `config_repl.c`: `ha->standby_endpoints` manages per-standby connections (promoted vote eps, late-joiner connects). Nobody calls `repl_add_standby()` from here. - `cluster.c`: `OPCode_walAck` handler exists (`ha_repl_ack_received`) but is unreachable because standbys never receive WAL entries to ACK. ## Desired Outcome 1. WAL entries written on the primary are transmitted to all connected standbys. 2. Each standby applies received WAL entries and sends an ACK. 3. The primary advances `committed_lsn` only after a majority of standbys ACK. 4. Job submits, state transitions, and queue changes survive primary failover. ## Approach Bridge the replicator and HA layer: 1. **Register**: When the HA layer connects to a standby (promoted vote ep or late-joiner), call `repl_add_standby()` to register that endpoint with the replicator. 2. **Deregister**: When a standby disconnects (RDHUP / err_handler), call `repl_remove_standby()`. 3. **Send**: Fill in `repl_broadcast()` to serialize the WAL header + payload into an `OPCode_walAck` or similar message and enqueue via `xce_request()`. 4. **ACK**: On the standby, when a WAL entry is applied, send a reply carrying the acked LSN. On the primary, `handle_wal_ack` calls `repl_on_ack()`. ## Scope - **In**: repl_broadcast TCP send, HA→replicator standby registration/deregistration, standby-side WAL ACK reply, integration with existing `repl_wait_majority_ack`. - **Out**: Changes to the heartbeat-driven decision loop (WAL streaming via NOP replies is already wired in ha-repl-hardening). Snapshot transfer path. Config replication path. ## Boundary Candidates - `repl_broadcast` send implementation (repl.c) - Standby registration bridge (ha.c / config_repl.c → repl_add_standby) - Standby WAL ACK reply (cluster.c / ha.c) ## Upstream / Downstream - **Upstream**: ha-config-bootstrap (repl_broadcast API, repl_wait_majority_ack), ha-repl-hardening (WAL catch-up payload, standby-side apply_wal_entry) - **Downstream**: None — this is the last missing link ## Existing Spec Touchpoints - **Reopens**: ha-config-bootstrap task 7.6 (repl_broadcast TCP send deferred) - **Adjacent**: ha-repl-hardening (WAL payload already wired to standby apply path) ## Constraints - Must use the existing `repl_add_standby` / `repl_remove_standby` API - Must reuse `OPCode_walAck` opcode for the ACK path - Standby WAL apply must be idempotent (replay-safe)