fix(controller-status): ctrlstatus_fill must prefer a live registry slot over a stale disconnected one (task 8.6, bug-1819)

Found live during task 8.5's optional catching-up-observation attempt:
a killed-then-reconnected standby can end up occupying a NEW
replicator_t.standbys[] slot while its OLD slot (host/port still
populated, ep == NULL) also matches the same configured peer.
ctrlstatus_fill's primary-side peer-status loop stopped at the first
host:port match, so it permanently reported that peer dead even
minutes after it had genuinely reconnected and fully caught up --
violating Requirement 13.3's "true, currently-known connection status"
guarantee.

Fix: scan all MAX_STANDBYS slots for a given peer and prefer any slot
with ep != NULL over a slot with ep == NULL; fall back to dead only
when no matching slot is live. Read-only, lock scope unchanged from
task 8.2.

TDD RED->GREEN (1216 Tests: 2 Failures -> 0), independently reproduced
by the reviewer (exact RED replay) plus two independent live
kill/backlog/restart/poll cycles on separate fresh clusters, both
confirming the previously-stuck standby now converges to ok.

design.md's Locked Decision 3 corrected to document the fix.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RLSNJhNoULvVcUhXEFafQC

.kiro/specs/controller-status/tasks.md

@@ -158,6 +158,8 @@
 - 6.1: `responderTimeMs @13: UInt64` appended to `CtrlStatus` (after `peerCount @12`, append-only) + matching `uint64_t responder_time_ms` in the hand-authored `xce_ctrlstatus_t` (`src/include/typemap.h`); codec wire offset 48, round-trips. Commit 65ddafe.
 - 6.2: `ctrlstatus_fill` role-branches committed sourcing — PRIMARY → own `persist.committed_lsn`/`committed_config_tag` (atomic acquire); else → `ha->last_known_committed_lsn` (atomic acquire) + `ha->last_known_committed_config_tag` (plain aligned read, NO `ha->lock` per design — `ha.h:172` is non-atomic, written under lock at `ha_convergence.c:377`). `appliedLsn` unchanged. Timestamp stamped from `CLOCK_REALTIME` (NOT monotonic). Read-only. STALENESS CAVEAT (in-boundary): `last_known_committed_lsn` refreshes at CtrlHello join/RDHUP, not per heartbeat — off-primary committed can lag; live-fresh = OUT of boundary → ha-heartbeat-liveness/ha-repl-hardening. Commit 3799bd6.
 - 6.3 LEARNING: the controller log timestamp formatter is NOT in `xcelog.h` (an 85-line thin wrapper that just `#include "tlog.h"`); the `YYYY-MM-DD HH:MM:SS,mmm` prefix is assembled inline in vendored tinylog (`tlog.c` `_tlog_root_default_format` ~393/398, via `my_tlog_format` in `src/lib/utils.c:221/228`) — no caller-facing helper, so it was REPLICATED byte-equivalently (`localtime_r` + `strftime "%Y-%m-%d %H:%M:%S"` + `,%03u` of `ms%1000`) in a new header-only `src/controller/testsuite/ctrlstatus_fmt.h` (`ctrlstatus_format_time_ms`), shared by `test.c` (`cmd_ctrlstatus` prints `responderTime:` after `fenced`) and `test_ctrlstatus.c` (TZ=UTC-pinned exact-string unit test). LOCAL time on both sides — comparable to `ctrl-N.log` under the same TZ.
+- 8.6 FIXED 2026-09-16 (bug-1819, found live during 8.5's optional attempt): `ctrlstatus_fill`'s primary-side peer-status scan stopped at the first `replicator_t.standbys[]` slot matching a peer's `host:port` — after a kill+rejoin, the stale (ep==NULL) slot from before the kill can have a lower index than the new live slot the reconnect landed in, so the peer was reported `dead` PERMANENTLY even after fully reconnecting and catching up. Fixed by scanning all `MAX_STANDBYS` slots and preferring any `ep != NULL` match over an `ep == NULL` one. TDD RED→GREEN (1216 Tests: 2 Failures → 0 Failures), independently reproduced by the reviewer including an exact RED replay. Two independent live re-checks (implementer + reviewer, separate deploy dirs/ports) both confirmed a killed-then-restarted standby now converges to `ok` on every poll instead of sticking at `dead`. Neither live run happened to construct the literal two-distinct-slots scenario (an unrelated, already-tracked bug-1474 angle-a symptom consumed the cluster's one spare slot both times) — the two new unit tests are the deterministic proof of the exact fix mechanism; the live runs are corroborating, not substituting. design.md's Locked Decision 3 corrected in the same pass to document the fix (matches this spec's established as-built-drift-correction convention).
+
 - 8.4 VERIFIED 2026-09-16: two independent live 3-node-cluster runs (implementer + reviewer, separate deploy dirs/ports) both confirmed the full Req 13 scenario cleanly with ZERO bugs found (unlike 5.1/6.4/7.6, which each found+fixed a real bug) — healthy primary reports all 3 controllers `ok`; killing a standby flips it to `dead` on the primary's view while the other two stay `ok`; querying the surviving standby directly shows `ok` for itself and `unknown` for every other peer, including the still-alive primary, confirming the asymmetric-visibility design decision (Locked Decision 5) holds live. The tasks 8.1-8.3 implementation required no live-only fixes.
 
 - 8.2 LEARNING: the new `replicator_t.lock` acquisition in `ctrlstatus_fill`'s peer-building loop is held around the whole per-peer loop (calloc + self/primary/non-primary branching for all configured peers), slightly wider than `repl_active_standby_count`'s own lock-and-scan shape that design.md's Constraints-checked note cites as the precedent. Still bounded/safe (n ≤ `MAX_HA_CANDIDATES`, inner scan ≤ `MAX_STANDBYS`, no I/O) and reviewer-approved as non-blocking, but a future touch to this loop should narrow the lock to wrap only the inner `MAX_STANDBYS` registry scan per non-self peer, matching the cited precedent exactly.
@@ -200,3 +202,12 @@
   - Observable: a live run captures at least one `ctrlstatus` query showing catching-up for a rejoining standby before it converges to ok.
   - _Requirements: 13.1_
   - _Depends: 8.4_
+
+- [x] 8.6 Fix: primary-side peer-status matching must prefer a live registry slot over a stale disconnected one
+  - Bug found live during task 8.5's attempt (read-only observation, no code touched at the time): `ctrlstatus_fill`'s primary-side peer-status loop (`src/controller/ctrlstatus.c`) scans `replicator_t.standbys[]` for a `host`/`port` match and stops at the FIRST match by slot index. When a standby is killed, its slot is freed (`ep` cleared) but keeps its `host`/`port` populated (existing bug-1474-documented "guard-holding slot" behavior, unrelated to this delta); when that standby reconnects, `repl_add_standby`'s `ep`-identity idempotency check cannot reuse the old slot (the old `ep` is gone) and allocates a NEW slot instead — so two slots now share the same configured peer's `host:port`, one stale (`ep == NULL`) and one live (`ep != NULL`). Because the stale slot has the lower index, `ctrlstatus_fill`'s first-match-wins scan reports that peer as `dead` PERMANENTLY, even minutes after it has genuinely reconnected and fully caught up — live-reproduced and log-cross-confirmed (see task 8.5's Status Report: primary's own log shows `acked` converging to the target LSN at a specific timestamp, while `ctrlstatus` kept reporting `dead` from restart through +5m45s). This is distinct from bug-1474's already-fixed angle (`on_standby_connected`'s live-duplicate-channel eviction, a different check) — this defect is specific to `ctrlstatus_fill`'s own status-derivation scan and violates Req 13.3's "true, currently-known connection status" guarantee for any standby that has ever been killed and rejoined.
+  - TDD (red first, in `test_ctrlstatus.c`): a fixture with TWO `replicator_t.standbys[]` slots matching the SAME configured peer's `host:port` — a lower-index slot with `ep == NULL` (stale) and a higher-index slot with `ep != NULL && !in_catch_up` (live, caught up) — must report that peer's status as `ok` (sourced from the live slot), not `dead` (the stale slot's would-be status); a sibling case with the live slot `in_catch_up` must report `catching-up`, not `dead`.
+  - Fix: change the matching loop to scan ALL `MAX_STANDBYS` slots for a given configured peer (not stop at the first match) and prefer any slot with `ep != NULL` over a slot with `ep == NULL` when multiple slots match the same `host:port`; if multiple live slots somehow match (shouldn't normally happen), any one of them is an acceptable source since they'd all be observing the same live peer. Keep the read bounded and lock-scoped exactly as task 8.2 left it (no scope regression).
+  - Observable: the new unit test goes red → green; a live re-run of task 8.5's kill-then-restart-under-backlog scenario shows the rejoined standby transition away from a permanent `dead` (ideally observing `catching-up` before settling to `ok`, satisfying 8.5's original optional Observable as a side effect of this fix).
+  - _Requirements: 13.1, 13.3_
+  - _Boundary: ctrlstatus_fill_
+  - _Depends: 8.2_