---
id: "bug-1819"
status: "fixed"
status_updated: "2026-09-24"
status_note: ""
component: "controller"
severity: "unknown"
timestamp: "2026-09-16T17:15:00Z"
last_seen: "2026-09-19T09:00:00Z"
occurrences: 1
files: ["src/controller/ctrlstatus.c (ctrlstatus_fill", "primary-side peer-status matching loop)", "src/controller/testsuite/test_ctrlstatus.c"]
tags: ["controller-status", "ctrlstatus", "ctrlstatus_fill", "repl", "standby_conn_t", "slot-reuse", "first-match-wins", "stale-slot", "live-e2e-found", "task-8.6", "tdd"]
related: ["bug-1474"]
legacy: true
---

# bug-1819: controller-status spec Requirement 13 delta, task 8.6 (bug found live during task 8.5's optional…

## Description
controller-status spec Requirement 13 delta, task 8.6 (bug found live during task 8.5's optional E2E attempt): ctrlstatus_fill's primary-side peer-status loop (src/controller/ctrlstatus.c) scanned replicator_t.standbys[] for a host/port match and stopped at the FIRST match by slot index. When a standby was killed and later reconnected into a DIFFERENT replicator slot than the one it originally occupied (its old slot keeps host/port populated with ep==NULL after the kill), the lower-indexed stale slot won the scan and the peer was reported dead PERMANENTLY -- even minutes after it had genuinely reconnected and fully caught up. Task 8.5's live run observed this directly: primary log showed acked_lsn converging to the committed tip at a specific timestamp while ctrlstatus kept reporting dead from restart through +5m45s.

## Environment
_Not recorded (migrated from buglog.json)._

## Reproduction
_Not recorded (migrated from buglog.json)._

## Root cause
The primary-side branch of ctrlstatus_fill's per-peer loop scanned replicator_t.standbys[] with a `for (j = 0; j < MAX_STANDBYS; j++) { if (host/port match) { ...; break; } }` pattern -- first host/port match wins, regardless of whether that slot is live (ep != NULL) or stale (ep == NULL). repl_add_standby's slot allocation is keyed by ep identity, not by configured-peer name, so a killed-then-reconnected standby's NEW connection can land in a different slot than its old, now-stale one (confirmed live this session via a related but distinct mechanism -- see bug-1474 cross-reference below); once that happens, the stale, lower-indexed slot permanently wins ctrlstatus_fill's first-match scan even though the peer is genuinely live and caught up in its new slot.

## Fix
FIXED (2026-09-16, TDD RED->GREEN). Changed the matching loop to scan ALL MAX_STANDBYS slots for a given configured peer (no `break` on the first match) and prefer any slot with ep != NULL over a slot with ep == NULL when multiple slots match the same host:port -- tracked via a `live_match` flag; the first slot with ep != NULL found during the scan determines the reported status (ok/catching-up per its own in_catch_up), and a stale (ep == NULL) match only sets the dead fallback if no live match has been seen yet. If more than one live slot somehow matches the same peer, any one of them is an acceptable source (design.md task 8.6 note) since they would all observe the same live peer. Scope: read-only, same MAX_STANDBYS==3 bounded scan and same repl->lock scope task 8.2 already established -- no lock-scope change (explicitly out of this task's scope per its own text). RED: 2 new unit tests (test_ctrlstatus_peer_status_prefers_live_slot_over_stale_ok, test_ctrlstatus_peer_status_prefers_live_slot_catching_up) added to test_ctrlstatus.c, each constructing a fixture with a lower-index stale slot (ep==NULL) and a higher-index live slot (ep!=NULL) matching the SAME configured peer's host:port -- failed pre-fix with 'Expected 1 Was 3' and 'Expected 2 Was 3' (the permanently-dead symptom, reproduced exactly). GREEN: same two tests pass post-fix; full controller_unit_test suite = 1216 Tests, 0 Failures (both the meson aggregate and the standalone controller_unit_test binary's own summary line cross-checked, per testing.md convention) -- no regressions. LIVE VERIFIED: fresh 3-node cluster (deploy-ha.sh, /tmp/xce-deploy-ctrlstatus86), killed standby ctrl-2, submitted an 80-job backlog while it was down (committedLsn advanced 1->161, confirming genuine WAL activity), restarted ctrl-2, polled ctrlstatus on the primary 8 times over ~24s -- ctrl-2 showed ok immediately on every poll after reconnect (never stuck at dead), while the primary's own log confirmed a genuine WAL catch-up stream (standby[1] acked 1->161) was still converging underneath, proving the fix reads the live registry state correctly rather than merely getting lucky on timing.

## Verification
_Not recorded (migrated from buglog.json)._

## References
- Related: bug-1474