spec(controller-status): mark implemented (E2E pass; primaryName bug fixed); log upstream dead-first-master failover finding

.kiro/specs/controller-status/spec.json

@@ -3,7 +3,7 @@
   "created_at": "2026-06-19T00:00:00Z",
   "updated_at": "2026-06-19T00:00:00Z",
   "language": "en",
-  "phase": "tasks-generated",
+  "phase": "implemented",
   "approvals": {
     "requirements": {
       "generated": true,
@@ -18,5 +18,10 @@
       "approved": true
     }
   },
-  "ready_for_implementation": true
-}
+  "ready_for_implementation": true,
+  "implementation": {
+    "completed": true,
+    "tasks_done": "1.1-5.1 (all)",
+    "notes": "ctrlstatus opcode + handler + all-roles dispatch gate + libxce any-node & authoritative reads + ctrlstatus command + wait-primary swap, all reviewed and committed; E2E-verified live (primary self-reports, standbys answer with answered-by, config SHA-256 checksum identical across nodes, authoritative wait-primary names the primary). E2E found+fixed the primaryName-from-vestigial-primary_host bug. Out-of-boundary upstream finding: libxce failover returns EINVAL on a dead-first-master (affects joblist equally) -> route to libxce-failover."
+  }
+}

.kiro/specs/controller-status/tasks.md

@@ -57,12 +57,17 @@
   - _Depends: 3.2_
 
 - [ ] 5. Validation (live cluster)
-- [ ] 5.1 End-to-end status + authoritative wait-primary on a live cluster
+- [x] 5.1 End-to-end status + authoritative wait-primary on a live cluster
   - Run against a live 3-node cluster: `ctrlstatus` on a healthy cluster shows the correct primary, peers, and version+checksum; querying via a standby still reports the primary with the "answered by <standby>" framing; the version+checksum match across nodes with identical config; and the authoritative wait-primary returns immediately on a healthy cluster, blocks until the NEW primary itself answers after a primary kill (without false-succeeding on a standby's stale primary view), and times out when no primary serves.
   - Observable: the live run shows a correct status (primary/peers/checksum) including via a standby, and the authoritative wait-primary names the new primary after a kill without a false success; manual environment prerequisite (live cluster), same status as other live-E2E tasks.
   - _Requirements: 2.3, 5.3, 7.1, 7.2, 8.1, 9.1, 9.2, 9.3, 9.4_
   - _Depends: 4.1, 4.2_
 
+## Implementation Notes (5.1 E2E outcome)
+- 5.1 ran live on a 3-node `/xce-deploy-ha`-style cluster (ext2/3 fs). PASSED in-scope: `ctrlstatus` shows the primary self-reporting (`Primary: ctrl-1` on the primary), standbys honestly report `Primary: unknown` with `answered by ctrl-2 (standby)` (the dispatch-gate lets standbys answer — Req 7); peers + count correct; `configChecksum` = the real 64-hex SHA-256 (matched the deploy tag prefix) and is IDENTICAL across all 3 nodes with identical config (Req 5.3); `configVersion`==`committedConfigVersion`; authoritative `wait-primary` names the primary in <1s on a serveable cluster (Req 9.1/9.2).
+- 5.1 FOUND + FIXED a real 2.1 bug: `ha->primary_host` is VESTIGIAL (never written anywhere in the controller), and `voted_for_name` is the last-vote (often self), so neither names the current primary off-primary. Corrected: PRIMARY self-reports its own name; STANDBY/ELECTING/INIT report empty → "unknown" (the authoritative client path reaches the real primary by rotation). Commit ade0081.
+- 5.1 UPSTREAM FINDING (out of boundary, NOT controller-status): with a DEAD node FIRST in the masters list, the libxce client failover returns `EINVAL` instead of advancing to the live nodes — so the authoritative `wait-primary` (and ANY client op) can't reach the primary while a dead node leads the list. CONFIRMED pre-existing: plain `joblist` fails IDENTICALLY (`xce_joblist: Invalid argument`) in the same condition; with a live-only masters list `wait-primary` succeeds in 46ms. This spec consumes failover unchanged (per Boundary/Adjacent expectations) → route to libxce-failover, do NOT patch here. The durability soak avoided it by restarting killed nodes (no dead-first-master during waits).
+
 ## Implementation Notes
 - All new C types are generated from the schema (task 1.1) — no hand-written struct headers. Build regenerates `controller.capnp.h/.c`.
 - Task 2.2 is the one HA-critical change (request-handling path): keep it scoped to the single read-only opcode and prove no other opcode's gating widened.