myxce/.kiro/specs/controller-status/spec.json

{
  "feature_name": "controller-status",
  "created_at": "2026-06-19T00:00:00Z",
  "updated_at": "2026-09-16T09:20:00Z",
  "language": "en",
  "phase": "implementation-complete",
  "approvals": {
    "requirements": {
      "generated": true,
      "approved": true
    },
    "design": {
      "generated": true,
      "approved": true
    },
    "tasks": {
      "generated": true,
      "approved": true
    }
  },
  "ready_for_implementation": true,
  "implementation": {
    "completed": true,
    "tasks_done": "1.1-5.1 (all)",
    "notes": "ctrlstatus opcode + handler + all-roles dispatch gate + libxce any-node & authoritative reads + ctrlstatus command + wait-primary swap, all reviewed and committed; E2E-verified live (primary self-reports, standbys answer with answered-by, config SHA-256 checksum identical across nodes, authoritative wait-primary names the primary). E2E found+fixed the primaryName-from-vestigial-primary_host bug. Out-of-boundary upstream finding: libxce failover returns EINVAL on a dead-first-master (affects joblist equally) -> route to libxce-failover."
  },
  "delta": {
    "reopened_at": "2026-09-15T00:00:00Z",
    "reason": "User enhancement request (2026-09-15): the xce_test ctrlstatus command should additionally report the term in which the current primary was elected and the wall-clock time at which it was elected, so operators can tell how long the current primary has served and correlate its election with a specific event in the logs. Standby-side fidelity is explicitly in scope (mirrors Req 10's off-primary fidelity pattern): a standby that already knows the current primary's identity (Req 2.3) must report the primary's learned election term/time, not an unset or stale local value.",
    "new_requirements": "12 (current primary's election term and time)",
    "needs": "design + tasks regen for Req 12 only; prior tasks 1.1-6.4 (Req 1-11) remain implemented",
    "impl_status": "7.1-7.6 all implemented + reviewed + committed and complete. Live E2E (7.6) found + fixed a real bug (bug-1818): cmd_ctrlstatus gated electedTerm/electedAt on NULLSTR(primaryName), which is unconditionally empty off-primary in this build, permanently hiding the correct, already-converging elected_at_ms data on every standby. Fixed by gating on elected_at_ms == 0 instead (commit 80839661); design.md's Locked Decision 3 corrected to match. Retry confirmed live convergence across a real primary kill/re-election.",
    "prior_delta": {
      "reopened_at": "2026-06-20T00:00:00Z",
      "reason": "Live E2E finding (2026-06-20): ctrlstatus answered by a standby reports committedLsn=0 / committedConfigVersion=0 while appliedLsn=38 — it reads the primary-only raw markers (persist.committed_lsn, ha->committed_config_tag) instead of the standby's learned committed position (ha->last_known_committed_lsn / last_known_committed_config_tag). Plus an observability gap: no responder timestamp to correlate a captured status against ctrl-N.log.",
      "new_requirements": "10 (off-primary committed-state fidelity), 11 (responder snapshot timestamp for log correlation)",
      "impl_status": "6.1-6.4 all implemented + reviewed + committed and complete, validated GO on 2026-07-17 (see validate_impl below)."
    }
  },
  "validate_impl": {
    "decision": "GO",
    "validated_at": "2026-09-16T09:55:00Z",
    "note": "Feature-level /kiro-validate-impl run for the Req 12 delta (all 24 tasks across 1.1-5.1/6.1-6.4/7.1-7.6 now [x]). Full suite green across all three touched components (controller 1210/0, lib 368/0, harness 173/0 failures, no retries needed), TBD/TODO + secrets grep clean across all 20 touched files, live 3-node smoke boot confirmed the delta's own bug-1818 fix holds (standby electedTerm/electedAt correctly match the primary, byte-matched against the controller's own log line). Cross-task integration traced end-to-end at the SOURCE level (not just design.md's narrative): schema -> ha_state_t -> promotion stamp (both roads, past every refusal path) -> heartbeat carry (unconditional store, reconcile-free) -> ctrlstatus_fill (unconditional read, no role-branch) -> CLI render (elected_at_ms==0 gate) -- no contradiction found anywhere in the chain, and Req 10/11's existing fields (committedLsn/committedConfigVersion/responderTimeMs) confirmed undisturbed (distinct wire ordinals, untouched code paths). Requirements coverage 12/12 groups mapped, including the 12.1-12.6 sub-criteria. Dependency direction clean (schema -> lib -> controller -> harness/test-client, one-way). Boundary audit found the Req 12 delta's File Structure Plan omitted 5 as-built files task 7.3 needed (request.h/request.c + 3 cascading signature-conformance test files) -- corrected in design.md the same day (commit ff757704), not a functional defect, purely a doc-completeness gap. Zero blocked tasks.",
    "known_limitations": [
      "A live observation during smoke testing: querying ctrlstatus on a standby within <1s of a fresh submit can show appliedLsn > committedLsn. This is the documented, in-boundary staleness caveat from design.md's Locked Decision #2 (last_known_committed_lsn refreshes only at CtrlHello join/RDHUP, not per-heartbeat) -- explicitly anticipated and accepted at design time, not a regression of the original zero-value bug this spec's delta phase fixed.",
      "cmd_ctrlstatus's print-gate logic (the exact class of code bug-1818 regressed in) has no checked-in unit/regression test -- this CLI print layer has never had one, for any of its lines, going back to Requirement 8's original implementation. Coverage is live-E2E (task 7.6) plus an ephemeral scratch-program check at fix time, not a permanent regression guard. Accepted as a pre-existing, deliberate scope boundary of this spec, documented in design.md's Req 12 delta File Structure Plan for whoever next touches this function -- not a blocker, but a real risk that a future refactor could silently reintroduce a similar primaryName-style gating mistake."
    ],
    "prior_validate_impl": {
      "decision": "GO",
      "validated_at": "2026-07-17T11:15:00Z",
      "note": "First post-implementation validation run -- all tasks (1.1-6.4) were already [x] with no prior validate_impl record, and this delta block's impl_status field had gone stale (still said 6.4 BLOCKED despite tasks.md recording it VERIFIED since 2026-07-02). Full suite 9/9 green (one transient ASAN failure during the run was traced to a concurrent second session competing for the same build directory, not a regression -- reproduced clean 3x in isolation), 36/36 requirement acceptance criteria mapped, cross-task wire contract verified field-for-field end to end (schema -> typemap -> handler -> dispatch -> libxce client -> CLI), two live smoke runs (a 3-round sequential HA soak plus a targeted live 3-node deploy exercising Req 10/11 directly) both PASS."
    }
  }
}