From 65ddafeb665c19da2e17db1987abd0bb814ea8c6 Mon Sep 17 00:00:00 2001 From: Rongsong Shen Date: Sat, 20 Jun 2026 08:56:51 +0000 Subject: [PATCH] feat(controller-status): add responderTimeMs @13 to CtrlStatus schema + typemap (task 6.1) Append-only UInt64 epoch-millis field for the responder snapshot timestamp (Req 11.1, wire-format setup). Hand-authored xce_ctrlstatus_t typemap gains a matching responder_time_ms member; codec round-trips it. Controller suite 1/1 OK. Co-Authored-By: Claude Opus 4.8 (1M context) --- .kiro/specs/controller-status/tasks.md | 26 ++++++++++++++++++++++++++ src/include/typemap.h | 1 + src/schema/controller.capnp | 1 + 3 files changed, 28 insertions(+) diff --git a/.kiro/specs/controller-status/tasks.md b/.kiro/specs/controller-status/tasks.md index d2135956..6deba1f6 100644 --- a/.kiro/specs/controller-status/tasks.md +++ b/.kiro/specs/controller-status/tasks.md @@ -63,6 +63,32 @@ - _Requirements: 2.3, 5.3, 7.1, 7.2, 8.1, 9.1, 9.2, 9.3, 9.4_ - _Depends: 4.1, 4.2_ +- [ ] 6. Delta: off-primary committed fidelity + responder timestamp (Req 10, 11) + > Sequential, single request path (schema → typemap → `ctrlstatus_fill` → render → E2E); no `(P)`. Read-only reporting only — the live-fresh-committed heartbeat change is OUT of boundary (route to ha-heartbeat-liveness / ha-repl-hardening). +- [x] 6.1 Add the responder snapshot timestamp field to the schema + hand-authored typemap + - Append a responder snapshot-time field to the status payload at the next free ordinal (after the peer count), carrying the responder's wall-clock at sample time as epoch milliseconds (append-only — never reorder existing field ordinals). Add the matching field to the hand-authored `xce_ctrlstatus_t` typedef in `src/include/typemap.h` and verify it field-for-field against the regenerated codec (per the 1.1/2.1 learning: `$C.mapname` types are NOT auto-generated here). + - Observable: the build regenerates `controller.capnp.h/.c` clean, the typemap matches the codec, and a codec round-trip carries the new field's value unchanged. + - _Requirements: 11.1_ + - _Boundary: schema + typemap_ +- [ ] 6.2 Source off-primary committed values + stamp the timestamp in `ctrlstatus_fill` + - TDD (red first, in `test_ctrlstatus.c`): a STANDBY whose learned committed position is non-zero while its local committed marker is zero and it has applied entries reports the LEARNED committed LSN and committed config version (not zero) and the real applied position (no applied-greater-than-committed inversion); a STANDBY that has not yet learned any committed position reports committed LSN and committed config version as zero/unknown; a PRIMARY reports its own authoritative local committed LSN and committed config tag (unchanged); and the responder snapshot timestamp is non-zero and within tolerance of a test-captured wall-clock sample. Assert the handler still mutates no state. + - Change `ctrlstatus_fill` (`src/controller/ctrlstatus.c`): role-branch the committed sourcing — when the responder is primary, keep its own committed LSN / committed config tag; otherwise report the learned committed LSN / learned committed config tag. Stamp the responder snapshot timestamp from the wall clock (`CLOCK_REALTIME`, NOT the monotonic clock) at snapshot time. All additions are atomic/aligned loads + a clock read — no state mutation. + - Observable: the new unit tests go red → green; the standby case reports the learned non-zero committed value instead of zero; the no-mutation assertion holds. + - _Requirements: 10.1, 10.2, 10.3, 10.4, 11.1_ + - _Boundary: ctrlstatus_fill_ + - _Depends: 6.1_ +- [ ] 6.3 Render the responder snapshot timestamp in the `ctrlstatus` client command + - Add one printed line to `cmd_ctrlstatus` (`src/controller/testsuite/test.c`) rendering the responder snapshot timestamp as a local `YYYY-MM-DD HH:MM:SS,mmm` string in the same form the controller logger uses, so it can be compared directly against `ctrl-N.log`. Confirm and reuse (or match) the exact wall-clock formatter in `src/include/xcelog.h`. The committed lines need no rendering change — task 6.2 only changed their values. + - Observable: piping `ctrlstatus` through the client prints a responder-timestamp line in the log timestamp format alongside the other status fields. + - _Requirements: 11.2, 11.3_ + - _Boundary: cmd_ctrlstatus_ + - _Depends: 6.1, 6.2_ +- [ ] 6.4 End-to-end validation on a live cluster (off-primary committed + log correlation) + - Run against a live 3-node cluster: submit jobs so the primary advances its committed position, then query a STANDBY. Assert the standby's reported committed LSN is non-zero (tracking the primary within the documented join-refresh caveat) rather than zero while it has applied entries, and that the printed responder timestamp matches that standby's own `ctrl-N.log` line for the same query. + - Observable: the live run shows a non-zero off-primary committed LSN (no zero-while-applied inversion) and a responder timestamp that lines up with the standby's log; manual environment prerequisite (live cluster), same status as the other live-E2E task (5.1). + - _Requirements: 10.1, 10.3, 11.2, 11.3_ + - _Depends: 6.2, 6.3_ + ## Implementation Notes (5.1 E2E outcome) - 5.1 ran live on a 3-node `/xce-deploy-ha`-style cluster (ext2/3 fs). PASSED in-scope: `ctrlstatus` shows the primary self-reporting (`Primary: ctrl-1` on the primary), standbys honestly report `Primary: unknown` with `answered by ctrl-2 (standby)` (the dispatch-gate lets standbys answer — Req 7); peers + count correct; `configChecksum` = the real 64-hex SHA-256 (matched the deploy tag prefix) and is IDENTICAL across all 3 nodes with identical config (Req 5.3); `configVersion`==`committedConfigVersion`; authoritative `wait-primary` names the primary in <1s on a serveable cluster (Req 9.1/9.2). - 5.1 FOUND + FIXED a real 2.1 bug: `ha->primary_host` is VESTIGIAL (never written anywhere in the controller), and `voted_for_name` is the last-vote (often self), so neither names the current primary off-primary. Corrected: PRIMARY self-reports its own name; STANDBY/ELECTING/INIT report empty → "unknown" (the authoritative client path reaches the real primary by rotation). Commit ade0081. diff --git a/src/include/typemap.h b/src/include/typemap.h index 54955802..019979c2 100644 --- a/src/include/typemap.h +++ b/src/include/typemap.h @@ -230,6 +230,7 @@ typedef struct { xce_ctrlpeer_t** list; } peers; uint32_t peerCount; + uint64_t responder_time_ms; /* @13: responder wall-clock at sample time (epoch ms) */ } xce_ctrlstatus_t; typedef struct { diff --git a/src/schema/controller.capnp b/src/schema/controller.capnp index 4fa99ab4..2a9c7874 100644 --- a/src/schema/controller.capnp +++ b/src/schema/controller.capnp @@ -507,6 +507,7 @@ struct CtrlStatus $C.mapname("xce_ctrlstatus_t") { fenced @10: Bool; peers @11: List(CtrlPeer) $C.mapname("peers.list") $C.maplistcount("peers.count"); peerCount @12: UInt32; + responderTimeMs @13: UInt64 $C.mapname("responder_time_ms"); # responder's wall-clock at sample time (epoch milliseconds) } struct ClusterConf $C.mapname("xce_clusterconf_t") {