diff --git a/.kiro/specs/ha-commit-durability/INVESTIGATION-failover-job-loss.md b/.kiro/specs/ha-commit-durability/INVESTIGATION-failover-job-loss.md new file mode 100644 index 00000000..6da3429b --- /dev/null +++ b/.kiro/specs/ha-commit-durability/INVESTIGATION-failover-job-loss.md @@ -0,0 +1,58 @@ +# Investigation: committed jobs lost across controller failover (bug-795 family) + +> Live investigation 2026-06-20. Status: **root-cause located, fix not yet applied.** This is the long-standing data-loss bug (PREDATES libxce-failover — confirmed by the user; libxce-failover + xce-test-e2e-primitives were *test-infrastructure* improvements, not the durability fix). Owning spec: **ha-commit-durability** (REOPENED for bug-795). Adjacent: ha-convergence-recovery, ha-single-leader-role, ha-peer-connection-reuse. + +## Symptom +Running the **controller failover test** (kill primary / restart nodes while a client submits jobs), committed jobs that were ACKed (`jobsubmit ok`) and visible in `joblist` **disappear** after enough kill/restart cycles. In the captured run, jobs survived terms 2→4 then vanished at the term-4→5 transition; `ctrlstatus` then showed them absent. NOT a `committedLsn` display bug — the data is genuinely gone. + +## DECISIVE EVIDENCE — it is real data loss, not a commit-index issue +Decoded the raw WAL bytes at `/tmp/xce-ha-deploy/controller/ctrl-{1,2,3}/data/wal/seg-0000000000000001.wal` (format: `wal_entry_hdr_t` 40B `{term,lsn,timestamp,type,payload_len,crc,prev_crc}` + payload; `src/include/wal_types.h`): + +| Node | WAL entries (physical) | JOB_SUBMITs | State | +|------|------------------------|-------------|-------| +| ctrl-1 | LSN **1–15** (starts NOOP@1) | **12** | full history — but **DOWN** (`connect peer failed -111`) | +| ctrl-2 | LSN **16–18** (starts NOOP@16) | 2 | live, **missing 1–15** | +| ctrl-3 | LSN **13–18** (starts NOOP@13) | 4 | live, **missing 1–12** | + +The committed prefix (LSN 2–12 = the lost jobs) is **physically absent** from both live nodes; it exists only on the down node. `snap/` dirs are **empty** on all nodes (no snapshot). + +## ROOT CAUSE (mechanism, code-grounded) +A node that (re)joins with an **empty WAL** is brought up to the cluster's **current committed watermark by WAL streaming `[watermark..committed]` — NOT from LSN 1 — and there is no snapshot to backfill `[1..watermark-1]`.** So it ends up with a permanent prefix gap. When the failover test eventually has the only full-log node down while the others rejoined at high watermarks, the committed jobs are gone for good. + +Why the `seg-...001` file contains LSN 16: a node booting empty defaults `current_lsn=1` (`persist.c:429`); the first write lazily calls `open_new_segment` which names the file by `current_lsn` → `seg-...001` and **`O_TRUNC`s** it (`persist.c:145`); catch-up then writes entries carrying explicit high LSNs (16…) from offset 0. So "writes from the beginning of the file" = **booted empty + caught up from the watermark**, not an in-place overwrite of recovered data. (`replay_wal` at `persist.c:1422` is correct — it opens existing segments **read-only** and advances `current_lsn` past what it recovers.) + +## OPEN FORKS (what to settle next) +1. **Test procedure — clean vs preserve.** `scripts/cleanup.sh` does `rm -rf .../data/*`; `scripts/start.sh` only creates `data/` if missing (preserves). The on-disk files (named `001`, starting at 16/13) match the **cleaned** case. **Confirm with the user whether the failover test cleans data dirs before restart or restarts with data intact.** + - If preserved + node still returns gapped → **recovery bug** (`replay_wal` failing to restore). + - If cleaned → **catch-up never backfills the prefix + no snapshot** = the architectural gap below. +2. **Catch-up start point (the actual bug site, NOT yet read).** Trace the `CtrlHello` handshake: does a rejoining standby advertise its **real `applied_lsn`** (fresh ⇒ 0 ⇒ should stream from 1) or a **learned/committed watermark** (⇒ streams from 13/16)? `CtrlHelloRequest.appliedLsn @2` is the value the primary streams `[applied+1..committed]` from. This single field decides the fix: "stream from 1 / install a snapshot first" vs "advertise real applied, not learned committed." +3. **Snapshots.** `snap/` is empty everywhere. Is snapshot *creation* implemented/triggered in this build, or only `load_snapshot` (`persist.c:1266`, format `"XCESNAP\0"` + 8-byte lsn header — looks like a STUB carrying only an LSN, NO entities)? If snapshots never carry job entities (the "entity-bearing snapshots" backlog item in ha-convergence-recovery), then ANY WAL pruning/empty-rejoin below committed is unrecoverable BY DESIGN. **Strong candidate for the architectural root.** + +## SECONDARY FINDINGS +- **`term=0` on every WAL entry.** `p->term` (the `hdr.term` source) is **never assigned** the election term anywhere (grep for `p->term =` / `->term =` in persist.c/ha.c returns nothing). So per-entry term is dead. XCE matches logs via the `prev_crc` chain, so possibly benign — but it means **no term-based divergence detection**. Worth a hard look (could let divergent same-LSN entries be treated as matching). +- **Cluster cannot hold a primary** under the test: peer connect `-111` (ECONNREFUSED), RDHUP, rapid term churn 2→5 in ~7 min, elections logged `safe_committed_lsn=X (from 1 granted voters)`. Logged separately in `ha-convergence-recovery/research.md` (2026-06-20 live finding). +- **Crash fixed en-route (unrelated):** `xce_strerror_r` global-buffer-overflow on HA error codes — bug-827, fixed in `src/lib/utils.c` (commit 1aa0761). + +## Is the ha-heartbeat-committed-propagation change implicated? +The loss **predates** it, so it is **not the origin**. But two real concerns it introduces (it writes `last_known_committed_lsn` per-heartbeat, decoupled from WAL possession): +- `ha_enter_real_election` (`ha.c:4137`) truncates WAL above `last_known_committed_lsn` — a heartbeat-fed value now feeds a truncation path (violates the Req 3.1 "advisory-only" boundary I claimed; guarded by `if (committed>0)` so 0 is safe, but still a real boundary leak). +- If the rejoin/catch-up advertised position is derived from `last_known_committed_lsn`, the per-heartbeat freshness would **widen** the prefix gap (fresh node learns committed=13 via heartbeat → advertises 13 → streams from 14). **This is exactly Open Fork #2 — read the CtrlHello advertise path; it also settles aggravation.** +- Candidate-advertise is SAFE: `ha_candidate_advertise_lsn` (`ha.c:2598`) returns bare `persist_durable_floor`, NOT last_known — so a behind node cannot fake election freshness. + +## REPRO / ARTIFACTS (on this host) +- Live deploy with the failed run: `/tmp/xce-ha-deploy/` — controller logs `logs/ctrl-{1,2,3}.log`, WAL `controller/ctrl-N/data/wal/seg-...001.wal`, empty `snap/`. +- Offline WAL decoder (Python, ad-hoc) — walks `wal_entry_hdr_t`, lists `lsn|term|type|payload_len`, flags submits by header timestamp. (To be productized as `waldump`, below.) +- Deployed binary includes controller-status (responderTime) + ha-heartbeat-committed (the `ha.c:4967 "learned committed_lsn=X from heartbeat"` logs are the consume path). + +## NEXT STEPS +1. **Build the WAL/snapshot inspection tooling in `xce_test`** (in progress — see plan below) so clean-vs-preserve and gap-location become a 30-second check each run. +2. **Read the CtrlHello catch-up advertise/start-point** (`config_repl.c` handle_ctrlhello + the standby's CtrlHelloRequest build) — Open Fork #2. +3. **Confirm whether snapshot creation exists** or is a stub — Open Fork #3 (likely the architectural root: no entity-bearing snapshot ⇒ no prefix backfill). +4. Decide fix layer: (a) full-WAL stream `[1..committed]` when a node has no snapshot, (b) implement entity-bearing snapshots + install-before-stream, or (c) prevent empty-rejoin from advancing committed without holding the entries. Likely a **question-the-architecture** moment (catch-up assumes a snapshot safety net that does not exist). + +## TOOLING PLAN (xce_test enhancement — requested by user) +Add read-only debug commands to `xce_test` (`src/controller/testsuite/test.c`; binary links `persist.c`/`job.c` per `controller/meson.build`, so the capnp `restore_job`/`read_Job` decode and `wal_types.h` are available): +- `waldump ` — per node/segment, list every entry `lsn | term | type | payload_len | jobid(submits, via capnp `read_Job`/`decode_Job`)`, flag **LSN gaps** (non-contiguous / missing prefix) and the recovered tip. +- `snapdump ` — decode the `XCESNAP` header (lsn) and report whether it carries entities (it likely does NOT — confirms Fork #3). +- `walcompare ` — diff two nodes' logs by `lsn → crc/jobid` to flag divergence/gaps instantly. +Start with `waldump` (highest value; decoder logic already proven in Python). Register in the `client_commands`/`top_commands` table (`test.c:175/207`). diff --git a/.wolf/buglog.json b/.wolf/buglog.json index b0e3bd89..2615ff53 100644 --- a/.wolf/buglog.json +++ b/.wolf/buglog.json @@ -7,7 +7,7 @@ "error_message": "Significant refactor of ", "file": "CLAUDE.md", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 23\u219230 lines (3 removed)", + "fix": "Rewrote 23→30 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -39,7 +39,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_config_repl.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 14\u219219 lines (2 removed)", + "fix": "Rewrote 14→19 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -55,7 +55,7 @@ "error_message": "Wrong reference: g_tmpdir should be g_instdir", "file": "src/controller/testsuite/test_config_repl.c", "root_cause": "Used \"g_tmpdir\" instead of \"g_instdir\"", - "fix": "Changed g_tmpdir \u2192 g_instdir", + "fix": "Changed g_tmpdir → g_instdir", "tags": [ "auto-detected", "wrong-reference", @@ -87,7 +87,7 @@ "error_message": "Significant refactor of ", "file": "src/docs/config-replication-design.md", "root_cause": "7 lines replaced/restructured", - "fix": "Rewrote 17\u219220 lines (7 removed)", + "fix": "Rewrote 17→20 lines (7 removed)", "tags": [ "auto-detected", "refactor", @@ -135,7 +135,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/config_repl.h", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 6\u21929 lines (2 removed)", + "fix": "Rewrote 6→9 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -151,7 +151,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/config_repl.c", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 11\u21925 lines (5 removed)", + "fix": "Rewrote 11→5 lines (5 removed)", "tags": [ "auto-detected", "refactor", @@ -167,7 +167,7 @@ "error_message": "Significant refactor of ", "file": "src/docs/config-replication-impl-status.md", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 10\u219212 lines (4 removed) | Also: | `handle_config_request()` (primary handler) | \u2705 ; | `fetch_config_from_primary()` (standby fetch) | | Also: | `handle_upgrade_config()` | \u2705 Compiled, gated | ; | `push_config_to_standbys()` | \u26a0 Stub (returns 0)", + "fix": "Rewrote 10→12 lines (4 removed) | Also: | `handle_config_request()` (primary handler) | ✅ ; | `fetch_config_from_primary()` (standby fetch) | | Also: | `handle_upgrade_config()` | ✅ Compiled, gated | ; | `push_config_to_standbys()` | ⚠ Stub (returns 0)", "tags": [ "auto-detected", "refactor", @@ -183,7 +183,7 @@ "error_message": "Significant refactor of ", "file": "src/docs/config-replication-impl-status.md", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 17\u219221 lines (4 removed)", + "fix": "Rewrote 17→21 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -199,7 +199,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 6\u21927 lines (3 removed)", + "fix": "Rewrote 6→7 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -215,7 +215,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-config-bootstrap/brief.md", "root_cause": "6 lines replaced/restructured", - "fix": "Rewrote 13\u219224 lines (6 removed)", + "fix": "Rewrote 13→24 lines (6 removed)", "tags": [ "auto-detected", "refactor", @@ -231,7 +231,7 @@ "error_message": "Significant refactor of ", "file": "../../../../../claude/.claude/plans/prancy-herding-nest.md", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 4\u219210 lines (4 removed)", + "fix": "Rewrote 4→10 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -247,7 +247,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-config-bootstrap/brief.md", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 8\u21929 lines (4 removed)", + "fix": "Rewrote 8→9 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -262,7 +262,7 @@ "timestamp": "2026-05-25T11:05:06.112Z", "error_message": "Missing error handling in send", "file": ".kiro/specs/ha-config-bootstrap/brief.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -278,7 +278,7 @@ "timestamp": "2026-05-25T11:06:26.871Z", "error_message": "Missing error handling in unknown", "file": "../../../../../claude/.claude/plans/prancy-herding-nest.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -295,7 +295,7 @@ "error_message": "Significant refactor of ", "file": "../../../../../claude/.claude/plans/prancy-herding-nest.md", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 23\u219240 lines (2 removed)", + "fix": "Rewrote 23→40 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -326,7 +326,7 @@ "timestamp": "2026-05-25T11:32:18.221Z", "error_message": "Missing error handling in types", "file": "../../../../../claude/.claude/plans/prancy-herding-nest.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -343,7 +343,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-config-bootstrap/brief.md", "root_cause": "36 lines replaced/restructured", - "fix": "Rewrote 63\u219279 lines (36 removed) | Also: 6. **No simultaneous primaries**: Only one node ca; 7. **Majority-commit WAL replication (CP writes)** | Also: - `handle_request_vote()` handler on all peers; - Config tag + LSN comparison for vote decision", + "fix": "Rewrote 63→79 lines (36 removed) | Also: 6. **No simultaneous primaries**: Only one node ca; 7. **Majority-commit WAL replication (CP writes)** | Also: - `handle_request_vote()` handler on all peers; - Config tag + LSN comparison for vote decision", "tags": [ "auto-detected", "refactor", @@ -391,7 +391,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-config-bootstrap/brief.md", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 11\u219228 lines (3 removed)", + "fix": "Rewrote 11→28 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -439,7 +439,7 @@ "error_message": "Significant refactor of ", "file": "../../../../../claude/.claude/plans/prancy-herding-nest.md", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 13\u219228 lines (2 removed)", + "fix": "Rewrote 13→28 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -471,7 +471,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-config-bootstrap/brief.md", "root_cause": "11 lines replaced/restructured", - "fix": "Rewrote 17\u219228 lines (11 removed)", + "fix": "Rewrote 17→28 lines (11 removed)", "tags": [ "auto-detected", "refactor", @@ -487,7 +487,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-config-bootstrap/brief.md", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 28\u219245 lines (4 removed)", + "fix": "Rewrote 28→45 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -519,7 +519,7 @@ "error_message": "Significant refactor of ", "file": "../../../../../claude/.claude/plans/prancy-herding-nest.md", "root_cause": "13 lines replaced/restructured", - "fix": "Rewrote 22\u219235 lines (13 removed) | Also: **Files**: `src/controller/main.c`, `src/lib/netio; 1. **Peer-count validation in `main.c`** (before `", + "fix": "Rewrote 22→35 lines (13 removed) | Also: **Files**: `src/controller/main.c`, `src/lib/netio; 1. **Peer-count validation in `main.c`** (before `", "tags": [ "auto-detected", "refactor", @@ -567,7 +567,7 @@ "error_message": "Significant refactor of ", "file": "../../../../../claude/.claude/plans/prancy-herding-nest.md", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 4\u21927 lines (2 removed)", + "fix": "Rewrote 4→7 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -583,7 +583,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-config-bootstrap/brief.md", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 9\u219211 lines (2 removed)", + "fix": "Rewrote 9→11 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -631,7 +631,7 @@ "error_message": "Significant refactor of ", "file": "../../../../../claude/.claude/plans/prancy-herding-nest.md", "root_cause": "18 lines replaced/restructured", - "fix": "Rewrote 29\u219234 lines (18 removed)", + "fix": "Rewrote 29→34 lines (18 removed)", "tags": [ "auto-detected", "refactor", @@ -679,7 +679,7 @@ "error_message": "Significant refactor of ", "file": "../../../../../claude/.claude/plans/prancy-herding-nest.md", "root_cause": "8 lines replaced/restructured", - "fix": "Rewrote 13\u219229 lines (8 removed) | Also: 8. **Register OP_requestVote handler** in cluster'; switch at line ~195): since vtaskmgr and iomgr run", + "fix": "Rewrote 13→29 lines (8 removed) | Also: 8. **Register OP_requestVote handler** in cluster'; switch at line ~195): since vtaskmgr and iomgr run", "tags": [ "auto-detected", "refactor", @@ -695,7 +695,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-config-bootstrap/brief.md", "root_cause": "9 lines replaced/restructured", - "fix": "Rewrote 13\u219260 lines (9 removed) | Also: 3. **Replace timestamp polling with `EPOLLRDHUP`**; is replaced by an epoll event handler. On `EPOLLRD | Also: - **Split vote recovery**: After failing quorum, s; (using existing `jitter_seed` xorshift64), retry o", + "fix": "Rewrote 13→60 lines (9 removed) | Also: 3. **Replace timestamp polling with `EPOLLRDHUP`**; is replaced by an epoll event handler. On `EPOLLRD | Also: - **Split vote recovery**: After failing quorum, s; (using existing `jitter_seed` xorshift64), retry o", "tags": [ "auto-detected", "refactor", @@ -711,7 +711,7 @@ "error_message": "Significant refactor of ", "file": "../../../../../claude/.claude/plans/prancy-herding-nest.md", "root_cause": "8 lines replaced/restructured", - "fix": "Rewrote 8\u219234 lines (8 removed) | Also: 2. **Replace `ha_check_heartbeat_nocluster()` with; - On `EPOLLRDHUP` from `primary_ctrl_fd`: **close ", + "fix": "Rewrote 8→34 lines (8 removed) | Also: 2. **Replace `ha_check_heartbeat_nocluster()` with; - On `EPOLLRDHUP` from `primary_ctrl_fd`: **close ", "tags": [ "auto-detected", "refactor", @@ -726,7 +726,7 @@ "timestamp": "2026-05-26T13:51:32.274Z", "error_message": "Missing error handling in unknown", "file": "../../../../../claude/.claude/plans/prancy-herding-nest.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block | Also: - [ ] `test_heartbeat_carries_committed_lsn`: heartbeat payl; `committed_lsn`; standby updates `ha.last_known_committed_ls", "tags": [ "auto-detected", @@ -742,7 +742,7 @@ "timestamp": "2026-05-26T13:52:49.044Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-config-bootstrap/brief.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block | Also: // Catch-up gate (Clarification 6): behind nodes must not in; if (c->ha.in_catch_up) {", "tags": [ "auto-detected", @@ -758,7 +758,7 @@ "timestamp": "2026-05-27T03:02:34.912Z", "error_message": "Missing error handling in unknown", "file": "../../../../../claude/.claude/plans/prancy-herding-nest.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block | Also: name, key, applied_lsn, and config_tag immediately after TCP; - [ ] `test_ctrl_hello_registers_standby_in_repl`: handle_ct", "tags": [ "auto-detected", @@ -774,7 +774,7 @@ "timestamp": "2026-05-27T03:03:26.016Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-config-bootstrap/brief.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -791,7 +791,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-config-bootstrap/brief.md", "root_cause": "9 lines replaced/restructured", - "fix": "Rewrote 37\u219251 lines (9 removed)", + "fix": "Rewrote 37→51 lines (9 removed)", "tags": [ "auto-detected", "refactor", @@ -807,7 +807,7 @@ "error_message": "Significant refactor of ", "file": "../../../../../claude/.claude/plans/prancy-herding-nest.md", "root_cause": "8 lines replaced/restructured", - "fix": "Rewrote 8\u219223 lines (8 removed) | Also: - [ ] `test_persistent_conn_single_node_noop`: ha_; when n_peers == 1; primary_ctrl_fd remains -1 (uni", + "fix": "Rewrote 8→23 lines (8 removed) | Also: - [ ] `test_persistent_conn_single_node_noop`: ha_; when n_peers == 1; primary_ctrl_fd remains -1 (uni", "tags": [ "auto-detected", "refactor", @@ -823,7 +823,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-config-bootstrap/brief.md", "root_cause": "10 lines replaced/restructured", - "fix": "Rewrote 34\u219251 lines (10 removed)", + "fix": "Rewrote 34→51 lines (10 removed)", "tags": [ "auto-detected", "refactor", @@ -838,7 +838,7 @@ "timestamp": "2026-05-27T03:56:41.037Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-config-bootstrap/brief.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -855,7 +855,7 @@ "error_message": "Significant refactor of ", "file": "../../../../../claude/.claude/plans/prancy-herding-nest.md", "root_cause": "11 lines replaced/restructured", - "fix": "Rewrote 21\u219228 lines (11 removed)", + "fix": "Rewrote 21→28 lines (11 removed)", "tags": [ "auto-detected", "refactor", @@ -870,7 +870,7 @@ "timestamp": "2026-05-27T03:59:16.145Z", "error_message": "Missing error handling in unknown", "file": "../../../../../claude/.claude/plans/prancy-herding-nest.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -887,7 +887,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-config-bootstrap/brief.md", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 10\u219213 lines (2 removed) | Also: **Replace LSF priority election with Raft-style qu; Five focused changes: | Also: 8. **Hard SSL/cert gate**: Controller refuses to b; 9. **Observable via logs**: Election results, vote", + "fix": "Rewrote 10→13 lines (2 removed) | Also: **Replace LSF priority election with Raft-style qu; Five focused changes: | Also: 8. **Hard SSL/cert gate**: Controller refuses to b; 9. **Observable via logs**: Election results, vote", "tags": [ "auto-detected", "refactor", @@ -902,7 +902,7 @@ "timestamp": "2026-05-27T06:45:34.951Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-config-bootstrap/brief.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -919,7 +919,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-config-bootstrap/brief.md", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 8\u219211 lines (2 removed)", + "fix": "Rewrote 8→11 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -967,7 +967,7 @@ "error_message": "Significant refactor of ", "file": "src/schema/controller.capnp", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 5\u21927 lines (2 removed)", + "fix": "Rewrote 5→7 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -982,7 +982,7 @@ "timestamp": "2026-05-28T02:55:40.362Z", "error_message": "Missing error handling in unknown", "file": "src/schema/controller.capnp", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -999,7 +999,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.h", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 5\u219240 lines (2 removed)", + "fix": "Rewrote 5→40 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -1014,7 +1014,7 @@ "timestamp": "2026-05-28T03:39:14.341Z", "error_message": "Missing error handling in struct", "file": "src/controller/ha.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -1030,7 +1030,7 @@ "timestamp": "2026-05-28T03:40:17.891Z", "error_message": "Missing error handling in struct", "file": "src/include/typemap.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -1047,7 +1047,7 @@ "error_message": "Significant refactor of ", "file": "src/include/typemap.h", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 4\u21926 lines (2 removed)", + "fix": "Rewrote 4→6 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -1095,7 +1095,7 @@ "error_message": "Wrong reference: vcpus should be nres", "file": "src/include/clusterconfig.h", "root_cause": "Used \"vcpus\" instead of \"nres\"", - "fix": "Changed vcpus \u2192 nres", + "fix": "Changed vcpus → nres", "tags": [ "auto-detected", "wrong-reference", @@ -1111,7 +1111,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/cluster.c", "root_cause": "8 lines replaced/restructured", - "fix": "Rewrote 12\u21926 lines (8 removed)", + "fix": "Rewrote 12→6 lines (8 removed)", "tags": [ "auto-detected", "refactor", @@ -1143,7 +1143,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_startup.c", "root_cause": "8 lines replaced/restructured", - "fix": "Rewrote 12\u219218 lines (8 removed)", + "fix": "Rewrote 12→18 lines (8 removed)", "tags": [ "auto-detected", "refactor", @@ -1159,7 +1159,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/cluster.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 109\u2192144 lines (4 removed)", + "fix": "Rewrote 109→144 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -1206,7 +1206,7 @@ "timestamp": "2026-05-28T07:13:42.089Z", "error_message": "Missing error handling in struct", "file": "src/controller/ha.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -1222,7 +1222,7 @@ "timestamp": "2026-05-28T07:14:03.724Z", "error_message": "Missing error handling in unknown", "file": "src/controller/ha.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -1255,7 +1255,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "14 lines replaced/restructured", - "fix": "Rewrote 92\u2192344 lines (14 removed)", + "fix": "Rewrote 92→344 lines (14 removed)", "tags": [ "auto-detected", "refactor", @@ -1271,7 +1271,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/myapp.c", "root_cause": "6 lines replaced/restructured", - "fix": "Rewrote 25\u219249 lines (6 removed)", + "fix": "Rewrote 25→49 lines (6 removed)", "tags": [ "auto-detected", "refactor", @@ -1335,7 +1335,7 @@ "error_message": "Wrong reference: capn_str_null should be str", "file": "src/controller/config_repl.c", "root_cause": "Used \"capn_str_null\" instead of \"str\"", - "fix": "Changed capn_str_null \u2192 str", + "fix": "Changed capn_str_null → str", "tags": [ "auto-detected", "wrong-reference", @@ -1351,7 +1351,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/config_repl.c", "root_cause": "7 lines replaced/restructured", - "fix": "Rewrote 17\u219284 lines (7 removed)", + "fix": "Rewrote 17→84 lines (7 removed)", "tags": [ "auto-detected", "refactor", @@ -1398,7 +1398,7 @@ "timestamp": "2026-05-28T13:34:49.513Z", "error_message": "Missing error handling in unknown", "file": "src/controller/repl.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -1415,7 +1415,7 @@ "error_message": "Significant refactor of ", "file": "src/include/repl.h", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 3\u21924 lines (3 removed) | Also: /* ACK handling \u2014 call from io_worker read callbac; /* Majority-quorum ACK wait \u2014 stub until task 7.3 ", + "fix": "Rewrote 3→4 lines (3 removed) | Also: /* ACK handling — call from io_worker read callbac; /* Majority-quorum ACK wait — stub until task 7.3 ", "tags": [ "auto-detected", "refactor", @@ -1430,7 +1430,7 @@ "timestamp": "2026-05-28T13:52:52.636Z", "error_message": "Missing error handling in unknown", "file": "src/controller/repl.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -1447,7 +1447,7 @@ "error_message": "Wrong reference: in_catch_up should be acked_config_lsn", "file": "src/controller/repl.c", "root_cause": "Used \"in_catch_up\" instead of \"acked_config_lsn\"", - "fix": "Changed in_catch_up \u2192 acked_config_lsn", + "fix": "Changed in_catch_up → acked_config_lsn", "tags": [ "auto-detected", "wrong-reference", @@ -1479,7 +1479,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/persist.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 20\u219229 lines (4 removed)", + "fix": "Rewrote 20→29 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -1495,7 +1495,7 @@ "error_message": "Wrong reference: ha_state_t should be _Atomic", "file": "src/controller/ha.h", "root_cause": "Used \"ha_state_t\" instead of \"_Atomic\"", - "fix": "Changed ha_state_t \u2192 _Atomic", + "fix": "Changed ha_state_t → _Atomic", "tags": [ "auto-detected", "wrong-reference", @@ -1511,7 +1511,7 @@ "error_message": "Wrong reference: socket should be epoll", "file": "src/controller/ha.c", "root_cause": "Used \"socket\" instead of \"epoll\"", - "fix": "Changed socket \u2192 epoll", + "fix": "Changed socket → epoll", "tags": [ "auto-detected", "wrong-reference", @@ -1559,7 +1559,7 @@ "error_message": "Wrong reference: XCE_ERR_STANDBY should be E_XCE_STANDBY", "file": "src/lib/request.c", "root_cause": "Used \"XCE_ERR_STANDBY\" instead of \"E_XCE_STANDBY\"", - "fix": "Changed XCE_ERR_STANDBY \u2192 E_XCE_STANDBY", + "fix": "Changed XCE_ERR_STANDBY → E_XCE_STANDBY", "tags": [ "auto-detected", "wrong-reference", @@ -1575,7 +1575,7 @@ "error_message": "Wrong reference: XCE_ERR_NOT_READY should be E_XCE_NOT_READY", "file": "src/lib/testsuite/test_libxce_retry.c", "root_cause": "Used \"XCE_ERR_NOT_READY\" instead of \"E_XCE_NOT_READY\"", - "fix": "Changed XCE_ERR_NOT_READY \u2192 E_XCE_NOT_READY", + "fix": "Changed XCE_ERR_NOT_READY → E_XCE_NOT_READY", "tags": [ "auto-detected", "wrong-reference", @@ -1590,7 +1590,7 @@ "timestamp": "2026-05-29T03:27:20.556Z", "error_message": "Missing error handling in unknown", "file": "src/controller/ha.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -1606,7 +1606,7 @@ "timestamp": "2026-05-29T03:29:44.753Z", "error_message": "Missing error handling in unknown", "file": "src/controller/config_repl.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -1623,7 +1623,7 @@ "error_message": "Wrong reference: data should be ctrl_hello_reply_t", "file": "src/include/typemap.h", "root_cause": "Used \"data\" instead of \"ctrl_hello_reply_t\"", - "fix": "Changed data \u2192 ctrl_hello_reply_t", + "fix": "Changed data → ctrl_hello_reply_t", "tags": [ "auto-detected", "wrong-reference", @@ -1638,7 +1638,7 @@ "timestamp": "2026-05-29T03:53:48.559Z", "error_message": "Missing error handling in unknown", "file": "src/schema/controller.capnp", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -1654,7 +1654,7 @@ "timestamp": "2026-05-29T03:54:23.205Z", "error_message": "Missing error handling in struct", "file": "src/include/typemap.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block | Also: wal_catchup_entry_t* walcatchupentrymsg;; snapshot_transfer_t* snapshottransfermsg; | Also: /* Forward declarations for types defined later in this file; typedef struct snapshot_transfer_t snapshot_transfer_fw", "tags": [ "auto-detected", @@ -1671,7 +1671,7 @@ "error_message": "Significant refactor of ", "file": "src/include/typemap.h", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 5\u21925 lines (2 removed)", + "fix": "Rewrote 5→5 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -1687,7 +1687,7 @@ "error_message": "Significant refactor of ", "file": "src/schema/controller.capnp", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 5\u21923 lines (2 removed)", + "fix": "Rewrote 5→3 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -1702,7 +1702,7 @@ "timestamp": "2026-05-29T04:04:12.693Z", "error_message": "Missing error handling in unknown", "file": "src/controller/config_repl.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -1719,7 +1719,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/config_repl.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 10\u219212 lines (2 removed)", + "fix": "Rewrote 10→12 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -1751,7 +1751,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 10\u219220 lines (5 removed)", + "fix": "Rewrote 10→20 lines (5 removed)", "tags": [ "auto-detected", "refactor", @@ -1783,7 +1783,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-config-bootstrap/manual-tests.md", "root_cause": "9 lines replaced/restructured", - "fix": "Rewrote 18\u219220 lines (9 removed)", + "fix": "Rewrote 18→20 lines (9 removed)", "tags": [ "auto-detected", "refactor", @@ -1798,7 +1798,7 @@ "timestamp": "2026-05-29T10:59:37.144Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-config-bootstrap/manual-tests.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -1815,7 +1815,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-config-bootstrap/manual-tests.md", "root_cause": "10 lines replaced/restructured", - "fix": "Rewrote 20\u219236 lines (10 removed) | Also: ### 11.1 Concurrent Startup (3 nodes); - Result: PASS / FAIL", + "fix": "Rewrote 20→36 lines (10 removed) | Also: ### 11.1 Concurrent Startup (3 nodes); - Result: PASS / FAIL", "tags": [ "auto-detected", "refactor", @@ -1831,7 +1831,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/config_repl.c", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 29\u219245 lines (5 removed)", + "fix": "Rewrote 29→45 lines (5 removed)", "tags": [ "auto-detected", "refactor", @@ -1847,7 +1847,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_bootstrap.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 4\u21928 lines (2 removed)", + "fix": "Rewrote 4→8 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -1879,7 +1879,7 @@ "error_message": "Wrong reference: nres should be vcpus", "file": "src/include/clusterconfig.h", "root_cause": "Used \"nres\" instead of \"vcpus\"", - "fix": "Changed nres \u2192 vcpus", + "fix": "Changed nres → vcpus", "tags": [ "auto-detected", "wrong-reference", @@ -1895,7 +1895,7 @@ "error_message": "Wrong reference: data should be request_vote_reply_t", "file": "src/include/typemap.h", "root_cause": "Used \"data\" instead of \"request_vote_reply_t\"", - "fix": "Changed data \u2192 request_vote_reply_t", + "fix": "Changed data → request_vote_reply_t", "tags": [ "auto-detected", "wrong-reference", @@ -1911,7 +1911,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "10 lines replaced/restructured", - "fix": "Rewrote 19\u219225 lines (10 removed)", + "fix": "Rewrote 19→25 lines (10 removed)", "tags": [ "auto-detected", "refactor", @@ -1926,7 +1926,7 @@ "timestamp": "2026-05-29T16:36:41.731Z", "error_message": "Missing error handling in unknown", "file": "src/controller/cluster.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -1943,7 +1943,7 @@ "error_message": "Wrong reference: cluster_init_phase2 should be cluster_register_listener", "file": "src/controller/include/cluster.h", "root_cause": "Used \"cluster_init_phase2\" instead of \"cluster_register_listener\"", - "fix": "Changed cluster_init_phase2 \u2192 cluster_register_listener", + "fix": "Changed cluster_init_phase2 → cluster_register_listener", "tags": [ "auto-detected", "wrong-reference", @@ -1975,7 +1975,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/myapp.c", "root_cause": "30 lines replaced/restructured", - "fix": "Rewrote 79\u219261 lines (30 removed)", + "fix": "Rewrote 79→61 lines (30 removed)", "tags": [ "auto-detected", "refactor", @@ -1991,7 +1991,7 @@ "error_message": "Significant refactor of ", "file": "src/include/xce/cluster_def.h", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 8\u219215 lines (5 removed)", + "fix": "Rewrote 8→15 lines (5 removed)", "tags": [ "auto-detected", "refactor", @@ -2007,7 +2007,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/cluster.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 7\u219216 lines (4 removed)", + "fix": "Rewrote 7→16 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -2023,7 +2023,7 @@ "error_message": "Wrong reference: CTRL_PERF_NHASH should be c", "file": "src/controller/cluster.c", "root_cause": "Used \"CTRL_PERF_NHASH\" instead of \"c\"", - "fix": "Changed CTRL_PERF_NHASH \u2192 c", + "fix": "Changed CTRL_PERF_NHASH → c", "tags": [ "auto-detected", "wrong-reference", @@ -2039,7 +2039,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/cluster.c", "root_cause": "36 lines replaced/restructured", - "fix": "Rewrote 77\u219241 lines (36 removed)", + "fix": "Rewrote 77→41 lines (36 removed)", "tags": [ "auto-detected", "refactor", @@ -2055,7 +2055,7 @@ "error_message": "Wrong reference: true should be false", "file": ".claude/skills/xce-build/SKILL.md", "root_cause": "Used \"true\" instead of \"false\"", - "fix": "Changed true \u2192 false", + "fix": "Changed true → false", "tags": [ "auto-detected", "wrong-reference", @@ -2071,7 +2071,7 @@ "error_message": "Wrong reference: 120 should be 300", "file": "src/controller/meson.build", "root_cause": "Used \"120\" instead of \"300\"", - "fix": "Changed 120 \u2192 300", + "fix": "Changed 120 → 300", "tags": [ "auto-detected", "wrong-reference", @@ -2086,7 +2086,7 @@ "timestamp": "2026-05-30T02:37:40.267Z", "error_message": "Missing error handling in unknown", "file": "src/controller/myapp.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -2103,7 +2103,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 15\u219221 lines (2 removed)", + "fix": "Rewrote 15→21 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -2119,7 +2119,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 9\u219213 lines (2 removed)", + "fix": "Rewrote 9→13 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -2135,7 +2135,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/election-iomgr-refactor/design.md", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 10\u219227 lines (3 removed)", + "fix": "Rewrote 10→27 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -2166,8 +2166,8 @@ "timestamp": "2026-05-30T06:38:09.325Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/election-iomgr-refactor/design.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", - "fix": "Added try/catch block | Also: /* After election determines role: */; int is_first_boot = !c->running; /* running=1 only after cl | Also: #### `ha_dispatch_gate` \u2014 CTRL_PRIMARY_COMMITTING case; ```c | Also: ### 3-node startup \u2014 ctrl-a wins (with COMMITTING phase); participant PeerC as ctrl-c endpoint | Also: - **`ha_start_election_async()` is fully non-blocking**: Arm; - *Initial startup* (`ha->phase2_done == 0`): calls `cluster", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", + "fix": "Added try/catch block | Also: /* After election determines role: */; int is_first_boot = !c->running; /* running=1 only after cl | Also: #### `ha_dispatch_gate` — CTRL_PRIMARY_COMMITTING case; ```c | Also: ### 3-node startup — ctrl-a wins (with COMMITTING phase); participant PeerC as ctrl-c endpoint | Also: - **`ha_start_election_async()` is fully non-blocking**: Arm; - *Initial startup* (`ha->phase2_done == 0`): calls `cluster", "tags": [ "auto-detected", "error-handling", @@ -2183,7 +2183,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/election-iomgr-refactor/design.md", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 19\u219226 lines (4 removed) | Also: \u251c\u2500\u2500 ha.h # Add to ha_state_t: clie; \u2502 # ha_repl_ack_re | Also: \u2514\u2500\u2500 myapp.c # Restructured: phase1 + ; # for(;;) retry loop, cluster_init_phase2,", + "fix": "Rewrote 19→26 lines (4 removed) | Also: ├── ha.h # Add to ha_state_t: clie; │ # ha_repl_ack_re | Also: └── myapp.c # Restructured: phase1 + ; # for(;;) retry loop, cluster_init_phase2,", "tags": [ "auto-detected", "refactor", @@ -2199,7 +2199,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/election-iomgr-refactor/design.md", "root_cause": "12 lines replaced/restructured", - "fix": "Rewrote 15\u219220 lines (12 removed) | Also: - `src/controller/ha.h` \u2014 Add `client_ssl_ctx`, `r; - `src/controller/ha.c` \u2014 Main changes above; `ha_ | Also: \u251c\u2500\u2500 delay.c # Add delay_task_ms() (ms; \u2502 # Add delay_task_cancel()", + "fix": "Rewrote 15→20 lines (12 removed) | Also: - `src/controller/ha.h` — Add `client_ssl_ctx`, `r; - `src/controller/ha.c` — Main changes above; `ha_ | Also: ├── delay.c # Add delay_task_ms() (ms; │ # Add delay_task_cancel()", "tags": [ "auto-detected", "refactor", @@ -2231,7 +2231,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/delay.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 53\u219285 lines (3 removed)", + "fix": "Rewrote 53→85 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -2247,7 +2247,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.h", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 7\u21928 lines (4 removed)", + "fix": "Rewrote 7→8 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -2327,7 +2327,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 14\u219217 lines (3 removed)", + "fix": "Rewrote 14→17 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -2359,7 +2359,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_start_election_async.c", "root_cause": "8 lines replaced/restructured", - "fix": "Rewrote 24\u219242 lines (8 removed)", + "fix": "Rewrote 24→42 lines (8 removed)", "tags": [ "auto-detected", "refactor", @@ -2390,7 +2390,7 @@ "timestamp": "2026-05-30T11:08:59.808Z", "error_message": "Missing error handling in XCE_TESTING", "file": "src/controller/ha.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -2406,7 +2406,7 @@ "timestamp": "2026-05-30T11:09:47.905Z", "error_message": "Missing error handling in unknown", "file": "src/controller/ha.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -2423,7 +2423,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "8 lines replaced/restructured", - "fix": "Rewrote 62\u2192164 lines (8 removed) | Also: /* Store candidate own tag/lsn in votes[0] for eva; * n_peers==1 \u2192 n_other==0, so votes[] has 0 entrie | Also: * election_timeout_ctx_t: argument for the delay_t; * ha_elect_evaluate_and_signal when the election d", + "fix": "Rewrote 62→164 lines (8 removed) | Also: /* Store candidate own tag/lsn in votes[0] for eva; * n_peers==1 → n_other==0, so votes[] has 0 entrie | Also: * election_timeout_ctx_t: argument for the delay_t; * ha_elect_evaluate_and_signal when the election d", "tags": [ "auto-detected", "refactor", @@ -2454,7 +2454,7 @@ "timestamp": "2026-05-30T11:43:44.829Z", "error_message": "Missing error handling in XCE_TESTING", "file": "src/controller/ha.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block | Also: /* Push config to standbys with lower config_tag (Req 7.1) *; #ifdef XCE_TESTING", "tags": [ "auto-detected", @@ -2470,7 +2470,7 @@ "timestamp": "2026-05-30T11:44:15.059Z", "error_message": "Missing error handling in XCE_TESTING", "file": "src/controller/ha.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -2486,7 +2486,7 @@ "timestamp": "2026-05-30T11:45:24.946Z", "error_message": "Missing error handling in unknown", "file": "src/controller/cluster.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -2519,7 +2519,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/cluster.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 22\u219232 lines (2 removed)", + "fix": "Rewrote 22→32 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -2631,7 +2631,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "10 lines replaced/restructured", - "fix": "Rewrote 15\u21925 lines (10 removed)", + "fix": "Rewrote 15→5 lines (10 removed)", "tags": [ "auto-detected", "refactor", @@ -2695,7 +2695,7 @@ "error_message": "Wrong reference: third should be second", "file": "src/controller/testsuite/test_ha_repl_ack.c", "root_cause": "Used \"third\" instead of \"second\"", - "fix": "Changed third \u2192 second", + "fix": "Changed third → second", "tags": [ "auto-detected", "wrong-reference", @@ -2727,7 +2727,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/myapp.c", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 10\u219215 lines (5 removed)", + "fix": "Rewrote 10→15 lines (5 removed)", "tags": [ "auto-detected", "refactor", @@ -2759,7 +2759,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "15 lines replaced/restructured", - "fix": "Rewrote 23\u219210 lines (15 removed)", + "fix": "Rewrote 23→10 lines (15 removed)", "tags": [ "auto-detected", "refactor", @@ -2775,7 +2775,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/config_repl.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 5\u21929 lines (2 removed)", + "fix": "Rewrote 5→9 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -2791,7 +2791,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/cluster.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 14\u219216 lines (2 removed)", + "fix": "Rewrote 14→16 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -2823,7 +2823,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "12 lines replaced/restructured", - "fix": "Rewrote 15\u219220 lines (12 removed)", + "fix": "Rewrote 15→20 lines (12 removed)", "tags": [ "auto-detected", "refactor", @@ -2839,7 +2839,7 @@ "error_message": "Significant refactor of ", "file": "src/include/typemap.h", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 4\u21928 lines (2 removed)", + "fix": "Rewrote 4→8 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -2855,7 +2855,7 @@ "error_message": "Significant refactor of ", "file": "src/xce.build/lib/controller.capnp.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 11\u219219 lines (4 removed) | Also: encode_Data_ptr(cs, &(d->sha256), s->sha256);; encode_Data_ptr(cs, &(d->bytes), s->bytes);", + "fix": "Rewrote 11→19 lines (4 removed) | Also: encode_Data_ptr(cs, &(d->sha256), s->sha256);; encode_Data_ptr(cs, &(d->bytes), s->bytes);", "tags": [ "auto-detected", "refactor", @@ -2871,7 +2871,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/cluster.c", "root_cause": "28 lines replaced/restructured", - "fix": "Rewrote 31\u21924 lines (28 removed)", + "fix": "Rewrote 31→4 lines (28 removed)", "tags": [ "auto-detected", "refactor", @@ -2887,7 +2887,7 @@ "error_message": "Significant refactor of ", "file": "src/include/typemap.h", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 8\u219218 lines (4 removed)", + "fix": "Rewrote 8→18 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -2903,7 +2903,7 @@ "error_message": "Significant refactor of ", "file": "src/include/request.h", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 10\u219219 lines (4 removed)", + "fix": "Rewrote 10→19 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -2935,7 +2935,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 5\u219210 lines (2 removed)", + "fix": "Rewrote 5→10 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -2951,7 +2951,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 9\u219211 lines (2 removed)", + "fix": "Rewrote 9→11 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -2966,7 +2966,7 @@ "timestamp": "2026-05-31T04:11:45.534Z", "error_message": "Missing error handling in struct", "file": ".kiro/specs/raft-library/design.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block | Also: Caller->>RaftSM: raft_on_vote_request(state, from, term); alt term < current_term", "tags": [ "auto-detected", @@ -2983,7 +2983,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/raft-library/design.md", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 8\u219226 lines (5 removed) | Also: * raft_init: initialize state for a node.; * All required callbacks (except on_trace and on_v | Also: - C standard library only (`stdint.h`, `pthread.h`; - `raft.c` may include `` for lock-fr | Also: | Threading | `pthread_mutex_t` (caller-owned) | P; | Atomics | None in library | Caller uses atomics ", + "fix": "Rewrote 8→26 lines (5 removed) | Also: * raft_init: initialize state for a node.; * All required callbacks (except on_trace and on_v | Also: - C standard library only (`stdint.h`, `pthread.h`; - `raft.c` may include `` for lock-fr | Also: | Threading | `pthread_mutex_t` (caller-owned) | P; | Atomics | None in library | Caller uses atomics ", "tags": [ "auto-detected", "refactor", @@ -3014,7 +3014,7 @@ "timestamp": "2026-05-31T04:58:21.609Z", "error_message": "Missing error handling in RAFT_H", "file": "src/lib/raft/raft.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -3063,7 +3063,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/testsuite/test_raft.c", "root_cause": "8 lines replaced/restructured", - "fix": "Rewrote 77\u2192107 lines (8 removed)", + "fix": "Rewrote 77→107 lines (8 removed)", "tags": [ "auto-detected", "refactor", @@ -3079,7 +3079,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/raft/raft.c", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 9\u219262 lines (5 removed)", + "fix": "Rewrote 9→62 lines (5 removed)", "tags": [ "auto-detected", "refactor", @@ -3095,7 +3095,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/raft/raft.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 8\u219257 lines (4 removed)", + "fix": "Rewrote 8→57 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -3111,7 +3111,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/raft/raft.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 15\u219238 lines (4 removed)", + "fix": "Rewrote 15→38 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -3127,7 +3127,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/raft/raft_sim.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 6\u21923 lines (2 removed)", + "fix": "Rewrote 6→3 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -3159,7 +3159,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "32 lines replaced/restructured", - "fix": "Rewrote 49\u219216 lines (32 removed)", + "fix": "Rewrote 49→16 lines (32 removed)", "tags": [ "auto-detected", "refactor", @@ -3174,7 +3174,7 @@ "timestamp": "2026-05-31T07:26:39.758Z", "error_message": "Missing error handling in before", "file": "src/controller/ha.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -3191,7 +3191,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 6\u219219 lines (3 removed) | Also: typedef struct {; ha_state_t *ha; | Also: /* \u2500\u2500 Type definitions needed by ha_elect_evaluate; *", + "fix": "Rewrote 6→19 lines (3 removed) | Also: typedef struct {; ha_state_t *ha; | Also: /* ── Type definitions needed by ha_elect_evaluate; *", "tags": [ "auto-detected", "refactor", @@ -3207,7 +3207,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "167 lines replaced/restructured", - "fix": "Rewrote 291\u2192137 lines (167 removed) | Also: /* Clear voted_for: the old primary is dead; any v; * is stale and must not block the new election. */", + "fix": "Rewrote 291→137 lines (167 removed) | Also: /* Clear voted_for: the old primary is dead; any v; * is stale and must not block the new election. */", "tags": [ "auto-detected", "refactor", @@ -3223,7 +3223,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/cluster.c", "root_cause": "12 lines replaced/restructured", - "fix": "Rewrote 28\u219268 lines (12 removed)", + "fix": "Rewrote 28→68 lines (12 removed)", "tags": [ "auto-detected", "refactor", @@ -3239,7 +3239,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 16\u219221 lines (2 removed) | Also: /* Task 6.3: delegate vote reply accumulation to r; * raft_on_vote_reply owns quorum tracking and fire", + "fix": "Rewrote 16→21 lines (2 removed) | Also: /* Task 6.3: delegate vote reply accumulation to r; * raft_on_vote_reply owns quorum tracking and fire", "tags": [ "auto-detected", "refactor", @@ -3255,7 +3255,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "7 lines replaced/restructured", - "fix": "Rewrote 10\u21924 lines (7 removed)", + "fix": "Rewrote 10→4 lines (7 removed)", "tags": [ "auto-detected", "refactor", @@ -3271,7 +3271,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 5\u219214 lines (2 removed)", + "fix": "Rewrote 5→14 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -3284,7 +3284,7 @@ { "id": "bug-206", "timestamp": "2026-05-31T08:40:00.000Z", - "error_message": "test_vote_reply_handler_term_fencing FAIL: Expected 10 Was 3 \u2014 ha->term must be atomically updated to peer term when peer term > ours", + "error_message": "test_vote_reply_handler_term_fencing FAIL: Expected 10 Was 3 — ha->term must be atomically updated to peer term when peer term > ours", "file": "src/controller/ha.c", "root_cause": "vote_reply_msg_handler delegated term fencing entirely to raft_cb_become_follower (via raft_on_vote_reply). In XCE_TESTING builds, vote_reply_dispatch calls xce_vote_reply_hook and returns early, bypassing raft_on_vote_reply and thus never updating ha->term.", "fix": "Added explicit CAS loop in vote_reply_msg_handler before vote_reply_dispatch to advance ha->term when vr_term > ha->term. This applies on both the production path and the XCE_TESTING hook path.", @@ -3306,7 +3306,7 @@ "error_message": "Wrong reference: test_raft_init_zero_peers_returns_einval should be test_raft_init_even_peers_returns_einval", "file": "src/lib/testsuite/test_raft.c", "root_cause": "Used \"test_raft_init_zero_peers_returns_einval\" instead of \"test_raft_init_even_peers_returns_einval\"", - "fix": "Changed test_raft_init_zero_peers_returns_einval \u2192 test_raft_init_even_peers_returns_einval", + "fix": "Changed test_raft_init_zero_peers_returns_einval → test_raft_init_even_peers_returns_einval", "tags": [ "auto-detected", "wrong-reference", @@ -3319,7 +3319,7 @@ { "id": "bug-208", "timestamp": "2026-05-31", - "error_message": "ASAN LeakSanitizer: detected memory leaks in scheduler testsuite \u2014 build exits with error code", + "error_message": "ASAN LeakSanitizer: detected memory leaks in scheduler testsuite — build exits with error code", "file": "src/scheduler/testsuite/test.c", "root_cause": "sched_meet_host_selection() lazily allocates w->host_eval_ctx via build_host_eval_ctx(). Four test functions (test_meet_host_selection_always_true_passes, test_meet_host_selection_always_false_fails, test_meet_host_selection_numeric_match, test_meet_host_selection_string_match) called zarray_fini() without first calling sched_worker_release_host_ctx() to free the lazily-allocated host_eval_ctx.", "fix": "Added sched_worker_release_host_ctx(&w) calls before zarray_fini() in all four affected test functions.", @@ -3356,7 +3356,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "24 lines replaced/restructured", - "fix": "Rewrote 28\u21923 lines (24 removed) | Also: if (my_config_tag == 0) {; INFO(\"ha_start_election_async: no local config (ta", + "fix": "Rewrote 28→3 lines (24 removed) | Also: if (my_config_tag == 0) {; INFO(\"ha_start_election_async: no local config (ta", "tags": [ "auto-detected", "refactor", @@ -3372,7 +3372,7 @@ "error_message": "Wrong reference: 2000 should be 5000", "file": "src/controller/ha.h", "root_cause": "Used \"2000\" instead of \"5000\"", - "fix": "Changed 2000 \u2192 5000", + "fix": "Changed 2000 → 5000", "tags": [ "auto-detected", "wrong-reference", @@ -3388,7 +3388,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/myapp.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 15\u219225 lines (3 removed)", + "fix": "Rewrote 15→25 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -3404,7 +3404,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.h", "root_cause": "8 lines replaced/restructured", - "fix": "Rewrote 10\u219214 lines (8 removed)", + "fix": "Rewrote 10→14 lines (8 removed)", "tags": [ "auto-detected", "refactor", @@ -3420,7 +3420,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 7\u219217 lines (4 removed) | Also: /* Trigger async config fetch from primary so stan; #ifndef XCE_TESTING | Also: handle_request_vote_reply(vctx->ha, vctx->ctx, vct; /* Fix 5 (HIGH): do NOT free vctx here.", + "fix": "Rewrote 7→17 lines (4 removed) | Also: /* Trigger async config fetch from primary so stan; #ifndef XCE_TESTING | Also: handle_request_vote_reply(vctx->ha, vctx->ctx, vct; /* Fix 5 (HIGH): do NOT free vctx here.", "tags": [ "auto-detected", "refactor", @@ -3467,7 +3467,7 @@ "timestamp": "2026-05-31T12:54:26.372Z", "error_message": "Missing error handling in unknown", "file": "src/controller/ha.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -3484,7 +3484,7 @@ "error_message": "Wrong reference: Request_body_nodata should be Request_body_nomsg", "file": "src/controller/ha.c", "root_cause": "Used \"Request_body_nodata\" instead of \"Request_body_nomsg\"", - "fix": "Changed Request_body_nodata \u2192 Request_body_nomsg", + "fix": "Changed Request_body_nodata → Request_body_nomsg", "tags": [ "auto-detected", "wrong-reference", @@ -3500,7 +3500,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 10\u219218 lines (4 removed)", + "fix": "Rewrote 10→18 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -3516,7 +3516,7 @@ "error_message": "Wrong reference: vote_reply_msg_handler should be vote_reply_msg_handler_for_test", "file": "src/controller/ha.c", "root_cause": "Used \"vote_reply_msg_handler\" instead of \"vote_reply_msg_handler_for_test\"", - "fix": "Changed vote_reply_msg_handler \u2192 vote_reply_msg_handler_for_test", + "fix": "Changed vote_reply_msg_handler → vote_reply_msg_handler_for_test", "tags": [ "auto-detected", "wrong-reference", @@ -3532,7 +3532,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/config_repl.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 13\u219236 lines (3 removed)", + "fix": "Rewrote 13→36 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -3548,7 +3548,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/config_repl.c", "root_cause": "20 lines replaced/restructured", - "fix": "Rewrote 36\u219213 lines (20 removed)", + "fix": "Rewrote 36→13 lines (20 removed)", "tags": [ "auto-detected", "refactor", @@ -3563,7 +3563,7 @@ "timestamp": "2026-06-01T01:49:30.537Z", "error_message": "Missing error handling in unknown", "file": "src/controller/cluster.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -3596,7 +3596,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/cluster.c", "root_cause": "42 lines replaced/restructured", - "fix": "Rewrote 64\u219219 lines (42 removed)", + "fix": "Rewrote 64→19 lines (42 removed)", "tags": [ "auto-detected", "refactor", @@ -3611,7 +3611,7 @@ "timestamp": "2026-06-01T02:27:55.942Z", "error_message": "Missing error handling in struct", "file": "src/controller/ha.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -3628,7 +3628,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 10\u219214 lines (5 removed)", + "fix": "Rewrote 10→14 lines (5 removed)", "tags": [ "auto-detected", "refactor", @@ -3644,7 +3644,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/CLAUDE.md", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 5\u219215 lines (5 removed)", + "fix": "Rewrote 5→15 lines (5 removed)", "tags": [ "auto-detected", "refactor", @@ -3660,7 +3660,7 @@ "error_message": "Wrong operator: && should be !=", "file": "src/controller/ha.c", "root_cause": "Used \"&&\" instead of \"!=\"", - "fix": "Changed operator && \u2192 !=", + "fix": "Changed operator && → !=", "tags": [ "auto-detected", "operator-fix", @@ -3676,7 +3676,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "10 lines replaced/restructured", - "fix": "Rewrote 16\u21927 lines (10 removed)", + "fix": "Rewrote 16→7 lines (10 removed)", "tags": [ "auto-detected", "refactor", @@ -3692,7 +3692,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "140 lines replaced/restructured", - "fix": "Rewrote 173\u219211 lines (140 removed)", + "fix": "Rewrote 173→11 lines (140 removed)", "tags": [ "auto-detected", "refactor", @@ -3708,7 +3708,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/myapp.c", "root_cause": "16 lines replaced/restructured", - "fix": "Rewrote 25\u219212 lines (16 removed)", + "fix": "Rewrote 25→12 lines (16 removed)", "tags": [ "auto-detected", "refactor", @@ -3740,7 +3740,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/config_repl.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 31\u219237 lines (2 removed)", + "fix": "Rewrote 31→37 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -3788,7 +3788,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.h", "root_cause": "21 lines replaced/restructured", - "fix": "Rewrote 28\u21925 lines (21 removed)", + "fix": "Rewrote 28→5 lines (21 removed)", "tags": [ "auto-detected", "refactor", @@ -3852,7 +3852,7 @@ "error_message": "Wrong reference: test_ha_connect should be test_ha_ctrlhello", "file": "src/controller/testsuite/controller_test.c", "root_cause": "Used \"test_ha_connect\" instead of \"test_ha_ctrlhello\"", - "fix": "Changed test_ha_connect \u2192 test_ha_ctrlhello", + "fix": "Changed test_ha_connect → test_ha_ctrlhello", "tags": [ "auto-detected", "wrong-reference", @@ -3868,7 +3868,7 @@ "error_message": "Wrong reference: vcpu should be vcpus", "file": "src/tools/deploy-ha.sh", "root_cause": "Used \"vcpu\" instead of \"vcpus\"", - "fix": "Changed vcpu \u2192 vcpus", + "fix": "Changed vcpu → vcpus", "tags": [ "auto-detected", "wrong-reference", @@ -3884,7 +3884,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/cluster.c", "root_cause": "10 lines replaced/restructured", - "fix": "Rewrote 10\u21926 lines (10 removed)", + "fix": "Rewrote 10→6 lines (10 removed)", "tags": [ "auto-detected", "refactor", @@ -3900,7 +3900,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 7\u219222 lines (2 removed)", + "fix": "Rewrote 7→22 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -3916,7 +3916,7 @@ "error_message": "Significant refactor of ", "file": "src/include/typemap.h", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 5\u219210 lines (3 removed)", + "fix": "Rewrote 5→10 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -3932,7 +3932,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/request.c", "root_cause": "32 lines replaced/restructured", - "fix": "Rewrote 49\u219260 lines (32 removed)", + "fix": "Rewrote 49→60 lines (32 removed)", "tags": [ "auto-detected", "refactor", @@ -3948,7 +3948,7 @@ "error_message": "Significant refactor of ", "file": ".claude/skills/xce-deploy-ha/SKILL.md", "root_cause": "7 lines replaced/restructured", - "fix": "Rewrote 17\u219232 lines (7 removed)", + "fix": "Rewrote 17→32 lines (7 removed)", "tags": [ "auto-detected", "refactor", @@ -3964,7 +3964,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/CLAUDE.md", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 6\u21929 lines (3 removed)", + "fix": "Rewrote 6→9 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -3979,7 +3979,7 @@ "timestamp": "2026-06-01T07:28:15.703Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_ha_connection_reuse.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -4012,7 +4012,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_connection_reuse.c", "root_cause": "19 lines replaced/restructured", - "fix": "Rewrote 67\u219286 lines (19 removed)", + "fix": "Rewrote 67→86 lines (19 removed)", "tags": [ "auto-detected", "refactor", @@ -4044,7 +4044,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-heartbeat-liveness/requirements.md", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 10\u219217 lines (5 removed)", + "fix": "Rewrote 10→17 lines (5 removed)", "tags": [ "auto-detected", "refactor", @@ -4060,7 +4060,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 6\u219210 lines (2 removed)", + "fix": "Rewrote 6→10 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -4092,7 +4092,7 @@ "error_message": "Wrong reference: startup_validation should be testsuite_test_ha_liveness", "file": "src/xce.build/build.ninja", "root_cause": "Used \"startup_validation\" instead of \"testsuite_test_ha_liveness\"", - "fix": "Changed startup_validation \u2192 testsuite_test_ha_liveness", + "fix": "Changed startup_validation → testsuite_test_ha_liveness", "tags": [ "auto-detected", "wrong-reference", @@ -4108,7 +4108,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "76 lines replaced/restructured", - "fix": "Rewrote 112\u2192155 lines (76 removed)", + "fix": "Rewrote 112→155 lines (76 removed)", "tags": [ "auto-detected", "refactor", @@ -4124,7 +4124,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_liveness.c", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 21\u219224 lines (5 removed)", + "fix": "Rewrote 21→24 lines (5 removed)", "tags": [ "auto-detected", "refactor", @@ -4140,7 +4140,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_failover.c", "root_cause": "7 lines replaced/restructured", - "fix": "Rewrote 29\u219247 lines (7 removed)", + "fix": "Rewrote 29→47 lines (7 removed)", "tags": [ "auto-detected", "refactor", @@ -4172,7 +4172,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "22 lines replaced/restructured", - "fix": "Rewrote 53\u219221 lines (22 removed)", + "fix": "Rewrote 53→21 lines (22 removed)", "tags": [ "auto-detected", "refactor", @@ -4188,7 +4188,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/cluster.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 8\u219210 lines (2 removed)", + "fix": "Rewrote 8→10 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -4219,7 +4219,7 @@ "timestamp": "2026-06-03T05:20:44.299Z", "error_message": "Missing error handling in lives", "file": ".kiro/specs/ha-config-bootstrap/research.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -4268,7 +4268,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.h", "root_cause": "11 lines replaced/restructured", - "fix": "Rewrote 14\u21923 lines (11 removed)", + "fix": "Rewrote 14→3 lines (11 removed)", "tags": [ "auto-detected", "refactor", @@ -4284,7 +4284,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_role.c", "root_cause": "79 lines replaced/restructured", - "fix": "Rewrote 87\u21923 lines (79 removed)", + "fix": "Rewrote 87→3 lines (79 removed)", "tags": [ "auto-detected", "refactor", @@ -4300,7 +4300,7 @@ "error_message": "Wrong reference: test_am_i_primary should be test_ha_cluster", "file": "src/controller/testsuite/controller_test.c", "root_cause": "Used \"test_am_i_primary\" instead of \"test_ha_cluster\"", - "fix": "Changed test_am_i_primary \u2192 test_ha_cluster", + "fix": "Changed test_am_i_primary → test_ha_cluster", "tags": [ "auto-detected", "wrong-reference", @@ -4316,7 +4316,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.h", "root_cause": "8 lines replaced/restructured", - "fix": "Rewrote 10\u21923 lines (8 removed)", + "fix": "Rewrote 10→3 lines (8 removed)", "tags": [ "auto-detected", "refactor", @@ -4332,7 +4332,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "21 lines replaced/restructured", - "fix": "Rewrote 26\u21924 lines (21 removed) | Also: /* ha_check_heartbeat_nocluster() removed from the; * ha_watchdog_thread(). XCE_TESTING shim preserve", + "fix": "Rewrote 26→4 lines (21 removed) | Also: /* ha_check_heartbeat_nocluster() removed from the; * ha_watchdog_thread(). XCE_TESTING shim preserve", "tags": [ "auto-detected", "refactor", @@ -4348,7 +4348,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_role.c", "root_cause": "63 lines replaced/restructured", - "fix": "Rewrote 68\u21924 lines (63 removed) | Also: /* SIGUSR1 must set failover_requested via async-s; static void test_sigusr1_sets_failover_requested(v | Also: RUN_TEST(test_watchdog_starts_and_stops);; RUN_TEST(test_heartbeat_no_action_when_no_primary)", + "fix": "Rewrote 68→4 lines (63 removed) | Also: /* SIGUSR1 must set failover_requested via async-s; static void test_sigusr1_sets_failover_requested(v | Also: RUN_TEST(test_watchdog_starts_and_stops);; RUN_TEST(test_heartbeat_no_action_when_no_primary)", "tags": [ "auto-detected", "refactor", @@ -4364,7 +4364,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-config-bootstrap/manual-tests.md", "root_cause": "6 lines replaced/restructured", - "fix": "Rewrote 8\u21928 lines (6 removed) | Also: **Expected within 1 second on both standbys:**; **Expected on the winner (needs 1 vote from the ot | Also: defer_config_fetch=1; Config fetch successful from primary (tag=...) | Also: Date:; Commit: (run: cd /home/rshen/projects/xce/worktree", + "fix": "Rewrote 8→8 lines (6 removed) | Also: **Expected within 1 second on both standbys:**; **Expected on the winner (needs 1 vote from the ot | Also: defer_config_fetch=1; Config fetch successful from primary (tag=...) | Also: Date:; Commit: (run: cd /home/rshen/projects/xce/worktree", "tags": [ "auto-detected", "refactor", @@ -4380,7 +4380,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-config-bootstrap/manual-tests.md", "root_cause": "95 lines replaced/restructured", - "fix": "Rewrote 158\u219265 lines (95 removed) | Also: - [ ] `ha-term.bin` file exists after election; - [ ] Magic bytes are `0xDEADBEEFCAFEBABE` (little", + "fix": "Rewrote 158→65 lines (95 removed) | Also: - [ ] `ha-term.bin` file exists after election; - [ ] Magic bytes are `0xDEADBEEFCAFEBABE` (little", "tags": [ "auto-detected", "refactor", @@ -4396,7 +4396,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "6 lines replaced/restructured", - "fix": "Rewrote 45\u219262 lines (6 removed)", + "fix": "Rewrote 45→62 lines (6 removed)", "tags": [ "auto-detected", "refactor", @@ -4411,7 +4411,7 @@ "timestamp": "2026-06-04T00:24:46.380Z", "error_message": "Missing error handling in unknown", "file": "src/controller/myapp.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -4428,7 +4428,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/CLAUDE.md", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 5\u21927 lines (4 removed)", + "fix": "Rewrote 5→7 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -4443,7 +4443,7 @@ "timestamp": "2026-06-04T09:46:14.610Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-repl-hardening/design.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -4476,7 +4476,7 @@ "error_message": "Significant refactor of ", "file": "src/schema/controller.capnp", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 4\u21924 lines (3 removed)", + "fix": "Rewrote 4→4 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -4492,7 +4492,7 @@ "error_message": "Significant refactor of ", "file": "src/include/typemap.h", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 5\u21927 lines (4 removed)", + "fix": "Rewrote 5→7 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -4508,7 +4508,7 @@ "error_message": "Significant refactor of ", "file": "src/schema/controller.capnp", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 4\u219213 lines (2 removed)", + "fix": "Rewrote 4→13 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -4524,7 +4524,7 @@ "error_message": "Significant refactor of ", "file": "src/include/typemap.h", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 4\u21928 lines (2 removed)", + "fix": "Rewrote 4→8 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -4539,8 +4539,8 @@ "timestamp": "2026-06-04T14:33:03.464Z", "error_message": "Missing error handling in XCE_TESTING", "file": "src/controller/testsuite/test_config_repl.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", - "fix": "Added try/catch block | Also: /* \u2500\u2500 Task 2.1: OPCode_walCatchupEntry handler \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500; #ifdef XCE_TESTING | Also: /* Task 2.1: OPCode_walCatchupEntry handler */; RUN_TEST(test_wal_catchup_entry_advances_applied_lsn);", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", + "fix": "Added try/catch block | Also: /* ── Task 2.1: OPCode_walCatchupEntry handler ─────────────; #ifdef XCE_TESTING | Also: /* Task 2.1: OPCode_walCatchupEntry handler */; RUN_TEST(test_wal_catchup_entry_advances_applied_lsn);", "tags": [ "auto-detected", "error-handling", @@ -4555,7 +4555,7 @@ "timestamp": "2026-06-04T14:41:35.073Z", "error_message": "Missing error handling in XCE_TESTING", "file": "src/controller/cluster.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block | Also: case OPCode_walCatchupEntry:; /* Task 2.1: standby applies a single WAL catch-up entry fro", "tags": [ "auto-detected", @@ -4572,7 +4572,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/cluster.c", "root_cause": "57 lines replaced/restructured", - "fix": "Rewrote 82\u219216 lines (57 removed)", + "fix": "Rewrote 82→16 lines (57 removed)", "tags": [ "auto-detected", "refactor", @@ -4587,7 +4587,7 @@ "timestamp": "2026-06-04T14:45:42.730Z", "error_message": "Missing error handling in unknown", "file": "src/controller/ha.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -4603,7 +4603,7 @@ "timestamp": "2026-06-04T14:46:44.652Z", "error_message": "Missing error handling in unknown", "file": "src/controller/ha.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block | Also: #include \"persist.h\" /* persist_t, apply_wal_entry */; #include \"typemap.h\" /* wal_catchup_entry_t */", "tags": [ "auto-detected", @@ -4619,7 +4619,7 @@ "timestamp": "2026-06-04T15:09:54.280Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_config_repl.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -4636,7 +4636,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "22 lines replaced/restructured", - "fix": "Rewrote 58\u2192114 lines (22 removed)", + "fix": "Rewrote 58→114 lines (22 removed)", "tags": [ "auto-detected", "refactor", @@ -4652,7 +4652,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/cluster.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 10\u219215 lines (3 removed)", + "fix": "Rewrote 10→15 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -4684,7 +4684,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 44\u219289 lines (4 removed) | Also: /* Decode stateReport and call convergence tracker; memset(&sr, 0, sizeof(sr));", + "fix": "Rewrote 44→89 lines (4 removed) | Also: /* Decode stateReport and call convergence tracker; memset(&sr, 0, sizeof(sr));", "tags": [ "auto-detected", "refactor", @@ -4732,7 +4732,7 @@ "error_message": "Wrong reference: test_evaluate_repl_action_stream_wal_from_zero should be test_evaluate_repl_action_send_snapshot_lsn_zero", "file": "src/controller/testsuite/test_ha_role.c", "root_cause": "Used \"test_evaluate_repl_action_stream_wal_from_zero\" instead of \"test_evaluate_repl_action_send_snapshot_lsn_zero\"", - "fix": "Changed test_evaluate_repl_action_stream_wal_from_zero \u2192 test_evaluate_repl_action_send_snapshot_lsn_zero", + "fix": "Changed test_evaluate_repl_action_stream_wal_from_zero → test_evaluate_repl_action_send_snapshot_lsn_zero", "tags": [ "auto-detected", "wrong-reference", @@ -4763,7 +4763,7 @@ "timestamp": "2026-06-04T17:34:23.150Z", "error_message": "Missing error handling in XCE_TESTING", "file": "src/controller/testsuite/test_config_repl.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -4780,7 +4780,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 71\u219293 lines (4 removed)", + "fix": "Rewrote 71→93 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -4795,7 +4795,7 @@ "timestamp": "2026-06-04T17:40:05.798Z", "error_message": "Missing error handling in XCE_TESTING", "file": "src/controller/config_repl.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -4844,7 +4844,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 6\u219215 lines (2 removed)", + "fix": "Rewrote 6→15 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -4859,7 +4859,7 @@ "timestamp": "2026-06-04T23:21:45.737Z", "error_message": "Missing error handling in XCE_TESTING", "file": "src/controller/config_repl.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -4876,7 +4876,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_config_repl.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 9\u219212 lines (2 removed)", + "fix": "Rewrote 9→12 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -4892,7 +4892,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-repl-hardening/tasks.md", "root_cause": "9 lines replaced/restructured", - "fix": "Rewrote 9\u219235 lines (9 removed)", + "fix": "Rewrote 9→35 lines (9 removed)", "tags": [ "auto-detected", "refactor", @@ -4908,7 +4908,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 7\u219210 lines (4 removed) | Also: /* Task 3.5: snapshot transfer.; * Creates a checkpoint in the delaymgr thread (not", + "fix": "Rewrote 7→10 lines (4 removed) | Also: /* Task 3.5: snapshot transfer.; * Creates a checkpoint in the delaymgr thread (not", "tags": [ "auto-detected", "refactor", @@ -4988,7 +4988,7 @@ "error_message": "Wrong reference: uintptr_t should be void", "file": "src/controller/testsuite/test_ha_evaluate_committing.c", "root_cause": "Used \"uintptr_t\" instead of \"void\"", - "fix": "Changed uintptr_t \u2192 void", + "fix": "Changed uintptr_t → void", "tags": [ "auto-detected", "wrong-reference", @@ -5004,7 +5004,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/config_repl.c", "root_cause": "149 lines replaced/restructured", - "fix": "Rewrote 207\u219238 lines (149 removed)", + "fix": "Rewrote 207→38 lines (149 removed)", "tags": [ "auto-detected", "refactor", @@ -5036,7 +5036,7 @@ "error_message": "Wrong reference: 10000 should be 60000", "file": "src/controller/ha.h", "root_cause": "Used \"10000\" instead of \"60000\"", - "fix": "Changed 10000 \u2192 60000", + "fix": "Changed 10000 → 60000", "tags": [ "auto-detected", "wrong-reference", @@ -5052,7 +5052,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 24\u219242 lines (5 removed)", + "fix": "Rewrote 24→42 lines (5 removed)", "tags": [ "auto-detected", "refactor", @@ -5068,7 +5068,7 @@ "error_message": "Wrong reference: 10000 should be HA_DEFAULT_COMMITTING_TIMEOUT_MS", "file": "src/controller/testsuite/test_ha_evaluate_committing.c", "root_cause": "Used \"10000\" instead of \"HA_DEFAULT_COMMITTING_TIMEOUT_MS\"", - "fix": "Changed 10000 \u2192 HA_DEFAULT_COMMITTING_TIMEOUT_MS", + "fix": "Changed 10000 → HA_DEFAULT_COMMITTING_TIMEOUT_MS", "tags": [ "auto-detected", "wrong-reference", @@ -5083,7 +5083,7 @@ "timestamp": "2026-06-05T05:10:13.646Z", "error_message": "Missing error handling in unknown", "file": "src/controller/config_repl.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -5099,7 +5099,7 @@ "timestamp": "2026-06-05T05:39:48.707Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_config_repl.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -5115,7 +5115,7 @@ "timestamp": "2026-06-05T06:50:00Z", "error_message": "3-node startup: majority convergence never fires; cluster falls through to committing_timeout degraded-mode WARN", "file": "src/controller/ha.c", - "root_cause": "ha->n_peers was set to candidates_count (3, total including self) in ha_start_election_async. ha_check_majority_convergence loops i=0..n_peers-1 checking standby_converged[], but standby_converged[] is indexed by votes_idx (relative, self excluded: 0..n_other-1). With n_peers=3, it checked standby_converged[2] which is never set, so n_converged (2) < n_peers (3) always \u2014 majority convergence path could never succeed.", + "root_cause": "ha->n_peers was set to candidates_count (3, total including self) in ha_start_election_async. ha_check_majority_convergence loops i=0..n_peers-1 checking standby_converged[], but standby_converged[] is indexed by votes_idx (relative, self excluded: 0..n_other-1). With n_peers=3, it checked standby_converged[2] which is never set, so n_converged (2) < n_peers (3) always — majority convergence path could never succeed.", "fix": "Changed ha->n_peers = n_peers to ha->n_peers = n_other (= n_peers-1 = other-node count). Moved the assignment after n_other is computed. ha_repl_ack_received quorum formula (ha->n_peers+2)/2 still yields correct majority for both old and new value.", "tags": [ "ha", @@ -5134,7 +5134,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 7\u219212 lines (2 removed) | Also: /* Disarm any existing poll ctx */; free(old);", + "fix": "Rewrote 7→12 lines (2 removed) | Also: /* Disarm any existing poll ctx */; free(old);", "tags": [ "auto-detected", "refactor", @@ -5149,7 +5149,7 @@ "timestamp": "2026-06-05T07:23:00.000Z", "error_message": "AddressSanitizer: attempting double-free on sctx in ha_connect_to_peer", "file": "src/controller/ha.c", - "root_cause": "endpoint_fini calls FREEUP(ep->data) which frees sctx, then ha_connect_to_peer also calls free(sctx) \u2014 double-free at lines 3594-3595 and 3600-3601", + "root_cause": "endpoint_fini calls FREEUP(ep->data) which frees sctx, then ha_connect_to_peer also calls free(sctx) — double-free at lines 3594-3595 and 3600-3601", "fix": "Set ep->data = NULL before endpoint_fini so FREEUP becomes a no-op; then free(sctx) explicitly in the error paths", "tags": [ "double-free", @@ -5165,7 +5165,7 @@ { "id": "bug-007", "timestamp": "2026-06-05T07:23:00.000Z", - "error_message": "AddressSanitizer: heap-use-after-free in liveness_poll_fn at ha.c:3293 \u2014 reading ctx->cancelled after ha_arm_heartbeat_timeout_vtask freed the old ctx", + "error_message": "AddressSanitizer: heap-use-after-free in liveness_poll_fn at ha.c:3293 — reading ctx->cancelled after ha_arm_heartbeat_timeout_vtask freed the old ctx", "file": "src/controller/ha.c", "root_cause": "ha_arm_heartbeat_timeout_vtask freed the old liveness_poll_ctx_t immediately after setting cancelled=1, but the delaymgr task was already in flight reading ctx->cancelled=0 and then continued to read ctx->ha after free", "fix": "ha_arm_heartbeat_timeout_vtask no longer frees old ctx; liveness_poll_fn now frees ctx when it fires with cancelled=1 (guarded by #ifndef XCE_TESTING to avoid bad-free on stack-allocated test ctxs)", @@ -5187,7 +5187,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 41\u219287 lines (2 removed)", + "fix": "Rewrote 41→87 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -5203,7 +5203,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "6 lines replaced/restructured", - "fix": "Rewrote 8\u21925 lines (6 removed)", + "fix": "Rewrote 8→5 lines (6 removed)", "tags": [ "auto-detected", "refactor", @@ -5219,7 +5219,7 @@ "error_message": "Wrong reference: DEBUG2 should be INFO", "file": "src/controller/ha.c", "root_cause": "Used \"DEBUG2\" instead of \"INFO\"", - "fix": "Changed DEBUG2 \u2192 INFO", + "fix": "Changed DEBUG2 → INFO", "tags": [ "auto-detected", "wrong-reference", @@ -5283,7 +5283,7 @@ "error_message": "Significant refactor of ", "file": "src/docs/cluster-config-split-analysis.md", "root_cause": "8 lines replaced/restructured", - "fix": "Rewrote 11\u219221 lines (8 removed) | Also: ### Phase 1: Schema split (wire-compatible); 4. **Keep the old `ClusterConfig` fields intact** ", + "fix": "Rewrote 11→21 lines (8 removed) | Also: ### Phase 1: Schema split (wire-compatible); 4. **Keep the old `ClusterConfig` fields intact** ", "tags": [ "auto-detected", "refactor", @@ -5299,7 +5299,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/myapp.c", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 27\u219236 lines (5 removed)", + "fix": "Rewrote 27→36 lines (5 removed)", "tags": [ "auto-detected", "refactor", @@ -5315,7 +5315,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/raft/raft.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 9\u219215 lines (4 removed)", + "fix": "Rewrote 9→15 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -5331,7 +5331,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "37 lines replaced/restructured", - "fix": "Rewrote 41\u21924 lines (37 removed)", + "fix": "Rewrote 41→4 lines (37 removed)", "tags": [ "auto-detected", "refactor", @@ -5471,7 +5471,7 @@ "error_message": "Wrong reference: last_heartbeat should be voted_for_term", "file": "src/controller/ha.h", "root_cause": "Used \"last_heartbeat\" instead of \"voted_for_term\"", - "fix": "Changed last_heartbeat \u2192 voted_for_term", + "fix": "Changed last_heartbeat → voted_for_term", "tags": [ "auto-detected", "wrong-reference", @@ -5487,7 +5487,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/myapp.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 13\u219217 lines (4 removed)", + "fix": "Rewrote 13→17 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -5538,7 +5538,7 @@ "timestamp": "2026-06-10T01:36:59.112Z", "error_message": "Missing error handling in unknown", "file": "src/schema/controller.capnp", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -5554,7 +5554,7 @@ "timestamp": "2026-06-10T01:37:29.771Z", "error_message": "Missing error handling in struct", "file": "src/include/typemap.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -5571,7 +5571,7 @@ "error_message": "Significant refactor of ", "file": "src/include/typemap.h", "root_cause": "6 lines replaced/restructured", - "fix": "Rewrote 16\u219231 lines (6 removed)", + "fix": "Rewrote 16→31 lines (6 removed)", "tags": [ "auto-detected", "refactor", @@ -5586,7 +5586,7 @@ "timestamp": "2026-06-10T01:38:31.262Z", "error_message": "Missing error handling in unknown", "file": "src/include/request.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -5618,7 +5618,7 @@ "timestamp": "2026-06-10T01:39:31.683Z", "error_message": "Missing error handling in unknown", "file": "src/lib/request.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block | Also: if (req->body_which == Request_body_walcatchupentrymsg; && req->body.walcatchupentrymsg != NULL) { | Also: /* wal_entry_to_wire: fill a wire WalCatchupEntry from a C e; * fields and raw payload bytes. Sub-entry nesting is one le", "tags": [ "auto-detected", @@ -5653,7 +5653,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/config_repl.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 29\u219232 lines (4 removed)", + "fix": "Rewrote 29→32 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -5669,7 +5669,7 @@ "error_message": "Wrong reference: payload should be raw_payload", "file": "src/controller/config_repl.c", "root_cause": "Used \"payload\" instead of \"raw_payload\"", - "fix": "Changed payload \u2192 raw_payload", + "fix": "Changed payload → raw_payload", "tags": [ "auto-detected", "wrong-reference", @@ -5685,7 +5685,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 13\u219216 lines (2 removed)", + "fix": "Rewrote 13→16 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -5701,7 +5701,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_role.c", "root_cause": "7 lines replaced/restructured", - "fix": "Rewrote 49\u2192137 lines (7 removed)", + "fix": "Rewrote 49→137 lines (7 removed)", "tags": [ "auto-detected", "refactor", @@ -5716,7 +5716,7 @@ "timestamp": "2026-06-10T01:49:36.997Z", "error_message": "Missing error handling in unknown", "file": "src/controller/ha.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -5785,7 +5785,7 @@ "error_message": "Significant refactor of ", "file": "src/scheduler/src/scheduler.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 4\u21926 lines (4 removed)", + "fix": "Rewrote 4→6 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -5838,7 +5838,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 9\u219215 lines (4 removed)", + "fix": "Rewrote 9→15 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -5870,7 +5870,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/cluster.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 8\u219213 lines (3 removed) | Also: /* Req 4.1: ctrlhello counts as primary activity \u2014; if ((ctrl_role_t)atomic_load_explicit(&c->ha.role,", + "fix": "Rewrote 8→13 lines (3 removed) | Also: /* Req 4.1: ctrlhello counts as primary activity —; if ((ctrl_role_t)atomic_load_explicit(&c->ha.role,", "tags": [ "auto-detected", "refactor", @@ -5886,7 +5886,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/config_repl.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 6\u219212 lines (3 removed)", + "fix": "Rewrote 6→12 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -5901,7 +5901,7 @@ "timestamp": "2026-06-10T07:56:21.347Z", "error_message": "Missing error handling in XCE_TESTING", "file": "src/controller/testsuite/test_repl_unit.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block | Also: /* =========================================================; * Task 1.1: repl_broadcast send path tests", "tags": [ "auto-detected", @@ -5918,7 +5918,7 @@ "error_message": "Wrong reference: WAL_JOB should be WAL_JOB_SUBMIT", "file": "src/controller/testsuite/test_repl_unit.c", "root_cause": "Used \"WAL_JOB\" instead of \"WAL_JOB_SUBMIT\"", - "fix": "Changed WAL_JOB \u2192 WAL_JOB_SUBMIT", + "fix": "Changed WAL_JOB → WAL_JOB_SUBMIT", "tags": [ "auto-detected", "wrong-reference", @@ -5933,7 +5933,7 @@ "timestamp": "2026-06-10T07:58:15.167Z", "error_message": "Missing error handling in XCE_TESTING", "file": "src/controller/repl.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -5950,7 +5950,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/repl.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 42\u219265 lines (3 removed)", + "fix": "Rewrote 42→65 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -5966,7 +5966,7 @@ "error_message": "Wrong reference: DEBUG3 should be TRACE", "file": "src/controller/repl.c", "root_cause": "Used \"DEBUG3\" instead of \"TRACE\"", - "fix": "Changed DEBUG3 \u2192 TRACE", + "fix": "Changed DEBUG3 → TRACE", "tags": [ "auto-detected", "wrong-reference", @@ -5982,7 +5982,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.h", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 12\u219216 lines (3 removed)", + "fix": "Rewrote 12→16 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -5998,7 +5998,7 @@ "error_message": "Wrong reference: request should be ep", "file": "src/controller/cluster.c", "root_cause": "Used \"request\" instead of \"ep\"", - "fix": "Changed request \u2192 ep", + "fix": "Changed request → ep", "tags": [ "auto-detected", "wrong-reference", @@ -6013,7 +6013,7 @@ "timestamp": "2026-06-10T08:47:48.296Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_ha_role.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block | Also: /* Task 3.3 (ha-wal-replication): dispatch gate standby-role; RUN_TEST(test_dispatch_gate_allows_wal_catchup_in_standby_ro", "tags": [ "auto-detected", @@ -6029,7 +6029,7 @@ "timestamp": "2026-06-10T08:50:27.397Z", "error_message": "Missing error handling in struct", "file": "src/include/typemap.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -6061,7 +6061,7 @@ "timestamp": "2026-06-10T09:59:08.979Z", "error_message": "Missing error handling in XCE_TESTING", "file": "src/controller/testsuite/test_p2_integration.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -6096,7 +6096,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_p2_integration.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 50\u219272 lines (4 removed)", + "fix": "Rewrote 50→72 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -6112,7 +6112,7 @@ "error_message": "Wrong reference: test_catchup_full_lifecycle_integration should be test_single_node_committed_lsn_unchanged", "file": "src/controller/testsuite/test_p2_integration.c", "root_cause": "Used \"test_catchup_full_lifecycle_integration\" instead of \"test_single_node_committed_lsn_unchanged\"", - "fix": "Changed test_catchup_full_lifecycle_integration \u2192 test_single_node_committed_lsn_unchanged", + "fix": "Changed test_catchup_full_lifecycle_integration → test_single_node_committed_lsn_unchanged", "tags": [ "auto-detected", "wrong-reference", @@ -6127,7 +6127,7 @@ "timestamp": "2026-06-10T10:37:49.028Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_repl_unit.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -6160,7 +6160,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/steering/product.md", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 13\u219222 lines (5 removed)", + "fix": "Rewrote 13→22 lines (5 removed)", "tags": [ "auto-detected", "refactor", @@ -6192,7 +6192,7 @@ "error_message": "Significant refactor of ", "file": "CLAUDE.md", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 5\u21925 lines (4 removed)", + "fix": "Rewrote 5→5 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -6224,7 +6224,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/CLAUDE.md", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 6\u21925 lines (3 removed) | Also: ### Controllers; - Struct: `type {Resource}Controller struct{}`", + "fix": "Rewrote 6→5 lines (3 removed) | Also: ### Controllers; - Struct: `type {Resource}Controller struct{}`", "tags": [ "auto-detected", "refactor", @@ -6256,7 +6256,7 @@ "error_message": "Wrong reference: signal should be stdio", "file": "src/lib/delay.c", "root_cause": "Used \"signal\" instead of \"stdio\"", - "fix": "Changed signal \u2192 stdio", + "fix": "Changed signal → stdio", "tags": [ "auto-detected", "wrong-reference", @@ -6271,7 +6271,7 @@ "timestamp": "2026-06-11T03:30:54.341Z", "error_message": "Missing error handling in XCE_TESTING", "file": "src/controller/ha.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -6320,7 +6320,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/job.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 15\u219228 lines (2 removed)", + "fix": "Rewrote 15→28 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -6352,7 +6352,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-consistency-invariants/research.md", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 7\u21927 lines (3 removed)", + "fix": "Rewrote 7→7 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -6387,7 +6387,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-consistency-invariants/requirements.md", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 3\u21924 lines (2 removed)", + "fix": "Rewrote 3→4 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -6466,7 +6466,7 @@ "timestamp": "2026-06-11T07:53:43.868Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-consistency-invariants/research.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -6498,7 +6498,7 @@ "timestamp": "2026-06-11T08:46:20.815Z", "error_message": "Missing error handling in unknown", "file": ".kiro/steering/roadmap.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -6579,7 +6579,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/steering/roadmap.md", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 6\u21928 lines (4 removed)", + "fix": "Rewrote 6→8 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -6594,7 +6594,7 @@ "timestamp": "2026-06-11T13:30:00Z", "error_message": "test_commit_engine fixture: Expected 0 Was 1 from repl_add_standby", "file": "src/controller/testsuite/test_commit_engine.c", - "root_cause": "repl_add_standby returns the slot index (>=0) on success, not 0 \u2014 second add legitimately returns 1", + "root_cause": "repl_add_standby returns the slot index (>=0) on success, not 0 — second add legitimately returns 1", "fix": "fixture asserts TEST_ASSERT_EQUAL_INT(i, repl_add_standby(...)) per standby slot", "tags": [ "controller", @@ -6611,8 +6611,8 @@ "timestamp": "2026-06-11T13:35:00Z", "error_message": "3 pre-existing controller testsuite failures masked by exit code 0: test_committed_lsn_does_not_advance_on_timeout (-110), test_repl_ack_after_promote_gate_allows_jobsub (-10013), test_rdhup_primary_dispatch_gate_opens (-10013)", "file": "src/controller/testsuite/controller_test.c", - "root_cause": "each test group calls UNITY_BEGIN/UNITY_END internally, resetting Unity counters \u2014 main()'s final UNITY_END returns 0 unless the LAST group fails, so meson reported the suite green despite 3 real failures; Unity also misattributes failure FILE names to whichever file's UNITY_BEGIN ran (e.g. reports test_repl_unit.c for a test living in test_p2_integration.c)", - "fix": "not fixed (out of task 1.1 boundary) \u2014 documented; failures exist identically on baseline without ha-commit-engine changes (verified by stash/rebuild). persist_wal_sync_append returns -ETIMEDOUT while the test expects rc==0 with sync_rc==-ETIMEDOUT; the two gate tests fail ha_has_write_quorum (standby_endpoints empty with n_peers=2)", + "root_cause": "each test group calls UNITY_BEGIN/UNITY_END internally, resetting Unity counters — main()'s final UNITY_END returns 0 unless the LAST group fails, so meson reported the suite green despite 3 real failures; Unity also misattributes failure FILE names to whichever file's UNITY_BEGIN ran (e.g. reports test_repl_unit.c for a test living in test_p2_integration.c)", + "fix": "not fixed (out of task 1.1 boundary) — documented; failures exist identically on baseline without ha-commit-engine changes (verified by stash/rebuild). persist_wal_sync_append returns -ETIMEDOUT while the test expects rc==0 with sync_rc==-ETIMEDOUT; the two gate tests fail ha_has_write_quorum (standby_endpoints empty with n_peers=2)", "tags": [ "controller", "testsuite", @@ -6631,7 +6631,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/commit.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 11\u219221 lines (2 removed) | Also: (void)timeout_ms; /* RETAIN never honors the timeo; * via commit_on_ack (Req 2.1). */", + "fix": "Rewrote 11→21 lines (2 removed) | Also: (void)timeout_ms; /* RETAIN never honors the timeo; * via commit_on_ack (Req 2.1). */", "tags": [ "auto-detected", "refactor", @@ -6646,8 +6646,8 @@ "timestamp": "2026-06-11T15:28:35.990Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_commit_engine.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", - "fix": "Added try/catch block | Also: /* \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550; * Task 3.1 \u2014 membership re-evaluation (commit_on_membership_ | Also: /* Task 3.1 \u2014 membership re-evaluation */; RUN_TEST(test_catchup_standby_ack_does_not_count_toward_pend", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", + "fix": "Added try/catch block | Also: /* ═════════════════════════════════════════════════════════; * Task 3.1 — membership re-evaluation (commit_on_membership_ | Also: /* Task 3.1 — membership re-evaluation */; RUN_TEST(test_catchup_standby_ack_does_not_count_toward_pend", "tags": [ "auto-detected", "error-handling", @@ -6662,7 +6662,7 @@ "timestamp": "2026-06-11T15:55:52.451Z", "error_message": "Missing error handling in unknown", "file": "src/include/repl.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -6678,7 +6678,7 @@ "timestamp": "2026-06-11T15:56:20.945Z", "error_message": "Missing error handling in unknown", "file": "src/controller/repl.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -6695,7 +6695,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_commit_engine.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 10\u219215 lines (2 removed)", + "fix": "Rewrote 10→15 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -6710,7 +6710,7 @@ "timestamp": "2026-06-11T15:57:43.900Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_commit_engine.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -6726,7 +6726,7 @@ "timestamp": "2026-06-11T16:15:06.660Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-commit-engine/tasks.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -6742,7 +6742,7 @@ "timestamp": "2026-06-11T16:19:30.370Z", "error_message": "Missing error handling in unknown", "file": "src/controller/ha.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -6758,8 +6758,8 @@ "timestamp": "2026-06-11T16:19:51.195Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_commit_engine.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", - "fix": "Added try/catch block | Also: /* Task 4.2 \u2014 wired gate: registry truth at ha_dispatch_gate; RUN_TEST(test_gate_wired_registry_catchup_standby_rejects_th", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", + "fix": "Added try/catch block | Also: /* Task 4.2 — wired gate: registry truth at ha_dispatch_gate; RUN_TEST(test_gate_wired_registry_catchup_standby_rejects_th", "tags": [ "auto-detected", "error-handling", @@ -6790,7 +6790,7 @@ "timestamp": "2026-06-11T16:23:59.453Z", "error_message": "Missing error handling in unknown", "file": "src/controller/ha.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -6806,8 +6806,8 @@ "timestamp": "2026-06-11T22:39:24.704Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_commit_engine.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", - "fix": "Added try/catch block | Also: /* \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550; * Task 5.1 \u2014 integration: ACK feed and the job-submit sync-a | Also: #include \"typemap.h\" /* wal_catchup_entry_t (task 5; #include \"wal_types.h\" /* WAL_JOB_SUBMIT (task 5.1 sy", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", + "fix": "Added try/catch block | Also: /* ═════════════════════════════════════════════════════════; * Task 5.1 — integration: ACK feed and the job-submit sync-a | Also: #include \"typemap.h\" /* wal_catchup_entry_t (task 5; #include \"wal_types.h\" /* WAL_JOB_SUBMIT (task 5.1 sy", "tags": [ "auto-detected", "error-handling", @@ -6855,7 +6855,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/repl.c", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 13\u219227 lines (5 removed)", + "fix": "Rewrote 13→27 lines (5 removed)", "tags": [ "auto-detected", "refactor", @@ -6870,7 +6870,7 @@ "timestamp": "2026-06-11T23:32:42.365Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-commit-engine/tasks.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -6919,7 +6919,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/persist.c", "root_cause": "16 lines replaced/restructured", - "fix": "Rewrote 40\u219267 lines (16 removed)", + "fix": "Rewrote 40→67 lines (16 removed)", "tags": [ "auto-detected", "refactor", @@ -6967,7 +6967,7 @@ "error_message": "Wrong reference: test_single_node_committed_lsn_unchanged should be test_single_node_committed_lsn_advances", "file": "src/controller/testsuite/test_p2_integration.c", "root_cause": "Used \"test_single_node_committed_lsn_unchanged\" instead of \"test_single_node_committed_lsn_advances\"", - "fix": "Changed test_single_node_committed_lsn_unchanged \u2192 test_single_node_committed_lsn_advances", + "fix": "Changed test_single_node_committed_lsn_unchanged → test_single_node_committed_lsn_advances", "tags": [ "auto-detected", "wrong-reference", @@ -7068,7 +7068,7 @@ "timestamp": "2026-06-12T04:13:06.326Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-commit-barrier-rollout/tasks.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -7101,7 +7101,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/persist.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 7\u219212 lines (4 removed) | Also: return persist_wal_async(p, wal_type, &js, sizeof(; persist_t* p = cls; | Also: int rc = persist_wal_async(p, wal_type, pkt, (uint; persist_t* p = cls;", + "fix": "Rewrote 7→12 lines (4 removed) | Also: return persist_wal_async(p, wal_type, &js, sizeof(; persist_t* p = cls; | Also: int rc = persist_wal_async(p, wal_type, pkt, (uint; persist_t* p = cls;", "tags": [ "auto-detected", "refactor", @@ -7149,7 +7149,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_commit_barrier_reply.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 3\u21925 lines (2 removed) | Also: static xce_signal_t g_sigkill = { .signo = SIGKIL; static xce_jobctrl_t g_jobctrl;", + "fix": "Rewrote 3→5 lines (2 removed) | Also: static xce_signal_t g_sigkill = { .signo = SIGKIL; static xce_jobctrl_t g_jobctrl;", "tags": [ "auto-detected", "refactor", @@ -7257,7 +7257,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_commit_barrier_reply.c", "root_cause": "7 lines replaced/restructured", - "fix": "Rewrote 12\u219256 lines (7 removed) | Also: xce_job_t jdef;; static int jobobj_ready; /* jobj module tables \u2014 o", + "fix": "Rewrote 12→56 lines (7 removed) | Also: xce_job_t jdef;; static int jobobj_ready; /* jobj module tables — o", "tags": [ "auto-detected", "refactor", @@ -7272,7 +7272,7 @@ "timestamp": "2026-06-12T11:05:00-07:00", "error_message": "va_list type confusion: WAL_JOB_RUSAGE hook payload reads job_t fields from an xce_job_rusages_t*", "file": "src/controller/job.c (caller) / src/controller/persist.c (consumer)", - "root_cause": "update_job_rusages fires AFTER_EVENT HOOK_DETAIL_JOB_RUSAGE_UPDATE with (cluster_t*, xce_job_rusages_t*) at job.c:1015-1016, but persist_job_hook_fun's default arm (persist.c:1354-1359) reads the second vararg as job_t* and dereferences j->job.state at offset 56 == sizeof(xce_job_rusages_t) \u2014 an out-of-bounds read on an exactly-sized allocation; the WAL_JOB_RUSAGE payload state field is garbage (jobid at offset 0 lines up by accident)", + "root_cause": "update_job_rusages fires AFTER_EVENT HOOK_DETAIL_JOB_RUSAGE_UPDATE with (cluster_t*, xce_job_rusages_t*) at job.c:1015-1016, but persist_job_hook_fun's default arm (persist.c:1354-1359) reads the second vararg as job_t* and dereferences j->job.state at offset 56 == sizeof(xce_job_rusages_t) — an out-of-bounds read on an exactly-sized allocation; the WAL_JOB_RUSAGE payload state field is garbage (jobid at offset 0 lines up by accident)", "fix": "NOT FIXED (discovered while writing task 3.3 receipt-ACK verdict pins; production out of boundary). Test test_update_op_rusage_receipt_reply_facts_retained uses a padded union so the pin stays in-bounds under ASAN without masking the bug; fix candidates: dedicated RUSAGE arm in persist_job_hook_fun reading xce_job_rusages_t*, or fire the hook with (c, j) like the other job hooks", "tags": [ "pre-existing", @@ -7313,7 +7313,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_repl_majority.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 3\u219210 lines (3 removed) | Also: * walAck half is engine-wired per-LSN; the configA; * byte-identical legacy behavior (config barrier o | Also: /* upgradeConfigAck with no pending ACKs \u2192 timeout; int rc_cfg = repl_wait_majority_ack(&g_mp.repl, 0,", + "fix": "Rewrote 3→10 lines (3 removed) | Also: * walAck half is engine-wired per-LSN; the configA; * byte-identical legacy behavior (config barrier o | Also: /* upgradeConfigAck with no pending ACKs → timeout; int rc_cfg = repl_wait_majority_ack(&g_mp.repl, 0,", "tags": [ "auto-detected", "refactor", @@ -7329,7 +7329,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/repl.c", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 6\u219211 lines (5 removed)", + "fix": "Rewrote 6→11 lines (5 removed)", "tags": [ "auto-detected", "refactor", @@ -7345,7 +7345,7 @@ "error_message": "Significant refactor of ", "file": "src/include/repl.h", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 4\u21927 lines (4 removed)", + "fix": "Rewrote 4→7 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -7361,7 +7361,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/config_repl.c", "root_cause": "28 lines replaced/restructured", - "fix": "Rewrote 73\u219285 lines (28 removed) | Also: /* 7. Push to standbys (best-effort; see push_conf; int acked = push_config_to_standbys(c, out_bytes, | Also: * 7. Push to standbys (best-effort); * 8. ACK admin tool", + "fix": "Rewrote 73→85 lines (28 removed) | Also: /* 7. Push to standbys (best-effort; see push_conf; int acked = push_config_to_standbys(c, out_bytes, | Also: * 7. Push to standbys (best-effort); * 8. ACK admin tool", "tags": [ "auto-detected", "refactor", @@ -7392,7 +7392,7 @@ "timestamp": "2026-06-13T00:19:39.908Z", "error_message": "Missing error handling in unknown", "file": "src/controller/config_repl.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -7430,7 +7430,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/config_repl.c", "root_cause": "7 lines replaced/restructured", - "fix": "Rewrote 14\u219219 lines (7 removed) | Also: cluster_t *c, const char *msg); reply.status = -EINVAL;", + "fix": "Rewrote 14→19 lines (7 removed) | Also: cluster_t *c, const char *msg); reply.status = -EINVAL;", "tags": [ "auto-detected", "refactor", @@ -7462,7 +7462,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/config_repl.h", "root_cause": "8 lines replaced/restructured", - "fix": "Rewrote 22\u219239 lines (8 removed)", + "fix": "Rewrote 22→39 lines (8 removed)", "tags": [ "auto-detected", "refactor", @@ -7478,7 +7478,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "6 lines replaced/restructured", - "fix": "Rewrote 10\u219218 lines (6 removed)", + "fix": "Rewrote 10→18 lines (6 removed)", "tags": [ "auto-detected", "refactor", @@ -7493,7 +7493,7 @@ "timestamp": "2026-06-13T01:14:15.668Z", "error_message": "Missing error handling in unknown", "file": "src/controller/join.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -7598,7 +7598,7 @@ "error_message": "Wrong reference: test_ha_cluster should be test_ha_role_lifecycle", "file": "src/controller/testsuite/controller_test.c", "root_cause": "Used \"test_ha_cluster\" instead of \"test_ha_role_lifecycle\"", - "fix": "Changed test_ha_cluster \u2192 test_ha_role_lifecycle", + "fix": "Changed test_ha_cluster → test_ha_role_lifecycle", "tags": [ "auto-detected", "wrong-reference", @@ -7630,7 +7630,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_role_lifecycle.c", "root_cause": "15 lines replaced/restructured", - "fix": "Rewrote 64\u2192108 lines (15 removed)", + "fix": "Rewrote 64→108 lines (15 removed)", "tags": [ "auto-detected", "refactor", @@ -7662,7 +7662,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_role_lifecycle.c", "root_cause": "21 lines replaced/restructured", - "fix": "Rewrote 35\u219257 lines (21 removed) | Also: /* \u2500\u2500 Case 3: HOOKS REGISTERED BY THE SHARED PROMO; * canonical promotion entry point ha_promote_to_pr", + "fix": "Rewrote 35→57 lines (21 removed) | Also: /* ── Case 3: HOOKS REGISTERED BY THE SHARED PROMO; * canonical promotion entry point ha_promote_to_pr", "tags": [ "auto-detected", "refactor", @@ -7678,7 +7678,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 10\u219214 lines (5 removed) | Also: /* Start primary heartbeat immediately in COMMITTI; * NOP during the committing window and their liven", + "fix": "Rewrote 10→14 lines (5 removed) | Also: /* Start primary heartbeat immediately in COMMITTI; * NOP during the committing window and their liven", "tags": [ "auto-detected", "refactor", @@ -7706,7 +7706,7 @@ }, { "error_message": "AddressSanitizer: global-buffer-overflow in xce_wal_batch_req_raw <- repl_broadcast <- persist_wal_async <- ha_arm_own_term_gate <- ha_promote_to_primary, during test_promotion_registers_persist_hooks (test_ha_role_lifecycle.c case 3)", - "file": "src/controller/testsuite/test_ha_role_lifecycle.c (fixture defect \u2014 OUT OF task 2.2 boundary)", + "file": "src/controller/testsuite/test_ha_role_lifecycle.c (fixture defect — OUT OF task 2.2 boundary)", "root_cause": "task-2.2's ha_promote_to_primary now appends a WAL_NOOP via persist_wal_async, which broadcasts to registered standbys. The lifecycle fixture (lifecycle_cluster_setup) registers fake `int g_lc_eps[]` endpoints via repl_add_standby and NEVER installs repl_broadcast_send_hook, so repl_broadcast calls xce_wal_batch_req_raw(s->ep) and reads ep->peer (offset 36) past the 4-byte int. IDENTICAL to the prior test_p2_integration tests 8/9 bug already in buglog.", "fix": "Test-side fix (parent must apply, mirrors the documented p2_integration fix): in test_ha_role_lifecycle.c add `extern int (*repl_broadcast_send_hook)(void*, wal_catchup_entry_t**, int);` and a no-op drop hook, install it in lifecycle_cluster_setup before promotion (reset to NULL in teardown), and cast endpoints to (void*). Production ha.c is correct per design (persist_wal_async appends+broadcasts by design).", "tags": [ @@ -7726,9 +7726,9 @@ }, { "error_message": "task-2.1 lifecycle fixture off-by-one: case 5/6/7 assert writable_lsn==noop_lsn and current_lsn==noop_lsn+1, but lifecycle_cluster_setup primes current_lsn = appended_lsn + 1, so a real persist_wal_async appends the noop at appended_lsn+1 (current_lsn->appended_lsn+2), giving noop_lsn=appended_lsn+1 != asserted appended_lsn.", - "file": "src/controller/testsuite/test_ha_role_lifecycle.c (fixture defect \u2014 OUT OF task 2.2 boundary)", + "file": "src/controller/testsuite/test_ha_role_lifecycle.c (fixture defect — OUT OF task 2.2 boundary)", "root_cause": "persist_wal_async assigns hdr.lsn = current_lsn++ (entry takes the CURRENT value). The fixture comment assumes the next append takes current_lsn-1, which is off by one. To make the noop land at `appended_lsn`, the prime must be `current_lsn = appended_lsn`, not `appended_lsn + 1`.", - "fix": "Test-side fix (parent): set g_lc.persist.current_lsn = appended_lsn in lifecycle_cluster_setup. Production noop_lsn = current_lsn-1 (post-append) is correct per design \u00a7Promotion Rewrite / brief.", + "fix": "Test-side fix (parent): set g_lc.persist.current_lsn = appended_lsn in lifecycle_cluster_setup. Production noop_lsn = current_lsn-1 (post-append) is correct per design §Promotion Rewrite / brief.", "tags": [ "ha-single-leader-role", "task-2.2", @@ -7748,7 +7748,7 @@ "timestamp": "2026-06-13T08:56:36.044Z", "error_message": "Missing error handling in XCE_TESTING", "file": "src/controller/testsuite/test_ha_role_lifecycle.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -7781,7 +7781,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_role_lifecycle.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 5\u21927 lines (2 removed)", + "fix": "Rewrote 5→7 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -7845,7 +7845,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "11 lines replaced/restructured", - "fix": "Rewrote 16\u21924 lines (11 removed) | Also: /* Task 5.2: After updating convergence tracking, ; * majority of standbys have converged and promote ", + "fix": "Rewrote 16→4 lines (11 removed) | Also: /* Task 5.2: After updating convergence tracking, ; * majority of standbys have converged and promote ", "tags": [ "auto-detected", "refactor", @@ -7861,7 +7861,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/meson.build", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 5\u21923 lines (2 removed)", + "fix": "Rewrote 5→3 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -7877,7 +7877,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/controller_test.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 5\u21923 lines (2 removed) | Also: TEST(test_ha_evaluate_committing);; TEST(test_ha_repl_ack);", + "fix": "Rewrote 5→3 lines (2 removed) | Also: TEST(test_ha_evaluate_committing);; TEST(test_ha_repl_ack);", "tags": [ "auto-detected", "refactor", @@ -7893,7 +7893,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_role.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 4\u219210 lines (2 removed)", + "fix": "Rewrote 4→10 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -7909,7 +7909,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_role.c", "root_cause": "196 lines replaced/restructured", - "fix": "Rewrote 230\u219212 lines (196 removed) | Also: /* \u2500\u2500 Task 5.3: committing timeout default value a; * Requirements: 6.4, 7.5 (ha-repl-hardening)", + "fix": "Rewrote 230→12 lines (196 removed) | Also: /* ── Task 5.3: committing timeout default value a; * Requirements: 6.4, 7.5 (ha-repl-hardening)", "tags": [ "auto-detected", "refactor", @@ -7925,7 +7925,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_rdhup_integration.c", "root_cause": "34 lines replaced/restructured", - "fix": "Rewrote 48\u21929 lines (34 removed) | Also: * Test 4: 3-node cluster RDHUP \u2014 two acks needed, ; * | Also: /* Drive to CTRL_PRIMARY via ack */; ha_repl_ack_received(&c.ha, &c); | Also: RUN_TEST(test_rdhup_committing_to_primary_on_ack);; RUN_TEST(test_rdhup_3node_one_ack_promotes);", + "fix": "Rewrote 48→9 lines (34 removed) | Also: * Test 4: 3-node cluster RDHUP — two acks needed, ; * | Also: /* Drive to CTRL_PRIMARY via ack */; ha_repl_ack_received(&c.ha, &c); | Also: RUN_TEST(test_rdhup_committing_to_primary_on_ack);; RUN_TEST(test_rdhup_3node_one_ack_promotes);", "tags": [ "auto-detected", "refactor", @@ -7941,7 +7941,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_connection_reuse.c", "root_cause": "26 lines replaced/restructured", - "fix": "Rewrote 35\u21928 lines (26 removed) | Also: /* \u2500\u2500 Test 10 (Task 7.3): promoted connection carr; *", + "fix": "Rewrote 35→8 lines (26 removed) | Also: /* ── Test 10 (Task 7.3): promoted connection carr; *", "tags": [ "auto-detected", "refactor", @@ -7956,7 +7956,7 @@ "timestamp": "2026-06-13T12:04:59.483Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-single-leader-role/tasks.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -7989,7 +7989,7 @@ "error_message": "Wrong reference: CTRL_PRIMARY_COMMITTING should be CTRL_PRIMARY", "file": "src/controller/ha.c", "root_cause": "Used \"CTRL_PRIMARY_COMMITTING\" instead of \"CTRL_PRIMARY\"", - "fix": "Changed CTRL_PRIMARY_COMMITTING \u2192 CTRL_PRIMARY", + "fix": "Changed CTRL_PRIMARY_COMMITTING → CTRL_PRIMARY", "tags": [ "auto-detected", "wrong-reference", @@ -8005,7 +8005,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_role.c", "root_cause": "21 lines replaced/restructured", - "fix": "Rewrote 24\u21925 lines (21 removed) | Also: /* Task 5.3: committing timeout config-default kno; * survives until task 5.2; the degraded-mode commi", + "fix": "Rewrote 24→5 lines (21 removed) | Also: /* Task 5.3: committing timeout config-default kno; * survives until task 5.2; the degraded-mode commi", "tags": [ "auto-detected", "refactor", @@ -8037,7 +8037,7 @@ "error_message": "Wrong reference: CTRL_PRIMARY_COMMITTING should be CTRL_PRIMARY", "file": "src/controller/testsuite/test_ha_connection_reuse.c", "root_cause": "Used \"CTRL_PRIMARY_COMMITTING\" instead of \"CTRL_PRIMARY\"", - "fix": "Changed CTRL_PRIMARY_COMMITTING \u2192 CTRL_PRIMARY", + "fix": "Changed CTRL_PRIMARY_COMMITTING → CTRL_PRIMARY", "tags": [ "auto-detected", "wrong-reference", @@ -8068,7 +8068,7 @@ "timestamp": "2026-06-13T13:14:03.917Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_ha_startup.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -8101,7 +8101,7 @@ "error_message": "Wrong reference: test_nonblocking_single_node_election_enters_committing should be test_nonblocking_single_node_election_enters_primary_directly", "file": "src/controller/testsuite/test_ha_startup.c", "root_cause": "Used \"test_nonblocking_single_node_election_enters_committing\" instead of \"test_nonblocking_single_node_election_enters_primary_directly\"", - "fix": "Changed test_nonblocking_single_node_election_enters_committing \u2192 test_nonblocking_single_node_election_enters_primary_directly", + "fix": "Changed test_nonblocking_single_node_election_enters_committing → test_nonblocking_single_node_election_enters_primary_directly", "tags": [ "auto-detected", "wrong-reference", @@ -8117,7 +8117,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_rdhup_integration.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 3\u21924 lines (3 removed)", + "fix": "Rewrote 3→4 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -8152,7 +8152,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.h", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 4\u21923 lines (2 removed)", + "fix": "Rewrote 4→3 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -8168,7 +8168,7 @@ "error_message": "Wrong reference: uint32_t should be ha_config_t", "file": "src/controller/ha.h", "root_cause": "Used \"uint32_t\" instead of \"ha_config_t\"", - "fix": "Changed uint32_t \u2192 ha_config_t", + "fix": "Changed uint32_t → ha_config_t", "tags": [ "auto-detected", "wrong-reference", @@ -8184,7 +8184,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_role.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 7\u21924 lines (2 removed)", + "fix": "Rewrote 7→4 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -8199,8 +8199,8 @@ "timestamp": "2026-06-13T14:43:54.194Z", "error_message": "Incorrect value in code", "file": ".kiro/steering/ha-async-io.md", - "root_cause": "Had `CTRL_PRIMARY_COMMITTING \u2192 CTRL_PRIMARY`", - "fix": "Changed to 's own-term commit rule (\u00a75.4.2): an election winn", + "root_cause": "Had `CTRL_PRIMARY_COMMITTING → CTRL_PRIMARY`", + "fix": "Changed to 's own-term commit rule (§5.4.2): an election winn", "tags": [ "auto-detected", "wrong-value", @@ -8215,8 +8215,8 @@ "timestamp": "2026-06-13T14:54:21.781Z", "error_message": "Incorrect value in code", "file": ".claude/skills/xce-deploy-ha/SKILL.md", - "root_cause": "Had `\u2192 CTRL_PRIMARY_COMMITTING`", - "fix": "Changed to `HA FSM (raft): \u2192 CTRL_PRIMARY (term=...)`", + "root_cause": "Had `→ CTRL_PRIMARY_COMMITTING`", + "fix": "Changed to `HA FSM (raft): → CTRL_PRIMARY (term=...)`", "tags": [ "auto-detected", "wrong-value", @@ -8231,8 +8231,8 @@ "timestamp": "2026-06-13T15:31:21.740Z", "error_message": "Incorrect value in code", "file": "../../../../../claude/.claude/projects/-home-rshen-projects-xce-xce/memory/project_ha_consistency_invariants.md", - "root_cause": "Had \"XCE HA Raft+CP umbrella \u2014 sub-specs 1&2 of 4 IMPL", - "fix": "Changed to \"XCE HA Raft+CP umbrella \u2014 sub-specs 1,2,3 of 4 IM", + "root_cause": "Had \"XCE HA Raft+CP umbrella — sub-specs 1&2 of 4 IMPL", + "fix": "Changed to \"XCE HA Raft+CP umbrella — sub-specs 1,2,3 of 4 IM", "tags": [ "auto-detected", "wrong-value", @@ -8248,7 +8248,7 @@ "error_message": "Wrong reference: stdint should be stdbool", "file": "src/controller/include/cluster.h", "root_cause": "Used \"stdint\" instead of \"stdbool\"", - "fix": "Changed stdint \u2192 stdbool", + "fix": "Changed stdint → stdbool", "tags": [ "auto-detected", "wrong-reference", @@ -8264,7 +8264,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_egress_filter.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 7\u219217 lines (4 removed) | Also: /* \u2500\u2500 Cross-module link stubs (dataex.c/misc.c/sch; /* clang-format off */", + "fix": "Rewrote 7→17 lines (4 removed) | Also: /* ── Cross-module link stubs (dataex.c/misc.c/sch; /* clang-format off */", "tags": [ "auto-detected", "refactor", @@ -8280,7 +8280,7 @@ "error_message": "Wrong reference: xlist_destroy should be xlist_fini", "file": "src/controller/testsuite/test_egress_filter.c", "root_cause": "Used \"xlist_destroy\" instead of \"xlist_fini\"", - "fix": "Changed xlist_destroy \u2192 xlist_fini", + "fix": "Changed xlist_destroy → xlist_fini", "tags": [ "auto-detected", "wrong-reference", @@ -8328,7 +8328,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_egress_filter.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 12\u219217 lines (2 removed) | Also: jdef.jobid = 202;; jdef.state = JOB_STAT_PEND;", + "fix": "Rewrote 12→17 lines (2 removed) | Also: jdef.jobid = 202;; jdef.state = JOB_STAT_PEND;", "tags": [ "auto-detected", "refactor", @@ -8360,7 +8360,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/persist.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 4\u21926 lines (2 removed) | Also: extern int restore_job(cluster_t* c, uint64_t jobi; extern int restore_job_old_unused(cluster_t* c, ui", + "fix": "Rewrote 4→6 lines (2 removed) | Also: extern int restore_job(cluster_t* c, uint64_t jobi; extern int restore_job_old_unused(cluster_t* c, ui", "tags": [ "auto-detected", "refactor", @@ -8408,7 +8408,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/cluster.c", "root_cause": "8 lines replaced/restructured", - "fix": "Rewrote 9\u21924 lines (8 removed)", + "fix": "Rewrote 9→4 lines (8 removed)", "tags": [ "auto-detected", "refactor", @@ -8440,7 +8440,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/include/cluster.h", "root_cause": "8 lines replaced/restructured", - "fix": "Rewrote 12\u21923 lines (8 removed)", + "fix": "Rewrote 12→3 lines (8 removed)", "tags": [ "auto-detected", "refactor", @@ -8472,7 +8472,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_demotion_convergence.c", "root_cause": "11 lines replaced/restructured", - "fix": "Rewrote 16\u219234 lines (11 removed)", + "fix": "Rewrote 16→34 lines (11 removed)", "tags": [ "auto-detected", "refactor", @@ -8488,7 +8488,7 @@ "error_message": "AddressSanitizer: heap-use-after-free in ha_rejoin_reconcile_fn (ha_convergence.c:236) during test_rejoin_second_call_no_second_restart", "file": "src/controller/testsuite/test_demotion_convergence.c", "root_cause": "dc_run_pending_task peeked tasks.head and ran entry(args) WITHOUT dequeuing. The once-guard correctly makes the 2nd ha_rejoin_reconcile schedule no new task, so head still pointed at the first (already-run) task whose body had freed its heap ctx (args). The 2nd dc_run_pending_task re-ran that task on the freed ctx. The convergence task bodies free their own args (production delaymgr killer frees only the delay_task_t node, never args; production execv never re-runs a one-shot).", - "fix": "Made dc_run_pending_task DEQUEUE via ordq_get(&mgr->tasks, NULL), run the body, then free(t)/free(it) \u2014 mirroring the real delaymgr worker (ordq_get -> runner_fn -> FREEUP for runcount==1). A 2nd call now returns DC_NO_TASK as production would. Production ha_rejoin_reconcile/ha_rejoin_reconcile_fn unchanged (the body-frees-ctx pattern is correct).", + "fix": "Made dc_run_pending_task DEQUEUE via ordq_get(&mgr->tasks, NULL), run the body, then free(t)/free(it) — mirroring the real delaymgr worker (ordq_get -> runner_fn -> FREEUP for runcount==1). A 2nd call now returns DC_NO_TASK as production would. Production ha_rejoin_reconcile/ha_rejoin_reconcile_fn unchanged (the body-frees-ctx pattern is correct).", "tags": [ "asan", "use-after-free", @@ -8524,7 +8524,7 @@ "timestamp": "2026-06-14T04:57:22.178Z", "error_message": "Missing error handling in unknown", "file": "src/controller/ha.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block | Also: /*; * ha_walack_continuation_for_test: test-only entry point for", "tags": [ "auto-detected", @@ -8540,7 +8540,7 @@ "timestamp": "2026-06-14T04:57:59.221Z", "error_message": "Missing error handling in XCE_TESTING", "file": "src/controller/ha_convergence.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -8556,7 +8556,7 @@ "timestamp": "2026-06-14T05:23:00.737Z", "error_message": "Missing error handling in unknown", "file": "src/controller/ha.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -8572,7 +8572,7 @@ "timestamp": "2026-06-14T05:24:33.033Z", "error_message": "Missing error handling in XCE_TESTING", "file": "src/controller/config_repl.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -8588,7 +8588,7 @@ "timestamp": "2026-06-14T05:25:04.524Z", "error_message": "Missing error handling in unknown", "file": "src/controller/ha.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -8620,8 +8620,8 @@ "timestamp": "2026-06-14T06:37:35.904Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_convergence_verification.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", - "fix": "Added try/catch block | Also: printf(\"\\n>> Regression pins: fan-out / snapshot fallback / ; \"re-inclusion (task 6.1 \u2014 PASS against current code)\\n\\n\"); | Also: /* \u2500\u2500 Fan-out counting send-hook (task 6.1 (a)/(d), umbrella; * repl_broadcast routes each enqueue through repl_broadcast_", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", + "fix": "Added try/catch block | Also: printf(\"\\n>> Regression pins: fan-out / snapshot fallback / ; \"re-inclusion (task 6.1 — PASS against current code)\\n\\n\"); | Also: /* ── Fan-out counting send-hook (task 6.1 (a)/(d), umbrella; * repl_broadcast routes each enqueue through repl_broadcast_", "tags": [ "auto-detected", "error-handling", @@ -8653,7 +8653,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_convergence_verification.c", "root_cause": "22 lines replaced/restructured", - "fix": "Rewrote 33\u21923 lines (22 removed)", + "fix": "Rewrote 33→3 lines (22 removed)", "tags": [ "auto-detected", "refactor", @@ -8720,7 +8720,7 @@ "timestamp": "2026-06-14T08:35:02.456Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_ha_vote_gate.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -8737,7 +8737,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "6 lines replaced/restructured", - "fix": "Rewrote 12\u219217 lines (6 removed)", + "fix": "Rewrote 12→17 lines (6 removed)", "tags": [ "auto-detected", "refactor", @@ -8753,7 +8753,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/cluster.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 8\u219216 lines (3 removed)", + "fix": "Rewrote 8→16 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -8785,7 +8785,7 @@ "error_message": "WAL HOLE on cold-start catch-up: a fresh EMPTY standby (standby_lsn=0) catching up to a primary holding [0,1] receives ONLY lsn 1, missing lsn 0", "file": "src/controller/config_repl.c", "root_cause": "ha_stream_wal_to_standby filter `if (hdr.lsn > standby_lsn)` excludes lsn 0 when standby_lsn==0 (0>0 is false). standby_lsn=0 is ambiguous: it means BOTH 'empty standby (applied nothing)' AND 'standby has applied lsn 0'. The streamer cannot distinguish, computes start_lsn=standby_lsn+1=1, and drops the genuine lsn-0 entry. Only non-cosmetic because lsn-0 entries can be REAL data (first-ever WAL entry of a fresh cluster is lsn 0 from a real write OR the promotion NOOP).", - "fix": "FIXED via reserve-lsn-0: persist_init() now sets current_lsn=1 on a fresh store (persist.c), so lsn 0 is reserved as the none/empty sentinel and the first-ever WAL entry (promotion NOOP) is lsn 1. The streamer filter hdr.lsn>standby_lsn now yields 1>0=true for an empty standby (standby_lsn=0), streaming the first entry \u2014 no hole. No streamer change needed; the reserved base makes the existing filter correct. Also fixes the uncommittable cold-start NOOP (now lsn 1, committable 0->1) and the own-term writable_lsn gate (now 1, effective). Test: test_coldstart_lsn0.c H1-H4 all green incl H4 no-hole. Base-shift fallout fixed in test_commit_engine, test_commit_barrier_rollout, test_commit_barrier_reply, test_p2_integration, test_egress_filter (removed now-redundant priming appends / wired engine).", + "fix": "FIXED via reserve-lsn-0: persist_init() now sets current_lsn=1 on a fresh store (persist.c), so lsn 0 is reserved as the none/empty sentinel and the first-ever WAL entry (promotion NOOP) is lsn 1. The streamer filter hdr.lsn>standby_lsn now yields 1>0=true for an empty standby (standby_lsn=0), streaming the first entry — no hole. No streamer change needed; the reserved base makes the existing filter correct. Also fixes the uncommittable cold-start NOOP (now lsn 1, committable 0->1) and the own-term writable_lsn gate (now 1, effective). Test: test_coldstart_lsn0.c H1-H4 all green incl H4 no-hole. Base-shift fallout fixed in test_commit_engine, test_commit_barrier_rollout, test_commit_barrier_reply, test_p2_integration, test_egress_filter (removed now-redundant priming appends / wired engine).", "tags": [ "ha", "replication", @@ -8806,7 +8806,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_egress_filter.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 7\u21929 lines (3 removed)", + "fix": "Rewrote 7→9 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -8822,7 +8822,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_p3_hardening.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 4\u21926 lines (4 removed)", + "fix": "Rewrote 4→6 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -8838,7 +8838,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_commit_barrier_reply.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 8\u21929 lines (4 removed)", + "fix": "Rewrote 8→9 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -8854,7 +8854,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_convergence_verification.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 32\u2192111 lines (3 removed)", + "fix": "Rewrote 32→111 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -8870,7 +8870,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.h", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 8\u219215 lines (2 removed)", + "fix": "Rewrote 8→15 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -8886,7 +8886,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 30\u219238 lines (4 removed)", + "fix": "Rewrote 30→38 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -8902,7 +8902,7 @@ "error_message": "Wrong reference: test_pin_dispatch_gate_degraded_allowlist should be test_empty_standby_fresh_cluster_streams_not_snapshot", "file": "src/controller/testsuite/test_convergence_verification.c", "root_cause": "Used \"test_pin_dispatch_gate_degraded_allowlist\" instead of \"test_empty_standby_fresh_cluster_streams_not_snapshot\"", - "fix": "Changed test_pin_dispatch_gate_degraded_allowlist \u2192 test_empty_standby_fresh_cluster_streams_not_snapshot", + "fix": "Changed test_pin_dispatch_gate_degraded_allowlist → test_empty_standby_fresh_cluster_streams_not_snapshot", "tags": [ "auto-detected", "wrong-reference", @@ -8934,7 +8934,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_convergence_verification.c", "root_cause": "18 lines replaced/restructured", - "fix": "Rewrote 35\u219254 lines (18 removed) | Also: /* HA_REPL_SEND_SNAPSHOT: standby_lsn predates the; \"heartbeat decision: standby_lsn below oldest_wal_", + "fix": "Rewrote 35→54 lines (18 removed) | Also: /* HA_REPL_SEND_SNAPSHOT: standby_lsn predates the; \"heartbeat decision: standby_lsn below oldest_wal_", "tags": [ "auto-detected", "refactor", @@ -8950,7 +8950,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/repl.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 5\u219210 lines (2 removed)", + "fix": "Rewrote 5→10 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -8966,7 +8966,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_p2_integration.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 5\u219220 lines (3 removed)", + "fix": "Rewrote 5→20 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -9030,7 +9030,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test.c", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 7\u21929 lines (5 removed) | Also: config.nmasters = 1;; config.masters[0] = get_setting(\"host\");", + "fix": "Rewrote 7→9 lines (5 removed) | Also: config.nmasters = 1;; config.masters[0] = get_setting(\"host\");", "tags": [ "auto-detected", "refactor", @@ -9046,7 +9046,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 4\u219214 lines (2 removed)", + "fix": "Rewrote 4→14 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -9062,7 +9062,7 @@ "error_message": "Wrong reference: myconf_t should be uint32_t", "file": "src/include/myconf.h", "root_cause": "Used \"myconf_t\" instead of \"uint32_t\"", - "fix": "Changed myconf_t \u2192 uint32_t", + "fix": "Changed myconf_t → uint32_t", "tags": [ "auto-detected", "wrong-reference", @@ -9118,8 +9118,8 @@ "timestamp": "2026-06-15T09:16:53.679025Z", "error_message": "jobsub returns ok (status 0, jobid) but joblist AND jobinfo show the job absent on a stable single primary (ctrl-1); reslist works.", "file": "src/lib/request.c", - "root_cause": "SOLVED (reproduced locally via /xce-build + 3-node deploy-ha, ASAN). cluster_start() \u2014 the ONLY place that sets c->running=1 \u2014 is ORPHANED (zero callers; grep confirms). The HA startup path (myapp_run) starts iomgr/vtaskmgr/delaymgr/persist_start itself but never calls cluster_start, so c->running stays 0 (calloc default) on the primary. check_controller() (the streaming gate passed to xce_reply_get for joblist/workerlist) returns -EINVAL when !c->running; xce_reply_get then sets nc=0 (chunk count), shipping an EMPTY list to the client even though collect_jobs/key_find found the job. reslist passes checkf=NULL so it was immune. DIAG proof: 'reply_get send: table=2 pos=0 nc=0 nrows=1 checkstatus=-22' (pre-fix) -> 'nc=1 ... checkstatus=1' (post-fix). Single-node/legacy cluster_init path called cluster_start, which is why non-replicated worked.", - "fix": "Set c->running=1 at the TOP of cluster_run() (src/controller/cluster.c), symmetric with the existing c->running=0 on shutdown \u2014 the orphaned cluster_start() used to set it. c->running is a liveness/shutdown flag: check_controller() is polled PER CHUNK inside xce_reply_get()'s streaming loop so a long multi-chunk fetch (e.g. thousands of jobs) ABORTS mid-stream (nc=0) if a shutdown request arrives during it. The bug: running started 0 and was never set under HA, so the gate fired on every reply, not just during shutdown. Fix restores running=1 for the run-loop lifetime; the shutdown abort path (running=0 when cluster_run's cbuf_get loop breaks on the stop signal) is unchanged. Verified E2E on a local 3-node deploy-ha (ASAN): jobs now appear in joblist; DIAG showed checkstatus/nc flip -22/0 -> 1/1. Final fix is in cluster_run (not myapp_run \u2014 moved there so the 1<->0 lifecycle is co-located in the function that receives the shutdown signal).", + "root_cause": "SOLVED (reproduced locally via /xce-build + 3-node deploy-ha, ASAN). cluster_start() — the ONLY place that sets c->running=1 — is ORPHANED (zero callers; grep confirms). The HA startup path (myapp_run) starts iomgr/vtaskmgr/delaymgr/persist_start itself but never calls cluster_start, so c->running stays 0 (calloc default) on the primary. check_controller() (the streaming gate passed to xce_reply_get for joblist/workerlist) returns -EINVAL when !c->running; xce_reply_get then sets nc=0 (chunk count), shipping an EMPTY list to the client even though collect_jobs/key_find found the job. reslist passes checkf=NULL so it was immune. DIAG proof: 'reply_get send: table=2 pos=0 nc=0 nrows=1 checkstatus=-22' (pre-fix) -> 'nc=1 ... checkstatus=1' (post-fix). Single-node/legacy cluster_init path called cluster_start, which is why non-replicated worked.", + "fix": "Set c->running=1 at the TOP of cluster_run() (src/controller/cluster.c), symmetric with the existing c->running=0 on shutdown — the orphaned cluster_start() used to set it. c->running is a liveness/shutdown flag: check_controller() is polled PER CHUNK inside xce_reply_get()'s streaming loop so a long multi-chunk fetch (e.g. thousands of jobs) ABORTS mid-stream (nc=0) if a shutdown request arrives during it. The bug: running started 0 and was never set under HA, so the gate fired on every reply, not just during shutdown. Fix restores running=1 for the run-loop lifetime; the shutdown abort path (running=0 when cluster_run's cbuf_get loop breaks on the stop signal) is unchanged. Verified E2E on a local 3-node deploy-ha (ASAN): jobs now appear in joblist; DIAG showed checkstatus/nc flip -22/0 -> 1/1. Final fix is in cluster_run (not myapp_run — moved there so the 1<->0 lifecycle is co-located in the function that receives the shutdown signal).", "tags": [ "joblist", "workerlist", @@ -9144,7 +9144,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/opget.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 6\u21923 lines (2 removed) | Also: {; extern uint64_t g_diag_last_jobid;", + "fix": "Rewrote 6→3 lines (2 removed) | Also: {; extern uint64_t g_diag_last_jobid;", "tags": [ "auto-detected", "refactor", @@ -9159,7 +9159,7 @@ "timestamp": "2026-06-15T10:46:08.441Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_ha_rdhup_integration.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -9175,7 +9175,7 @@ "timestamp": "2026-06-15T10:46:31.815Z", "error_message": "Missing error handling in unknown", "file": "src/controller/ha.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -9192,7 +9192,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 5\u21924 lines (3 removed)", + "fix": "Rewrote 5→4 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -9231,7 +9231,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/myapp.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 3\u219211 lines (2 removed)", + "fix": "Rewrote 3→11 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -9263,7 +9263,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/steering/product.md", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 4\u21927 lines (2 removed)", + "fix": "Rewrote 4→7 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -9295,7 +9295,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "9 lines replaced/restructured", - "fix": "Rewrote 13\u21925 lines (9 removed)", + "fix": "Rewrote 13→5 lines (9 removed)", "tags": [ "auto-detected", "refactor", @@ -9308,10 +9308,10 @@ { "id": "bug-340", "timestamp": "2026-06-15T15:00:00.000Z", - "error_message": "After kill+RESTART of a controller (ctrl-1), the restarted node loops forever: ha_on_primary_ctrl_fd_rdhup \u2192 CTRL_ELECTING \u2192 raft_cb_become_follower \u2192 CTRL_STANDBY every ~heartbeat_timeout, term climbing unbounded (473+), while ctrl-2 stays PRIMARY at term 3. On ctrl-2: every heartbeat tick logs `xce_encode op 0 (NOP)` then `WARN endpoint_update_epoll rc1=-1 < 0` (netio.c:575) \u2014 a wedged outbound standby endpoint.", + "error_message": "After kill+RESTART of a controller (ctrl-1), the restarted node loops forever: ha_on_primary_ctrl_fd_rdhup → CTRL_ELECTING → raft_cb_become_follower → CTRL_STANDBY every ~heartbeat_timeout, term climbing unbounded (473+), while ctrl-2 stays PRIMARY at term 3. On ctrl-2: every heartbeat tick logs `xce_encode op 0 (NOP)` then `WARN endpoint_update_epoll rc1=-1 < 0` (netio.c:575) — a wedged outbound standby endpoint.", "file": "src/controller/ha.c, src/controller/testsuite/test_ha_liveness.c", - "root_cause": "primary_heartbeat_fn never evicted a dead/wedged outbound standby endpoint from ha->standby_endpoints. When a peer is killed/restarted, its outbound ep on the primary wedges: handle_error (netio.c:420) runs iomgr_unmonitor (EPOLL_CTL_DEL, does NOT close the fd) and the err handler may leave the ep in standby_endpoints; epoll_ctl(EPOLL_CTL_MOD) on the DEL'd fd then returns ENOENT (rc1=-1) on every heartbeat enqueue. xce_nop_req still returns >=0 (enqueue ok) so the heartbeat layer never detected the wedge. Critically, primary_heartbeat_fn step 2 (reconnect) treats a peer as 'found' (connected) if ANY ep for it is in standby_endpoints \u2014 so the wedged ep SUPPRESSED reconnection. ctrl-2 therefore never opened a fresh connection to the restarted ctrl-1, never sent a ctrlhello, so ctrl-1 never set primary_ctrl_fd / refreshed last_activity and its liveness watchdog re-elected every heartbeat_timeout forever. (The earlier suspected 'become_follower doesn't reset last_activity' was NOT a bug: ha_arm_heartbeat_timeout_vtask already stamps last_activity=now, so the ~11s period is exactly one full timeout \u2014 confirming the watchdog is correct and the sole cause is the broken channel.)", - "fix": "In primary_heartbeat_fn (ha.c) step 1, added liveness eviction symmetric with the standby's own liveness poll: for each standby ep, if ep->ts_last_in > 0 (was responsive at least once) AND (now_wall - ep->ts_last_in) > heartbeat_timeout_s, remove it from ha->standby_endpoints (zarray_del, under ha->lock) and collect it; after releasing ha->lock, repl_remove_standby() each evicted ep. Removal is the load-bearing action \u2014 the reconnect scan (step 2) then re-establishes a fresh ctrlhello channel to the restarted peer on this/next tick, and ctrl-1 settles via ha_discover_primary (no term gate, ha.c:3807) \u2192 become_follower \u2192 STANDBY. ts_last_in/ts_last_out are existing endpoint_t fields (wall clock, set in handle_input/handle_output), so no libnetio change. The evicted ep object itself is reclaimed by its err handler when the dead connection finally surfaces an epoll event (bounded leak, consistent with codebase). Eviction keys on ts_last_in>0 so a just-connected joiner (ts_last_in==0) gets grace. Regression tests: test_primary_hb_evicts_silent_standby (RED\u2192GREEN), test_primary_hb_keeps_live_standby, test_primary_hb_keeps_unconfirmed_standby in test_ha_liveness.c.", + "root_cause": "primary_heartbeat_fn never evicted a dead/wedged outbound standby endpoint from ha->standby_endpoints. When a peer is killed/restarted, its outbound ep on the primary wedges: handle_error (netio.c:420) runs iomgr_unmonitor (EPOLL_CTL_DEL, does NOT close the fd) and the err handler may leave the ep in standby_endpoints; epoll_ctl(EPOLL_CTL_MOD) on the DEL'd fd then returns ENOENT (rc1=-1) on every heartbeat enqueue. xce_nop_req still returns >=0 (enqueue ok) so the heartbeat layer never detected the wedge. Critically, primary_heartbeat_fn step 2 (reconnect) treats a peer as 'found' (connected) if ANY ep for it is in standby_endpoints — so the wedged ep SUPPRESSED reconnection. ctrl-2 therefore never opened a fresh connection to the restarted ctrl-1, never sent a ctrlhello, so ctrl-1 never set primary_ctrl_fd / refreshed last_activity and its liveness watchdog re-elected every heartbeat_timeout forever. (The earlier suspected 'become_follower doesn't reset last_activity' was NOT a bug: ha_arm_heartbeat_timeout_vtask already stamps last_activity=now, so the ~11s period is exactly one full timeout — confirming the watchdog is correct and the sole cause is the broken channel.)", + "fix": "In primary_heartbeat_fn (ha.c) step 1, added liveness eviction symmetric with the standby's own liveness poll: for each standby ep, if ep->ts_last_in > 0 (was responsive at least once) AND (now_wall - ep->ts_last_in) > heartbeat_timeout_s, remove it from ha->standby_endpoints (zarray_del, under ha->lock) and collect it; after releasing ha->lock, repl_remove_standby() each evicted ep. Removal is the load-bearing action — the reconnect scan (step 2) then re-establishes a fresh ctrlhello channel to the restarted peer on this/next tick, and ctrl-1 settles via ha_discover_primary (no term gate, ha.c:3807) → become_follower → STANDBY. ts_last_in/ts_last_out are existing endpoint_t fields (wall clock, set in handle_input/handle_output), so no libnetio change. The evicted ep object itself is reclaimed by its err handler when the dead connection finally surfaces an epoll event (bounded leak, consistent with codebase). Eviction keys on ts_last_in>0 so a just-connected joiner (ts_last_in==0) gets grace. Regression tests: test_primary_hb_evicts_silent_standby (RED→GREEN), test_primary_hb_keeps_live_standby, test_primary_hb_keeps_unconfirmed_standby in test_ha_liveness.c.", "tags": [ "ha", "heartbeat", @@ -9332,7 +9332,7 @@ "timestamp": "2026-06-16T00:10:49.295Z", "error_message": "Incorrect value in code", "file": ".kiro/specs/ha-prevote/research.md", - "root_cause": "Had 't understand PreVote (older binary) \u2014 does the ca", + "root_cause": "Had 't understand PreVote (older binary) — does the ca", "fix": "Changed to 's \"", "tags": [ "auto-detected", @@ -9364,7 +9364,7 @@ "timestamp": "2026-06-16T00:26:16.193Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-prevote/research.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -9413,7 +9413,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-prevote/tasks.md", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 7\u21927 lines (3 removed) | Also: - [ ] 5.1 Wire the PreVote reply opcode to the can; - Dispatch the incoming PreVote-reply opcode to th", + "fix": "Rewrote 7→7 lines (3 removed) | Also: - [ ] 5.1 Wire the PreVote reply opcode to the can; - Dispatch the incoming PreVote-reply opcode to th", "tags": [ "auto-detected", "refactor", @@ -9429,7 +9429,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/raft/raft.h", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 4\u21925 lines (2 removed)", + "fix": "Rewrote 4→5 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -9444,7 +9444,7 @@ "timestamp": "2026-06-16T04:18:41.611Z", "error_message": "Missing error handling in unknown", "file": "src/lib/testsuite/test_raft.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -9460,7 +9460,7 @@ "timestamp": "2026-06-16T04:19:25.513Z", "error_message": "Missing error handling in never", "file": "src/lib/raft/raft.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -9508,8 +9508,8 @@ "timestamp": "2026-06-16T04:50:38.855Z", "error_message": "Missing error handling in struct", "file": "src/controller/testsuite/test_ha_vote_gate.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", - "fix": "Added try/catch block | Also: /* Task 3.1 \u2014 voter-side PreVote dispatch gate (stickiness +; * no mutation, reply term unchanged) */", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", + "fix": "Added try/catch block | Also: /* Task 3.1 — voter-side PreVote dispatch gate (stickiness +; * no mutation, reply term unchanged) */", "tags": [ "auto-detected", "error-handling", @@ -9525,7 +9525,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ha_vote_gate.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 4\u21924 lines (2 removed)", + "fix": "Rewrote 4→4 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -9540,7 +9540,7 @@ "timestamp": "2026-06-16T04:57:16.230Z", "error_message": "Missing error handling in unknown", "file": "src/controller/cluster.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -9557,7 +9557,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/cluster.c", "root_cause": "6 lines replaced/restructured", - "fix": "Rewrote 20\u219225 lines (6 removed)", + "fix": "Rewrote 20→25 lines (6 removed)", "tags": [ "auto-detected", "refactor", @@ -9572,7 +9572,7 @@ "timestamp": "2026-06-16T05:21:02.997Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-prevote/tasks.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -9589,7 +9589,7 @@ "error_message": "Wrong reference: test_commit_engine should be test_ha_prevote", "file": "src/controller/testsuite/controller_test.c", "root_cause": "Used \"test_commit_engine\" instead of \"test_ha_prevote\"", - "fix": "Changed test_commit_engine \u2192 test_ha_prevote", + "fix": "Changed test_commit_engine → test_ha_prevote", "tags": [ "auto-detected", "wrong-reference", @@ -9653,7 +9653,7 @@ "error_message": "Wrong reference: test_commit_engine should be test_ha_failover_prevote", "file": "src/controller/testsuite/controller_test.c", "root_cause": "Used \"test_commit_engine\" instead of \"test_ha_failover_prevote\"", - "fix": "Changed test_commit_engine \u2192 test_ha_failover_prevote", + "fix": "Changed test_commit_engine → test_ha_failover_prevote", "tags": [ "auto-detected", "wrong-reference", @@ -9669,7 +9669,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "104 lines replaced/restructured", - "fix": "Rewrote 157\u219290 lines (104 removed)", + "fix": "Rewrote 157→90 lines (104 removed)", "tags": [ "auto-detected", "refactor", @@ -9716,7 +9716,7 @@ "timestamp": "2026-06-16T06:58:28.916Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_ha_prevote.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -9733,7 +9733,7 @@ "error_message": "Significant refactor of ", "file": ".claude/settings.json", "root_cause": "9 lines replaced/restructured", - "fix": "Rewrote 13\u21923 lines (9 removed) | Also: },; {", + "fix": "Rewrote 13→3 lines (9 removed) | Also: },; {", "tags": [ "auto-detected", "refactor", @@ -9748,7 +9748,7 @@ "timestamp": "2026-06-16T07:55:35.829Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-prevote/tasks.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -9765,7 +9765,7 @@ "error_message": "Significant refactor of ", "file": ".claude/settings.json", "root_cause": "9 lines replaced/restructured", - "fix": "Rewrote 13\u21923 lines (9 removed) | Also: },; {", + "fix": "Rewrote 13→3 lines (9 removed) | Also: },; {", "tags": [ "auto-detected", "refactor", @@ -9796,7 +9796,7 @@ "timestamp": "2026-06-16T15:55:00Z", "error_message": "controller unit test TIMEOUT 300.01s killed by SIGTERM; suite exceeds 300s meson cap", "file": "src/controller/persist.c", - "root_cause": "fsync_worker slept the full wal_sync_ms in one uninterruptible nanosleep() then checked fsync_stop. persist_stop sets the flag and immediately pthread_join()s, so the join blocked until the in-progress sleep returned \u2014 up to a full wal_sync_ms per teardown. Tests set wal_sync_ms=5000 ('slow fsync, no races'), so each persist_stop cost ~5s; across ~7 suites with many fixtures the cumulative teardown time blew past the 300s controller-test cap. The 'commit: WAIT deadline expired' warnings in the log were unrelated intentional negative tests (50-300ms).", + "root_cause": "fsync_worker slept the full wal_sync_ms in one uninterruptible nanosleep() then checked fsync_stop. persist_stop sets the flag and immediately pthread_join()s, so the join blocked until the in-progress sleep returned — up to a full wal_sync_ms per teardown. Tests set wal_sync_ms=5000 ('slow fsync, no races'), so each persist_stop cost ~5s; across ~7 suites with many fixtures the cumulative teardown time blew past the 300s controller-test cap. The 'commit: WAIT deadline expired' warnings in the log were unrelated intentional negative tests (50-300ms).", "fix": "Replaced the nanosleep with pthread_cond_timedwait on a new fsync_mutex/fsync_cond (added to persist_t, init in persist_init, destroyed in persist_fini). persist_stop now sets fsync_stop and pthread_cond_signal(fsync_cond) under fsync_mutex before join, so the worker wakes immediately. fdatasync still runs under p->lock, released before re-acquiring fsync_mutex (no lock nesting). Full controller suite now completes in ~10s (was >300s); 0 failures.", "tags": [ "controller", @@ -9839,10 +9839,10 @@ { "id": "bug-606", "timestamp": "2026-06-16T17:46:00Z", - "error_message": "After killing the primary (ctrl-1), the two surviving standbys (ctrl-2,ctrl-3) livelock in synchronized elections: PreVote win \u2192 CTRL_ELECTING \u2192 become_follower (term bumped) \u2192 ~10s \u2192 repeat, term climbing unbounded (3\u21929\u2192\u221e), no new primary ever elected. Reproduce: deploy 3-node HA, elect primary, submit 3-5 jobs, kill ctrl-1.", + "error_message": "After killing the primary (ctrl-1), the two surviving standbys (ctrl-2,ctrl-3) livelock in synchronized elections: PreVote win → CTRL_ELECTING → become_follower (term bumped) → ~10s → repeat, term climbing unbounded (3→9→∞), no new primary ever elected. Reproduce: deploy 3-node HA, elect primary, submit 3-5 jobs, kill ctrl-1.", "file": "src/controller/ha.c (ha_begin_failover, ha_election_stagger_ms, ha_failover_stagger_fn); src/controller/ha.h; test: src/controller/testsuite/test_ha_role.c", - "root_cause": "Dueling-candidates split-vote livelock \u2014 Raft \u00a75.2 randomized election timeout is missing on the FAILOVER path. Both survivors lose the primary at the same instant; their liveness polls fire on the identical fixed heartbeat_timeout, so they start PreVote+real-election in perfect lock-step (ms-aligned timestamps). Both win PreVote (neither sees a primary, so each grants the other \u2014 PreVote cannot break this tie), then both campaign at the same term, each votes for itself and denies the other (quorum 2, only 2 live nodes) \u2192 split vote \u2192 election times out (fixed election_timeout_ms, raft.c:105, NO jitter) \u2192 become_follower, term+1 \u2192 repeat forever. The priority/jitter config (election_jitter_max_ms, election_priority_step_ms) is applied ONLY at startup (ha_wait_for_heartbeat_or_timeout); the failover re-election path (liveness_poll_fn \u2192 ha_on_primary_ctrl_fd_rdhup \u2192 PreVote \u2192 ha_start_election_async \u2192 raft_start_election) has no de-synchronization. raft_on_leader_lost DOES have a priority stagger (raft.c:421) but the failover loop does not go through it.", - "fix": "Added ha_election_stagger_ms() (ha.c/ha.h): per-node failover delay = my_priority*election_priority_step_ms + random[0,election_jitter_max_ms], re-randomized each call (mirrors the startup stagger). ha_begin_failover() now schedules the PreVote start via delaymgr after that stagger (production only; c==NULL/no-delaymgr stays synchronous so existing tests are unaffected). The delaymgr callback ha_failover_stagger_fn guards with role==STANDBY && ha_heartbeat_is_fresh \u2192 skips PreVote if a primary connected during the stagger. Effect: two survivors no longer campaign in lock-step; the higher-priority survivor wins and its heartbeats reset the other. TDD: test_election_stagger_desyncs_by_priority (bounds + step>jitter ordering). VERIFIED on kill-primary repro: ctrl-2 (stagger 630ms) won \u2192 PRIMARY term=1; ctrl-3 (stagger 1072ms) received ctrl-2's ctrlhello first \u2192 STANDBY, then its stagger timer hit the guard and skipped PreVote ('a primary is alive \u2014 skipping PreVote'). No term inflation, no livelock. NOTE: optional raft.c:105 give-up-watchdog randomization NOT needed \u2014 the failover-start stagger alone resolves it; left as future defense-in-depth.", + "root_cause": "Dueling-candidates split-vote livelock — Raft §5.2 randomized election timeout is missing on the FAILOVER path. Both survivors lose the primary at the same instant; their liveness polls fire on the identical fixed heartbeat_timeout, so they start PreVote+real-election in perfect lock-step (ms-aligned timestamps). Both win PreVote (neither sees a primary, so each grants the other — PreVote cannot break this tie), then both campaign at the same term, each votes for itself and denies the other (quorum 2, only 2 live nodes) → split vote → election times out (fixed election_timeout_ms, raft.c:105, NO jitter) → become_follower, term+1 → repeat forever. The priority/jitter config (election_jitter_max_ms, election_priority_step_ms) is applied ONLY at startup (ha_wait_for_heartbeat_or_timeout); the failover re-election path (liveness_poll_fn → ha_on_primary_ctrl_fd_rdhup → PreVote → ha_start_election_async → raft_start_election) has no de-synchronization. raft_on_leader_lost DOES have a priority stagger (raft.c:421) but the failover loop does not go through it.", + "fix": "Added ha_election_stagger_ms() (ha.c/ha.h): per-node failover delay = my_priority*election_priority_step_ms + random[0,election_jitter_max_ms], re-randomized each call (mirrors the startup stagger). ha_begin_failover() now schedules the PreVote start via delaymgr after that stagger (production only; c==NULL/no-delaymgr stays synchronous so existing tests are unaffected). The delaymgr callback ha_failover_stagger_fn guards with role==STANDBY && ha_heartbeat_is_fresh → skips PreVote if a primary connected during the stagger. Effect: two survivors no longer campaign in lock-step; the higher-priority survivor wins and its heartbeats reset the other. TDD: test_election_stagger_desyncs_by_priority (bounds + step>jitter ordering). VERIFIED on kill-primary repro: ctrl-2 (stagger 630ms) won → PRIMARY term=1; ctrl-3 (stagger 1072ms) received ctrl-2's ctrlhello first → STANDBY, then its stagger timer hit the guard and skipped PreVote ('a primary is alive — skipping PreVote'). No term inflation, no livelock. NOTE: optional raft.c:105 give-up-watchdog randomization NOT needed — the failover-start stagger alone resolves it; left as future defense-in-depth.", "tags": [ "ha", "raft", @@ -9868,7 +9868,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/libxce-failover/requirements.md", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 5\u21928 lines (5 removed) | Also: 1. If a controller returns a not-ready (no current; shall retry the request with backoff.", + "fix": "Rewrote 5→8 lines (5 removed) | Also: 1. If a controller returns a not-ready (no current; shall retry the request with backoff.", "tags": [ "auto-detected", "refactor", @@ -9904,7 +9904,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/libxce-failover/requirements.md", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 6\u21929 lines (2 removed)", + "fix": "Rewrote 6→9 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -9936,7 +9936,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/libxce-failover/design.md", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 20\u219227 lines (3 removed) | Also: - On every reply, copy `reply.status` and **deep-c; with the reply \u2014 see request.c:243-250 hazard) + ` | Also: - Preconditions: `ctx` initialized; `conf->nmaster; exhaustion returns `-E_XCE_NOT_READY`; ctx left us", + "fix": "Rewrote 20→27 lines (3 removed) | Also: - On every reply, copy `reply.status` and **deep-c; with the reply — see request.c:243-250 hazard) + ` | Also: - Preconditions: `ctx` initialized; `conf->nmaster; exhaustion returns `-E_XCE_NOT_READY`; ctx left us", "tags": [ "auto-detected", "refactor", @@ -9952,7 +9952,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/libxce-failover/design.md", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 6\u219210 lines (4 removed)", + "fix": "Rewrote 6→10 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -10000,7 +10000,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/libxce-failover/research.md", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 4\u219210 lines (4 removed)", + "fix": "Rewrote 4→10 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -10016,7 +10016,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/libxce-failover/design.md", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 12\u219220 lines (2 removed) | Also: F->>E: reconnect to address; F->>E: reconnect to next master | Also: | 5.1, 5.3 | Reads+writes; no silent drop | Xcecli; | XcecliRetryLoop | client (`client.c`) | `do/whil | Also: - `src/lib/client.c` \u2014 wrap each `xcecli_*` op bod; (xcecli_retry_next(ac,&rc,is_write,&st));` loop; a", + "fix": "Rewrote 12→20 lines (2 removed) | Also: F->>E: reconnect to address; F->>E: reconnect to next master | Also: | 5.1, 5.3 | Reads+writes; no silent drop | Xcecli; | XcecliRetryLoop | client (`client.c`) | `do/whil | Also: - `src/lib/client.c` — wrap each `xcecli_*` op bod; (xcecli_retry_next(ac,&rc,is_write,&st));` loop; a", "tags": [ "auto-detected", "refactor", @@ -10048,7 +10048,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/libxce-failover/tasks.md", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 3\u21923 lines (3 removed)", + "fix": "Rewrote 3→3 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -10064,7 +10064,7 @@ "error_message": "Wrong reference: address should be hint", "file": ".kiro/specs/libxce-failover/tasks.md", "root_cause": "Used \"address\" instead of \"hint\"", - "fix": "Changed address \u2192 hint", + "fix": "Changed address → hint", "tags": [ "auto-detected", "wrong-reference", @@ -10080,7 +10080,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/client.c", "root_cause": "11 lines replaced/restructured", - "fix": "Rewrote 22\u219210 lines (11 removed)", + "fix": "Rewrote 22→10 lines (11 removed)", "tags": [ "auto-detected", "refactor", @@ -10096,7 +10096,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/request.c", "root_cause": "81 lines replaced/restructured", - "fix": "Rewrote 106\u219214 lines (81 removed)", + "fix": "Rewrote 106→14 lines (81 removed)", "tags": [ "auto-detected", "refactor", @@ -10112,7 +10112,7 @@ "error_message": "Significant refactor of ", "file": "src/include/request.h", "root_cause": "22 lines replaced/restructured", - "fix": "Rewrote 40\u219217 lines (22 removed)", + "fix": "Rewrote 40→17 lines (22 removed)", "tags": [ "auto-detected", "refactor", @@ -10128,7 +10128,7 @@ "error_message": "Wrong reference: XCE_MAXHOST_LEN should be XCE_MAXHOSTNAME_LEN", "file": ".kiro/specs/libxce-failover/design.md", "root_cause": "Used \"XCE_MAXHOST_LEN\" instead of \"XCE_MAXHOSTNAME_LEN\"", - "fix": "Changed XCE_MAXHOST_LEN \u2192 XCE_MAXHOSTNAME_LEN", + "fix": "Changed XCE_MAXHOST_LEN → XCE_MAXHOSTNAME_LEN", "tags": [ "auto-detected", "wrong-reference", @@ -10160,7 +10160,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/client_failover.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 13\u219223 lines (4 removed)", + "fix": "Rewrote 13→23 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -10192,7 +10192,7 @@ "error_message": "Significant refactor of ", "file": "src/include/xce/client_failover.h", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 7\u21923 lines (3 removed) | Also: * EndpointRotator \u2014 choose the next connect target; *", + "fix": "Rewrote 7→3 lines (3 removed) | Also: * EndpointRotator — choose the next connect target; *", "tags": [ "auto-detected", "refactor", @@ -10208,7 +10208,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/client_failover.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 3\u21924 lines (2 removed)", + "fix": "Rewrote 3→4 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -10240,7 +10240,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/client_failover.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 9\u219221 lines (2 removed)", + "fix": "Rewrote 9→21 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -10272,7 +10272,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/client_internal.h", "root_cause": "6 lines replaced/restructured", - "fix": "Rewrote 10\u21927 lines (6 removed)", + "fix": "Rewrote 10→7 lines (6 removed)", "tags": [ "auto-detected", "refactor", @@ -10304,7 +10304,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/client_failover.c", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 35\u219253 lines (5 removed)", + "fix": "Rewrote 35→53 lines (5 removed)", "tags": [ "auto-detected", "refactor", @@ -10336,7 +10336,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/client.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 5\u21927 lines (2 removed)", + "fix": "Rewrote 5→7 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -10400,7 +10400,7 @@ "error_message": "Wrong reference: test_p1_integration should be test_join_redirect", "file": "src/controller/testsuite/controller_test.c", "root_cause": "Used \"test_p1_integration\" instead of \"test_join_redirect\"", - "fix": "Changed test_p1_integration \u2192 test_join_redirect", + "fix": "Changed test_p1_integration → test_join_redirect", "tags": [ "auto-detected", "wrong-reference", @@ -10416,7 +10416,7 @@ "error_message": "Wrong reference: test_recover_resets_err_state_to_run should be test_decide_jitter_staggers_many_clients", "file": "src/lib/testsuite/test_libxce_retry.c", "root_cause": "Used \"test_recover_resets_err_state_to_run\" instead of \"test_decide_jitter_staggers_many_clients\"", - "fix": "Changed test_recover_resets_err_state_to_run \u2192 test_decide_jitter_staggers_many_clients", + "fix": "Changed test_recover_resets_err_state_to_run → test_decide_jitter_staggers_many_clients", "tags": [ "auto-detected", "wrong-reference", @@ -10429,7 +10429,7 @@ { "id": "bug-609", "timestamp": "2026-06-17T04:45:00Z", - "error_message": "After primary killed, libxce client xce_joblist/jobsubmit returns 'Invalid argument' (-EINVAL) on every fresh call \u2014 transparent failover did not engage", + "error_message": "After primary killed, libxce client xce_joblist/jobsubmit returns 'Invalid argument' (-EINVAL) on every fresh call — transparent failover did not engage", "file": "src/lib/client_failover.c", "root_cause": "handle_disconnect leaves xce_ctx_t.state=XCE_CTX_ERR after the primary's connection drops; the per-op xceapi guard (xceapi.c:14) returns -EINVAL when state!=XCE_CTX_RUN, BEFORE the op reaches a primary. xcecli_retry_decide classified -EINVAL as TERMINAL (non-retriable), so the failover loop bailed without rotating/recovering. The ctx->state ERR->RUN recovery only ran AFTER a rotate, which a terminal classification never triggered. Surfaced by the live 3-node E2E (task 6.2); the unit tests fed status codes directly to decide() and bypassed the real guard, so they missed it.", "fix": "In xcecli_retry_decide, before classifying *rc, detect ac->ctx.state==XCE_CTX_ERR (known-dead connection) and treat it as a prompt transport-style failover cause (XCE_FO_RETRY_NOW, no backoff) regardless of *rc, so the loop rotates+recovers. On budget exhaustion surface -E_XCE_NOT_READY (cluster-not-ready), never -EINVAL. Regression: test_decide_known_dead_ctx_retries_not_terminal.", @@ -10450,10 +10450,10 @@ { "id": "bug-610", "timestamp": "2026-06-17T15:00:00Z", - "error_message": "ASan SEGV in endpoint_connect (netio.c:672) via xce_failover_rotate \u2192 xcecli_retry_next \u2192 xcecli_joblist during live master failover (NULL ep->iomgr deref)", + "error_message": "ASan SEGV in endpoint_connect (netio.c:672) via xce_failover_rotate → xcecli_retry_next → xcecli_joblist during live master failover (NULL ep->iomgr deref)", "file": "src/lib/client_failover.c", - "root_cause": "xce_failover_rotate reconnected via endpoint_fini + endpoint_init + endpoint_connect, but endpoint_init (do_endpoint_init) resets the endpoint struct including ep->iomgr, and the rotate never re-bound it. The client's endpoint gets ep->iomgr = &ctx->iomgr only via xce_ctx_set_endpoint (called once in xcecli_start). endpoint_connect derefs iomgr->ssl_ctx when ep->ssl==NULL (true after endpoint_fini frees the SSL) \u2192 NULL deref crash. Also the rotate never re-set owner/msg_handler/err_handler nor iomgr_monitor'd the new fd, so even without the crash the reconnected endpoint would not route replies. Pure unit tests could not exercise the live socket/iomgr path; only the live failover E2E hit it.", - "fix": "Added xce_ctx_reconnect(ctx,host,port) in xcectx.c (where static handle_msg/handle_disconnect live) that mirrors the connection-establishment of xcecli_start/xce_ctx_start without restarting iomgr/vtaskmgr or touching ctx->state: endpoint_fini \u2192 endpoint_init \u2192 endpoint_set_peer \u2192 re-bind ep->iomgr=&ctx->iomgr \u2192 endpoint_connect \u2192 re-set owner+msg/err handlers \u2192 iomgr_monitor. xce_failover_rotate now delegates the reconnect to it.", + "root_cause": "xce_failover_rotate reconnected via endpoint_fini + endpoint_init + endpoint_connect, but endpoint_init (do_endpoint_init) resets the endpoint struct including ep->iomgr, and the rotate never re-bound it. The client's endpoint gets ep->iomgr = &ctx->iomgr only via xce_ctx_set_endpoint (called once in xcecli_start). endpoint_connect derefs iomgr->ssl_ctx when ep->ssl==NULL (true after endpoint_fini frees the SSL) → NULL deref crash. Also the rotate never re-set owner/msg_handler/err_handler nor iomgr_monitor'd the new fd, so even without the crash the reconnected endpoint would not route replies. Pure unit tests could not exercise the live socket/iomgr path; only the live failover E2E hit it.", + "fix": "Added xce_ctx_reconnect(ctx,host,port) in xcectx.c (where static handle_msg/handle_disconnect live) that mirrors the connection-establishment of xcecli_start/xce_ctx_start without restarting iomgr/vtaskmgr or touching ctx->state: endpoint_fini → endpoint_init → endpoint_set_peer → re-bind ep->iomgr=&ctx->iomgr → endpoint_connect → re-set owner+msg/err handlers → iomgr_monitor. xce_failover_rotate now delegates the reconnect to it.", "tags": [ "libxce", "failover", @@ -10493,7 +10493,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/netio.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 25\u219239 lines (3 removed)", + "fix": "Rewrote 25→39 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -10509,7 +10509,7 @@ "error_message": "Wrong reference: 611 should be 646", "file": "src/lib/testsuite/test_netio_arm_failure.c", "root_cause": "Used \"611\" instead of \"646\"", - "fix": "Changed 611 \u2192 646", + "fix": "Changed 611 → 646", "tags": [ "auto-detected", "wrong-reference", @@ -10525,7 +10525,7 @@ "error_message": "Wrong reference: 611 should be 646", "file": "src/lib/netio.c", "root_cause": "Used \"611\" instead of \"646\"", - "fix": "Changed 611 \u2192 646", + "fix": "Changed 611 → 646", "tags": [ "auto-detected", "wrong-reference", @@ -10567,7 +10567,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/netio.c", "root_cause": "13 lines replaced/restructured", - "fix": "Rewrote 48\u219266 lines (13 removed)", + "fix": "Rewrote 48→66 lines (13 removed)", "tags": [ "auto-detected", "refactor", @@ -10623,9 +10623,9 @@ { "id": "bug-650", "timestamp": "2026-06-17T18:30:00Z", - "error_message": "xce_test jobsub hangs in cbuf_get (xceapi.c:304) after failover even with bug-646/648 deployed. Log shows 'endpoint marked dead' (bug-646 firing) then a permanent hang; target primary (ctrl-2) is fully idle (no vtask in handle_jobsubmit) and the client fd is an unconnected TCP socket \u2014 the request was never transmitted yet the client waits for a reply forever.", + "error_message": "xce_test jobsub hangs in cbuf_get (xceapi.c:304) after failover even with bug-646/648 deployed. Log shows 'endpoint marked dead' (bug-646 firing) then a permanent hang; target primary (ctrl-2) is fully idle (no vtask in handle_jobsubmit) and the client fd is an unconnected TCP socket — the request was never transmitted yet the client waits for a reply forever.", "file": "src/lib/request.c", - "root_cause": "xce_request() and the 5 other request.c senders (xce_reply, xce_replconfigpush_req_raw, xce_wal_batch_req_raw, xce_snapshot_chunk_req_raw, xce_replconfig_reply_raw) called endpoint_enq(ep,&pkt->link) and IGNORED its return, always `return 0`. So when the transport was dead, endpoint_enq() correctly returned <0 and ran handle_error() (bug-646 fix) \u2014 but xce_request() swallowed that <0 and reported success. xce_jobsubmit() (xceapi.c:299-304) then saw rc>=0 and fell through to the BLOCKING cbuf_get(); the request was never flushed (server idle) and when err_handler was NULL after a failed reconnect, handle_disconnect never injected -ECONNRESET, so nothing woke the waiter. The bug-646 netio fix was defeated one layer up: the request layer reported success on a failed send. This was the missing link making every prior netio fix insufficient for the live hang.", + "root_cause": "xce_request() and the 5 other request.c senders (xce_reply, xce_replconfigpush_req_raw, xce_wal_batch_req_raw, xce_snapshot_chunk_req_raw, xce_replconfig_reply_raw) called endpoint_enq(ep,&pkt->link) and IGNORED its return, always `return 0`. So when the transport was dead, endpoint_enq() correctly returned <0 and ran handle_error() (bug-646 fix) — but xce_request() swallowed that <0 and reported success. xce_jobsubmit() (xceapi.c:299-304) then saw rc>=0 and fell through to the BLOCKING cbuf_get(); the request was never flushed (server idle) and when err_handler was NULL after a failed reconnect, handle_disconnect never injected -ECONNRESET, so nothing woke the waiter. The bug-646 netio fix was defeated one layer up: the request layer reported success on a failed send. This was the missing link making every prior netio fix insufficient for the live hang.", "fix": "All six request.c senders propagate the enq result: `int enq_rc = endpoint_enq(ep,&pkt->link); return (enq_rc < 0) ? enq_rc : 0;`. A failed send now returns <0 through xce_jobsubmit_req -> xce_jobsubmit, which returns before cbuf_get; the failover loop classifies the transport error and rotates instead of blocking on a reply that never arrives. TDD: test_request_send_propagation.c builds an OPCode_nop request (encodes OK) on a dead endpoint (EPOLL_CTL_MOD ENOENT) and asserts xce_request returns <0 (verified red with old `return 0`, green with fix). libxce 169/0; controller_unit_test links clean.", "tags": [ "libxce", @@ -10651,7 +10651,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/netio.c", "root_cause": "8 lines replaced/restructured", - "fix": "Rewrote 14\u219219 lines (8 removed)", + "fix": "Rewrote 14→19 lines (8 removed)", "tags": [ "auto-detected", "refactor", @@ -10683,7 +10683,7 @@ "error_message": "Significant refactor of ", "file": "src/lib/testsuite/test_netio_arm_failure.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 11\u219213 lines (3 removed)", + "fix": "Rewrote 11→13 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -10699,7 +10699,7 @@ "error_message": "Wrong reference: test_enq_arm_failure_is_loud_and_fires_errhandler should be test_enq_arm_failure_is_loud_no_errhandler", "file": "src/lib/testsuite/test_netio_arm_failure.c", "root_cause": "Used \"test_enq_arm_failure_is_loud_and_fires_errhandler\" instead of \"test_enq_arm_failure_is_loud_no_errhandler\"", - "fix": "Changed test_enq_arm_failure_is_loud_and_fires_errhandler \u2192 test_enq_arm_failure_is_loud_no_errhandler", + "fix": "Changed test_enq_arm_failure_is_loud_and_fires_errhandler → test_enq_arm_failure_is_loud_no_errhandler", "tags": [ "auto-detected", "wrong-reference", @@ -10714,7 +10714,7 @@ "timestamp": "2026-06-17T19:10:00Z", "error_message": "Controller (primary) self-deadlocks on ha->lock after bug-646/650 deploy: jobsub now works, but the controller freezes. gdb shows Thread 3 in primary_heartbeat_fn holding ha->lock (owner=self) and re-acquiring it via xce_nop_req -> xce_request -> endpoint_enq -> handle_error -> vote_ep_err_handler (ha.c:3478); Thread 9 (io_worker) also blocked on the same ha->lock. Whole controller frozen.", "file": "src/lib/netio.c", - "root_cause": "REGRESSION from bug-646: endpoint_enq() invoked handle_error() synchronously on the send path when the poll arm failed. handle_error() calls the endpoint's err_handler, which on a controller vote endpoint is vote_ep_err_handler() \u2014 it locks ha->lock. But endpoint_enq runs on the CALLER's thread, and the controller's primary_heartbeat_fn already holds ha->lock when it sends heartbeat NOPs. Non-recursive mutex -> self-deadlock. The err_handler is contractually an io_worker-thread callback (off an epoll error event, holding no caller locks); calling it from the synchronous send path violated that. With bug-650 now propagating the enq error up the return chain, the handle_error call in endpoint_enq was both redundant and harmful.", + "root_cause": "REGRESSION from bug-646: endpoint_enq() invoked handle_error() synchronously on the send path when the poll arm failed. handle_error() calls the endpoint's err_handler, which on a controller vote endpoint is vote_ep_err_handler() — it locks ha->lock. But endpoint_enq runs on the CALLER's thread, and the controller's primary_heartbeat_fn already holds ha->lock when it sends heartbeat NOPs. Non-recursive mutex -> self-deadlock. The err_handler is contractually an io_worker-thread callback (off an epoll error event, holding no caller locks); calling it from the synchronous send path violated that. With bug-650 now propagating the enq error up the return chain, the handle_error call in endpoint_enq was both redundant and harmful.", "fix": "endpoint_enq() no longer calls handle_error(); on arm failure it just returns a RETRIABLE -ECONNRESET. The caller propagates it (request.c senders return it -> client failover classifies ECONNRESET as RETRY_NOW and rotates; controller heartbeat path simply gets the error, no lock re-entry). The real teardown stays on the io_worker thread (handle_input/handle_output -> io_handler -> handle_error), which holds no caller locks. Updated test_netio_arm_failure.c: endpoint_enq must return <0 AND must NOT invoke the err_handler (g_err_count==0). libxce 169/0; controller_unit_test links clean.", "tags": [ "libxce", @@ -10772,8 +10772,8 @@ "timestamp": "2026-06-17T22:00:00Z", "error_message": "Failover election takes minutes (11+ PreVote rounds) to converge. A standby keeps denying a legitimately-higher-term candidate's PreVote with reason=fresh_heartbeat for ~2.5 min AFTER its primary died (ctrl-3 killed 21:44:14, ctrl-2 logs 'PreVote DENIED ... reason=fresh_heartbeat (my_term=3)' until 21:46:58).", "file": "src/controller/ha.c", - "root_cause": "The PreVote/vote eligibility gate raft_cb_vote_eligible (ha.c:311) denied on `role==CTRL_STANDBY && ha_heartbeat_is_fresh(ha)` \u2014 checking freshness ALONE. ha_heartbeat_is_fresh derives from ha->last_activity, but ha_arm_heartbeat_timeout_vtask resets last_activity=now on EVERY watchdog (re-)arm, and ha_prevote_rearm_liveness re-arms it after every lost PreVote. So a node with NO live primary perpetually refreshes its own 'fresh heartbeat' each failover cycle and denies all other candidates (reason=fresh_heartbeat), blocking convergence until it happens to restart. The sibling gate in handle_request_vote (ha.c:2205) already required `primary_ctrl_fd >= 0 && is_fresh`; raft_cb_vote_eligible was missing the primary_ctrl_fd guard. The existing fresh-deny tests even documented the intent ('the deny gate requires a live connection AND a fresh heartbeat', set primary_ctrl_fd=999), but production didn't enforce it.", - "fix": "Added `ha->primary_ctrl_fd >= 0 &&` to the ha.c:311 eligibility gate (mirrors ha.c:2205). primary_ctrl_fd is -1 on init / after the primary RDHUPs / after the liveness watchdog fires, and a real fd only via config_repl.c:1451 (first ctrlhello from the live primary) \u2014 so a dead/absent primary no longer masquerades as fresh and the standby GRANTS, converging in ~1 round. Kept cluster.c:526 denial-reason logging in sync. TDD: test_pre_vote_dead_primary_grants_despite_fresh_activity (fresh last_activity + primary_ctrl_fd=-1 -> GRANT); verified RED (Expected 1 Was 0) without the guard, GREEN with it, and the 3 existing fresh-deny tests (fd=999) unaffected. Full controller suite + libxce 169/0, zero regressions.", + "root_cause": "The PreVote/vote eligibility gate raft_cb_vote_eligible (ha.c:311) denied on `role==CTRL_STANDBY && ha_heartbeat_is_fresh(ha)` — checking freshness ALONE. ha_heartbeat_is_fresh derives from ha->last_activity, but ha_arm_heartbeat_timeout_vtask resets last_activity=now on EVERY watchdog (re-)arm, and ha_prevote_rearm_liveness re-arms it after every lost PreVote. So a node with NO live primary perpetually refreshes its own 'fresh heartbeat' each failover cycle and denies all other candidates (reason=fresh_heartbeat), blocking convergence until it happens to restart. The sibling gate in handle_request_vote (ha.c:2205) already required `primary_ctrl_fd >= 0 && is_fresh`; raft_cb_vote_eligible was missing the primary_ctrl_fd guard. The existing fresh-deny tests even documented the intent ('the deny gate requires a live connection AND a fresh heartbeat', set primary_ctrl_fd=999), but production didn't enforce it.", + "fix": "Added `ha->primary_ctrl_fd >= 0 &&` to the ha.c:311 eligibility gate (mirrors ha.c:2205). primary_ctrl_fd is -1 on init / after the primary RDHUPs / after the liveness watchdog fires, and a real fd only via config_repl.c:1451 (first ctrlhello from the live primary) — so a dead/absent primary no longer masquerades as fresh and the standby GRANTS, converging in ~1 round. Kept cluster.c:526 denial-reason logging in sync. TDD: test_pre_vote_dead_primary_grants_despite_fresh_activity (fresh last_activity + primary_ctrl_fd=-1 -> GRANT); verified RED (Expected 1 Was 0) without the guard, GREEN with it, and the 3 existing fresh-deny tests (fd=999) unaffected. Full controller suite + libxce 169/0, zero regressions.", "tags": [ "controller", "ha", @@ -10795,7 +10795,7 @@ "error_message": "3-node failover test lost committed jobs: 7 earliest JobIds (WAL LSN<=20), majority-ACKed and survived first failover, permanently gone after a restarted node with near-empty WAL won election. joblist dropped from 10 to 3.", "file": "src/controller/persist.c, src/controller/commit.c, src/controller/repl.c", "root_cause": "'committed' does not imply 'durable'. WAL fsync is asynchronous (persist.c:173 fsync_worker, timer wal_sync_ms); persist_append and the standby WalAck path do NOT fdatasync before ACK. commit_on_ack (commit.c:303) advances committed_lsn on majority ACK = OS page cache only, durable on zero disks. On kill+restart, killed nodes lose the not-yet-fsynced tail; persist_recover hits CRC break at zeroed region (prev_crc=00000000, offset 40) and truncates replay to ~0 entries, yet the hollow node stays eligible, wins election under degraded required=1 quorum (PreVote+freshness gate cannot fence it because the committed entries were durable on nobody), self-promotes advertising committed_lsn=21 over an empty job table, and the node holding the real data converges DOWN. Violates Leader-Completeness assumption (majority-ACK != durable-on-majority) in src/controller/CLAUDE.md.", - "fix": "NOT YET FIXED. Discovery -> new spec ha-commit-durability (Path C, brief written). R1 fsync-before-ACK (standby fdatasync before WalAck, primary self-count only after local fdatasync, committed_lsn advances only on durable majority, group-commit to bound latency). R2 promotion fence (WAL CRC/replay failure => ineligible to lead until caught up, FATAL not silent truncate; never lead below known committed high-water even under degraded quorum). Live E2E reproduces via xce_test jobsub + primary kills on /xce-deploy-ha. DETERMINISTIC REPRO (task 1.3, 2026-06-18): `bash src/tools/test-commit-durability.sh --deterministic` launches the cluster with --wal-sync-ms 30000 (plumbed via deploy-ha.sh start.sh `$*` passthrough, EXTRA_CTRL_ARGS) so the primary's fsync_worker never fires in-window -> every committed entry is page-cache-only on the primary (majority_candidate_locked counts cap=highest_appended_lsn=current_lsn-1, NOT durable). It then drives rolling PRIMARY kills (each right after committing to that primary) + restart, so hollow nodes re-lead. RED CONFIRMED: 6-round run lost the 3 EARLIEST committed JobIds permanently (rounds 4 and 6 + final check), Results 5 passed/3 failed, exit=1 \u2014 matching the incident signature (earliest jobs gone). NOTE: a single double-kill does NOT repro \u2014 this codebase enforces strict majority (lone survivor will NOT self-elect: log shows 'no single-voter primary within 40s'); loss needs the ROLLING/term-accumulating path the incident used. Evidence log: dur-det-run.log in worktree.", + "fix": "NOT YET FIXED. Discovery -> new spec ha-commit-durability (Path C, brief written). R1 fsync-before-ACK (standby fdatasync before WalAck, primary self-count only after local fdatasync, committed_lsn advances only on durable majority, group-commit to bound latency). R2 promotion fence (WAL CRC/replay failure => ineligible to lead until caught up, FATAL not silent truncate; never lead below known committed high-water even under degraded quorum). Live E2E reproduces via xce_test jobsub + primary kills on /xce-deploy-ha. DETERMINISTIC REPRO (task 1.3, 2026-06-18): `bash src/tools/test-commit-durability.sh --deterministic` launches the cluster with --wal-sync-ms 30000 (plumbed via deploy-ha.sh start.sh `$*` passthrough, EXTRA_CTRL_ARGS) so the primary's fsync_worker never fires in-window -> every committed entry is page-cache-only on the primary (majority_candidate_locked counts cap=highest_appended_lsn=current_lsn-1, NOT durable). It then drives rolling PRIMARY kills (each right after committing to that primary) + restart, so hollow nodes re-lead. RED CONFIRMED: 6-round run lost the 3 EARLIEST committed JobIds permanently (rounds 4 and 6 + final check), Results 5 passed/3 failed, exit=1 — matching the incident signature (earliest jobs gone). NOTE: a single double-kill does NOT repro — this codebase enforces strict majority (lone survivor will NOT self-elect: log shows 'no single-voter primary within 40s'); loss needs the ROLLING/term-accumulating path the incident used. Evidence log: dur-det-run.log in worktree.", "tags": [ "controller", "ha", @@ -10841,7 +10841,7 @@ "error_message": "Significant refactor of ", "file": "src/tools/test-commit-durability.sh", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 19\u219235 lines (4 removed)", + "fix": "Rewrote 19→35 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -10872,7 +10872,7 @@ "timestamp": "2026-06-18T00:04:40.582Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-commit-durability/brief.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -10937,7 +10937,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-commit-durability/design.md", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 6\u21926 lines (5 removed) | Also: - **Persistence & consistency:** `durable_lsn` adv; - **Concurrency strategy:** advanced under `p->loc", + "fix": "Rewrote 6→6 lines (5 removed) | Also: - **Persistence & consistency:** `durable_lsn` adv; - **Concurrency strategy:** advanced under `p->loc", "tags": [ "auto-detected", "refactor", @@ -10952,7 +10952,7 @@ "timestamp": "2026-06-18T03:14:30.410Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-commit-durability/design.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -10985,7 +10985,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-commit-durability/tasks.md", "root_cause": "7 lines replaced/restructured", - "fix": "Rewrote 18\u219225 lines (7 removed) | Also: - _Depends: 4.1, 3.3_; - Before a member becomes primary, refuse promotio", + "fix": "Rewrote 18→25 lines (7 removed) | Also: - _Depends: 4.1, 3.3_; - Before a member becomes primary, refuse promotio", "tags": [ "auto-detected", "refactor", @@ -11017,7 +11017,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-commit-durability/tasks.md", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 5\u21926 lines (2 removed)", + "fix": "Rewrote 5→6 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -11032,7 +11032,7 @@ "timestamp": "2026-06-18T03:59:55.411Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-commit-durability/design.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -11065,7 +11065,7 @@ "error_message": "Wrong reference: myconf_t should be uint32_t", "file": "src/include/myconf.h", "root_cause": "Used \"myconf_t\" instead of \"uint32_t\"", - "fix": "Changed myconf_t \u2192 uint32_t", + "fix": "Changed myconf_t → uint32_t", "tags": [ "auto-detected", "wrong-reference", @@ -11112,7 +11112,7 @@ "timestamp": "2026-06-18T05:00:22.007Z", "error_message": "Missing error handling in unknown", "file": "src/tools/test-commit-durability.sh", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -11129,7 +11129,7 @@ "error_message": "Significant refactor of ", "file": "src/tools/test-commit-durability.sh", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 85\u2192202 lines (4 removed)", + "fix": "Rewrote 85→202 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -11160,7 +11160,7 @@ "timestamp": "2026-06-18T05:07:20.589Z", "error_message": "Missing error handling in unknown", "file": "src/tools/test-commit-durability.sh", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -11192,7 +11192,7 @@ "timestamp": "2026-06-18T05:42:21.676Z", "error_message": "Missing error handling in advances", "file": "src/controller/testsuite/test_commit_durability.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -11225,7 +11225,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/persist.c", "root_cause": "7 lines replaced/restructured", - "fix": "Rewrote 41\u2192152 lines (7 removed) | Also: /* Wake any WAIT caller blocked on the commit eval; * advanced, so a majority candidate may now cover ", + "fix": "Rewrote 41→152 lines (7 removed) | Also: /* Wake any WAIT caller blocked on the commit eval; * advanced, so a majority candidate may now cover ", "tags": [ "auto-detected", "refactor", @@ -11256,7 +11256,7 @@ "timestamp": "2026-06-18T05:55:28.140Z", "error_message": "Missing error handling in FAILS", "file": "src/controller/testsuite/test_commit_durability.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -11273,7 +11273,7 @@ "error_message": "Wrong reference: else should be RUN_TEST", "file": "src/controller/testsuite/test_commit_durability.c", "root_cause": "Used \"else\" instead of \"RUN_TEST\"", - "fix": "Changed else \u2192 RUN_TEST", + "fix": "Changed else → RUN_TEST", "tags": [ "auto-detected", "wrong-reference", @@ -11289,7 +11289,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/persist.c", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 24\u219252 lines (5 removed) | Also: * already joined and will not advance it again). *; fdatasync(p->wal_fd);", + "fix": "Rewrote 24→52 lines (5 removed) | Also: * already joined and will not advance it again). *; fdatasync(p->wal_fd);", "tags": [ "auto-detected", "refactor", @@ -11404,7 +11404,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/commit.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 12\u219224 lines (2 removed)", + "fix": "Rewrote 12→24 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -11435,7 +11435,7 @@ "timestamp": "2026-06-18T06:15:12.451Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_commit_engine.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -11467,7 +11467,7 @@ "timestamp": "2026-06-18T06:16:09.746Z", "error_message": "Missing error handling in XCE_TESTING", "file": "src/controller/testsuite/test_commit_durability.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -11484,7 +11484,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_commit_engine.c", "root_cause": "40 lines replaced/restructured", - "fix": "Rewrote 49\u21926 lines (40 removed)", + "fix": "Rewrote 49→6 lines (40 removed)", "tags": [ "auto-detected", "refactor", @@ -11539,7 +11539,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/persist.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 3\u219222 lines (2 removed) | Also: int fsrc = fdatasync(p->wal_fd);; if (fsrc != 0 || flusher_fsync_forced_fail(p)) {", + "fix": "Rewrote 3→22 lines (2 removed) | Also: int fsrc = fdatasync(p->wal_fd);; if (fsrc != 0 || flusher_fsync_forced_fail(p)) {", "tags": [ "auto-detected", "refactor", @@ -11587,7 +11587,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_commit_durability.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 7\u219210 lines (4 removed)", + "fix": "Rewrote 7→10 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -11602,7 +11602,7 @@ "timestamp": "2026-06-18T06:50:13.989Z", "error_message": "Missing error handling in XCE_TESTING", "file": "src/controller/testsuite/test_commit_durability.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -11635,7 +11635,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/persist.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 8\u219210 lines (2 removed)", + "fix": "Rewrote 8→10 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -11651,7 +11651,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/persist.c", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 9\u21926 lines (5 removed) | Also: /* RED-PHASE: invariant assert DISABLED to let the; * the over-advance instead of aborting the binary.", + "fix": "Rewrote 9→6 lines (5 removed) | Also: /* RED-PHASE: invariant assert DISABLED to let the; * the over-advance instead of aborting the binary.", "tags": [ "auto-detected", "refactor", @@ -11682,7 +11682,7 @@ "timestamp": "2026-06-18T07:15:03.763Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_commit_durability.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -11698,7 +11698,7 @@ "timestamp": "2026-06-18T07:15:30.633Z", "error_message": "Missing error handling in unknown", "file": "src/include/persist.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -11715,7 +11715,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/persist.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 10\u219223 lines (2 removed) | Also: break; /* clean end of segment */; break; /* truncated or pre-allocated zeros */", + "fix": "Rewrote 10→23 lines (2 removed) | Also: break; /* clean end of segment */; break; /* truncated or pre-allocated zeros */", "tags": [ "auto-detected", "refactor", @@ -11731,7 +11731,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_commit_durability.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 12\u219214 lines (2 removed)", + "fix": "Rewrote 12→14 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -11746,7 +11746,7 @@ "timestamp": "2026-06-18T07:51:32.985Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-commit-durability/design.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -11762,8 +11762,8 @@ "timestamp": "2026-06-18T07:52:20.595Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_commit_durability.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", - "fix": "Added try/catch block | Also: /* task 3.2 \u2014 Boot Policy Decision Table skeletons (IGNORED,; * 3.3/4.3). Registered so each row is reported as IGNORED, n", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", + "fix": "Added try/catch block | Also: /* task 3.2 — Boot Policy Decision Table skeletons (IGNORED,; * 3.3/4.3). Registered so each row is reported as IGNORED, n", "tags": [ "auto-detected", "error-handling", @@ -11778,7 +11778,7 @@ "timestamp": "2026-06-18T08:04:18.857Z", "error_message": "Missing error handling in enum", "file": "src/include/persist.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block | Also: /* Bounded catch-up window (ha-commit-durability task 3.3, R; * The FINITE deadline a recovery-truncated / behind member i", "tags": [ "auto-detected", @@ -11794,7 +11794,7 @@ "timestamp": "2026-06-18T08:05:05.233Z", "error_message": "Missing error handling in of", "file": "src/controller/persist.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -11811,7 +11811,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_commit_durability.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 7\u219251 lines (4 removed) | Also: TEST_IGNORE_MESSAGE(; \"row 4b (CLEAN + below + no peer + bounded catch-u", + "fix": "Rewrote 7→51 lines (4 removed) | Also: TEST_IGNORE_MESSAGE(; \"row 4b (CLEAN + below + no peer + bounded catch-u", "tags": [ "auto-detected", "refactor", @@ -11826,7 +11826,7 @@ "timestamp": "2026-06-18T08:07:00.123Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_commit_durability.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -11842,7 +11842,7 @@ "timestamp": "2026-06-18T08:18:57.946Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_commit_durability.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -11874,7 +11874,7 @@ "timestamp": "2026-06-18T08:32:37.878Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-commit-durability/design.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -11890,7 +11890,7 @@ "timestamp": "2026-06-18T08:39:54.889Z", "error_message": "Incorrect value in code", "file": ".kiro/specs/ha-commit-durability/design.md", - "root_cause": "Had `durable_floor \u2264 applied_lsn`", + "root_cause": "Had `durable_floor ≤ applied_lsn`", "fix": "Changed to `applied_lsn`", "tags": [ "auto-detected", @@ -11907,7 +11907,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 9\u219214 lines (3 removed) | Also: /* RED-DEMO (temporary): simulate the PRE-FIX bug ; * with a stale-high learned value so the candidate", + "fix": "Rewrote 9→14 lines (3 removed) | Also: /* RED-DEMO (temporary): simulate the PRE-FIX bug ; * with a stale-high learned value so the candidate", "tags": [ "auto-detected", "refactor", @@ -11923,7 +11923,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/myapp.c", "root_cause": "7 lines replaced/restructured", - "fix": "Rewrote 11\u219217 lines (7 removed)", + "fix": "Rewrote 11→17 lines (7 removed)", "tags": [ "auto-detected", "refactor", @@ -11939,7 +11939,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/cluster.c", "root_cause": "6 lines replaced/restructured", - "fix": "Rewrote 10\u219215 lines (6 removed) | Also: uint64_t local_lsn = (uint64_t)atomic_load_explici; &c->ha.applied_lsn, memory_order_acquire);", + "fix": "Rewrote 10→15 lines (6 removed) | Also: uint64_t local_lsn = (uint64_t)atomic_load_explici; &c->ha.applied_lsn, memory_order_acquire);", "tags": [ "auto-detected", "refactor", @@ -11986,7 +11986,7 @@ "timestamp": "2026-06-18T09:07:33.096Z", "error_message": "Missing error handling in unknown", "file": "src/controller/ha.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -12002,8 +12002,8 @@ "timestamp": "2026-06-18T09:08:09.215Z", "error_message": "Missing error handling in unknown", "file": "src/controller/ha.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", - "fix": "Added try/catch block | Also: /* \u2500\u2500 ha-commit-durability task 4.3: pre-promotion eligibili; * Refuse promotion BEFORE the node becomes authoritative CTR", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", + "fix": "Added try/catch block | Also: /* ── ha-commit-durability task 4.3: pre-promotion eligibili; * Refuse promotion BEFORE the node becomes authoritative CTR", "tags": [ "auto-detected", "error-handling", @@ -12018,7 +12018,7 @@ "timestamp": "2026-06-18T09:09:28.348Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_commit_durability.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block | Also: /* Empty first-boot WAL: durable=0, HWM=0 -> ELIGIBLE. A fr; * member is not behind anything and must be promotable (no f", "tags": [ "auto-detected", @@ -12035,7 +12035,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_commit_durability.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 7\u219236 lines (4 removed) | Also: TEST_IGNORE_MESSAGE(; \"row 3 (TRUNCATED + peer + catch-up ok -> FENCE+CA | Also: TEST_IGNORE_MESSAGE(; \"row 5 (snapshot-sentinel CLEAN + below + peer -> ", + "fix": "Rewrote 7→36 lines (4 removed) | Also: TEST_IGNORE_MESSAGE(; \"row 3 (TRUNCATED + peer + catch-up ok -> FENCE+CA | Also: TEST_IGNORE_MESSAGE(; \"row 5 (snapshot-sentinel CLEAN + below + peer -> ", "tags": [ "auto-detected", "refactor", @@ -12098,7 +12098,7 @@ "timestamp": "2026-06-18T09:49:58.840Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_commit_durability.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block | Also: /* Test-injectable send hook (defined in repl.c under XCE_TE; * repl_broadcast from dereferencing the dummy endpoints as r", "tags": [ "auto-detected", @@ -12114,7 +12114,7 @@ "timestamp": "2026-06-18T10:22:00.509Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/ha-commit-durability/tasks.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -12130,7 +12130,7 @@ "timestamp": "2026-06-18T10:25:02.673Z", "error_message": "Missing error handling in unknown", "file": "src/controller/ha.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block | Also: /* ha_cancel_promotion_fence_timer: cancel a pending bounded; * (idempotent; no-op when none is armed). Called when the m", "tags": [ "auto-detected", @@ -12146,7 +12146,7 @@ "timestamp": "2026-06-18T10:26:12.640Z", "error_message": "Missing error handling in XCE_TESTING", "file": "src/controller/ha.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -12163,7 +12163,7 @@ "error_message": "Wrong operator: != should be ==", "file": "src/controller/ha.c", "root_cause": "Used \"!=\" instead of \"==\"", - "fix": "Changed operator != \u2192 ==", + "fix": "Changed operator != → ==", "tags": [ "auto-detected", "operator-fix", @@ -12178,8 +12178,8 @@ "timestamp": "2026-06-18T10:30:18.073Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_commit_durability.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", - "fix": "Added try/catch block | Also: /* task 4.4 \u2014 bounded-window catch-up before FATAL (no synch; * deferred-timer FATAL only after a real bounded failure; fe", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", + "fix": "Added try/catch block | Also: /* task 4.4 — bounded-window catch-up before FATAL (no synch; * deferred-timer FATAL only after a real bounded failure; fe", "tags": [ "auto-detected", "error-handling", @@ -12194,7 +12194,7 @@ "timestamp": "2026-06-18T10:42:48.932Z", "error_message": "Missing error handling in unknown", "file": "src/include/persist.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -12211,7 +12211,7 @@ "error_message": "Significant refactor of ", "file": "src/include/persist.h", "root_cause": "10 lines replaced/restructured", - "fix": "Rewrote 17\u219227 lines (10 removed)", + "fix": "Rewrote 17→27 lines (10 removed)", "tags": [ "auto-detected", "refactor", @@ -12226,7 +12226,7 @@ "timestamp": "2026-06-18T10:44:38.816Z", "error_message": "Missing error handling in unknown", "file": "src/controller/persist.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -12243,7 +12243,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "11 lines replaced/restructured", - "fix": "Rewrote 15\u219222 lines (11 removed)", + "fix": "Rewrote 15→22 lines (11 removed)", "tags": [ "auto-detected", "refactor", @@ -12258,7 +12258,7 @@ "timestamp": "2026-06-18T10:47:24.587Z", "error_message": "Missing error handling in unknown", "file": "src/controller/ha.h", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -12275,7 +12275,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.h", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 6\u21929 lines (3 removed)", + "fix": "Rewrote 6→9 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -12290,7 +12290,7 @@ "timestamp": "2026-06-18T10:48:41.920Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_commit_durability.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -12307,7 +12307,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/ha-commit-durability/design.md", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 8\u21929 lines (2 removed) | Also: | Intent | Distinguish clean log end from corrupti; - FATAL policy: if a recovery-truncated member can | Also: > **Status:** DECISION recorded (task 3.2). No pro; **Decision (Req 2.2, 2.4):** A member that boots w", + "fix": "Rewrote 8→9 lines (2 removed) | Also: | Intent | Distinguish clean log end from corrupti; - FATAL policy: if a recovery-truncated member can | Also: > **Status:** DECISION recorded (task 3.2). No pro; **Decision (Req 2.2, 2.4):** A member that boots w", "tags": [ "auto-detected", "refactor", @@ -12338,7 +12338,7 @@ "timestamp": "2026-06-18T11:07:54.918Z", "error_message": "Missing error handling in unknown", "file": "src/controller/CLAUDE.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -12354,7 +12354,7 @@ "timestamp": "2026-06-18T11:08:39.844Z", "error_message": "Missing error handling in unknown", "file": ".kiro/steering/ha-async-io.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -12370,7 +12370,7 @@ "timestamp": "2026-06-18T11:14:25.437Z", "error_message": "Missing error handling in gap", "file": "src/controller/CLAUDE.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -12400,10 +12400,10 @@ { "id": "bug-760", "timestamp": "2026-06-18T13:00:00Z", - "error_message": "test_commit_barrier_rollout.c:446 test_fact_hook_append_is_retained_and_nonblocking FAIL: Expected 1 Was 0 \u2014 a covering ACK must advance committed_lsn over the retained fact (6/6 deterministic)", + "error_message": "test_commit_barrier_rollout.c:446 test_fact_hook_append_is_retained_and_nonblocking FAIL: Expected 1 Was 0 — a covering ACK must advance committed_lsn over the retained fact (6/6 deterministic)", "file": "src/controller/testsuite/test_commit_barrier_rollout.c", "root_cause": "ha-commit-durability task 2.2 capped the primary commit self-contribution at durable_lsn (advanced asynchronously by the demand-driven group-commit flusher) and wired a durable-advance commit-wake callback. This adjacent-spec (ha-commit-barrier-rollout) test uses a REAL persist+flusher and checks committed_lsn SYNCHRONOUSLY the instant after repl_on_ack(fact_lsn); the flusher has not yet fdatasync'd LSN1, so durable_lsn==0, self-cap==0, majority candidate==0, committed_lsn stays 0. Pre-spec the test passed (self counted the appended tip immediately). Task 2.2 primed durable_lsn in the in-memory stub fixtures (barrier_setup_vcpus) but did NOT update this real-flusher test. The implementation-note claim that this is a pre-existing /tmp-overlay fdatasync-EIO artifact is FALSE in this env: fdatasync returns rc=0 on /tmp here, and the test passed pre-spec.", - "fix": "FIXED \u2014 test_commit_barrier_rollout.c:445: insert persist_flush(g_fp) + assert persist_durable_lsn>=fact_lsn BEFORE repl_on_ack, so the covering ACK (commit_on_ack runs synchronously inside repl_on_ack) deterministically completes the durable majority. Mirrors task 2.2's durable_lsn priming of the in-memory barrier_setup_vcpus stubs. Verified: was 6/6 deterministic FAIL, now fact_hook PASS across 4 runs + meson controller suite Ok:1 Fail:0, all groups 0 Failures. Production was never affected (RETAIN facts have no waiter; WAIT-mode replies block on commit barrier commit.c:334).", + "fix": "FIXED — test_commit_barrier_rollout.c:445: insert persist_flush(g_fp) + assert persist_durable_lsn>=fact_lsn BEFORE repl_on_ack, so the covering ACK (commit_on_ack runs synchronously inside repl_on_ack) deterministically completes the durable majority. Mirrors task 2.2's durable_lsn priming of the in-memory barrier_setup_vcpus stubs. Verified: was 6/6 deterministic FAIL, now fact_hook PASS across 4 runs + meson controller suite Ok:1 Fail:0, all groups 0 Failures. Production was never affected (RETAIN facts have no waiter; WAIT-mode replies block on commit barrier commit.c:334).", "tags": [ "ha-commit-durability", "commit-barrier-rollout", @@ -12474,7 +12474,7 @@ "timestamp": "2026-06-18T14:55:00.167Z", "error_message": "Missing error handling in unknown", "file": ".kiro/steering/error-handling.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -12488,10 +12488,10 @@ { "id": "bug-765", "timestamp": "2026-06-18T14:30:00Z", - "error_message": "scheduler-controller-msg-gap marked 36/36 [x] but NOT truly implemented: schedUpdateJob notify path hits the bulk CREATE path (sched_update_existing_job is dead code) \u2192 corrupts/duplicates live job state, no -ENOENT; Req 3 broken at dequeue, Req 6 stores nothing (rusage never stored).", - "file": "src/scheduler/ (sched_update_existing_job dead; schedUpdateJob \u2192 CREATE path)", + "error_message": "scheduler-controller-msg-gap marked 36/36 [x] but NOT truly implemented: schedUpdateJob notify path hits the bulk CREATE path (sched_update_existing_job is dead code) → corrupts/duplicates live job state, no -ENOENT; Req 3 broken at dequeue, Req 6 stores nothing (rusage never stored).", + "file": "src/scheduler/ (sched_update_existing_job dead; schedUpdateJob → CREATE path)", "root_cause": "Tasks were checked [x] without the dequeue/update path and rusage storage actually being wired; gap surfaced only by cross-task re-audit (per-task review missed it).", - "fix": "FIXED 2026-06-18 (commits 5c1f715,93124c5,080ff78,276e26c): G-8/G-4 (route notify job-update to in-place sched_update_existing_job, all mutable fields, -ENOENT no-create); G-5 (store rusage on xce_sched_job_t via sched_apply_job_rusage + free_JobRusages_ptr); G-3 (free all 5 killer op payloads with generated deep-freers); G-7 (doc reconciliation \u2014 sched_op_name stays static in scheduler.c, lib must not depend on scheduler headers); G-6 (real white-box tests: test_handlers.c 6/6 for 9.2/9.3, test_job_update.c 10/10 for 9.4/9.5/9.7, registered as meson tests). Scheduler suite 31/31 GREEN, ASAN-clean, boundary-clean. validate-impl GO. RESIDUAL (documented, non-blocking): 9.6 dequeue worker-resvals APPLY unit test (dormant path R-3 \u2014 enqueue tested), 9.8 documented compile-time absence, 9.9 log-capture gated, 9.2 -ENOMEM fault-injection.", + "fix": "FIXED 2026-06-18 (commits 5c1f715,93124c5,080ff78,276e26c): G-8/G-4 (route notify job-update to in-place sched_update_existing_job, all mutable fields, -ENOENT no-create); G-5 (store rusage on xce_sched_job_t via sched_apply_job_rusage + free_JobRusages_ptr); G-3 (free all 5 killer op payloads with generated deep-freers); G-7 (doc reconciliation — sched_op_name stays static in scheduler.c, lib must not depend on scheduler headers); G-6 (real white-box tests: test_handlers.c 6/6 for 9.2/9.3, test_job_update.c 10/10 for 9.4/9.5/9.7, registered as meson tests). Scheduler suite 31/31 GREEN, ASAN-clean, boundary-clean. validate-impl GO. RESIDUAL (documented, non-blocking): 9.6 dequeue worker-resvals APPLY unit test (dormant path R-3 — enqueue tested), 9.8 documented compile-time absence, 9.9 log-capture gated, 9.2 -ENOMEM fault-injection.", "tags": [ "scheduler", "controller", @@ -12527,7 +12527,7 @@ "error_message": "Significant refactor of ", "file": "src/scheduler/src/schedobj.c", "root_cause": "7 lines replaced/restructured", - "fix": "Rewrote 18\u219212 lines (7 removed)", + "fix": "Rewrote 18→12 lines (7 removed)", "tags": [ "auto-detected", "refactor", @@ -12559,7 +12559,7 @@ "error_message": "Significant refactor of ", "file": "src/scheduler/src/scheduler.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 18\u219233 lines (2 removed)", + "fix": "Rewrote 18→33 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -12575,7 +12575,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/scheduler-controller-msg-gap/tasks.md", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 9\u219211 lines (4 removed)", + "fix": "Rewrote 9→11 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -12591,7 +12591,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/scheduler-controller-msg-gap/design.md", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 3\u21925 lines (2 removed) | Also: #### New function: `xce_sched_op_name`; Signature: char* xce_sched_op_name(xce_sched_op_t ", + "fix": "Rewrote 3→5 lines (2 removed) | Also: #### New function: `xce_sched_op_name`; Signature: char* xce_sched_op_name(xce_sched_op_t ", "tags": [ "auto-detected", "refactor", @@ -12607,7 +12607,7 @@ "error_message": "Wrong reference: xce_sched_op_name should be sched_op_name", "file": ".kiro/specs/scheduler-controller-msg-gap/design.md", "root_cause": "Used \"xce_sched_op_name\" instead of \"sched_op_name\"", - "fix": "Changed xce_sched_op_name \u2192 sched_op_name", + "fix": "Changed xce_sched_op_name → sched_op_name", "tags": [ "auto-detected", "wrong-reference", @@ -12655,7 +12655,7 @@ "error_message": "Significant refactor of ", "file": "src/scheduler/testsuite/test_handlers.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 5\u219211 lines (2 removed)", + "fix": "Rewrote 5→11 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -12671,7 +12671,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/scheduler-controller-msg-gap/tasks.md", "root_cause": "7 lines replaced/restructured", - "fix": "Rewrote 7\u21927 lines (7 removed)", + "fix": "Rewrote 7→7 lines (7 removed)", "tags": [ "auto-detected", "refactor", @@ -12686,8 +12686,8 @@ "timestamp": "2026-06-19T13:00:00Z", "error_message": "3-node rolling-SIGKILL soak round 3: 'no new primary elected within 40s'; soak reports 'DATA LOSS 5/5 jobs vanished'. Both surviving nodes recovered RECOVER_TRUNCATED after SIGKILL (torn WAL tail) and stay permanently fenced/non-promotable even though durable_floor==cluster_hwm (they hold ALL committed data).", "file": "src/controller/ha.c (raft_cb_become_leader promotion fence release) + persist.c (recovery_truncated verdict stickiness); design ha-commit-durability Req 2.2/2.6 + Boot Policy Decision Table", - "root_cause": "A SIGKILL tears the WAL tail; on restart replay_wal hits a CRC-break/short-read at the tail -> RECOVER_TRUNCATED (sticky flag). The promotion-fence release condition requires `!recovery_truncated`, but the truncated flag is NEVER cleared by catch-up \u2014 so a node whose intact durable prefix (durable_floor) already reaches/exceeds the cluster committed high-water (durable_floor>=hwm, i.e. holds ALL committed entries; the torn tail is entirely UNCOMMITTED) stays permanently non-promotable. The design assumed 'TRUNCATED => provably below HWM', which is FALSE for a torn TAIL above the committed prefix. When all survivors are in this state (after multiple rolling failovers), none can promote => election/availability WEDGE. Observed: ctrl-1 recovery_truncated=1 durable_floor=13 cluster_hwm=13, PreVote-DENY peers reason=catch_up, indefinitely.", - "fix": "DATA-LOSS dimension RESOLVED by the bug-778 fix (commit 64ab210): the cold-restart loss was the CRC-chain clobber; with the chain intact, cold restart recovers all committed jobs (soak 5/5, joblist 6/6 after cold restart). The AVAILABILITY-WEDGE dimension was a CONSEQUENCE of the spurious RECOVER_TRUNCATED from the broken chain \u2014 no longer triggered (nodes now recover CLEAN; round-3 re-elects normally). RESIDUAL HARDENING (latent, not currently triggered): the promotion fence keeps recovery_truncated sticky even when durable_floor>=HWM, so a GENUINE torn-tail that leaves a node RECOVER_TRUNCATED while holding all committed data could still over-fence. Optional follow-up: clear the truncated fence when durable_floor>=cluster_hwm (the torn tail is provably uncommitted). Not blocking \u2014 the systematic trigger is gone.", + "root_cause": "A SIGKILL tears the WAL tail; on restart replay_wal hits a CRC-break/short-read at the tail -> RECOVER_TRUNCATED (sticky flag). The promotion-fence release condition requires `!recovery_truncated`, but the truncated flag is NEVER cleared by catch-up — so a node whose intact durable prefix (durable_floor) already reaches/exceeds the cluster committed high-water (durable_floor>=hwm, i.e. holds ALL committed entries; the torn tail is entirely UNCOMMITTED) stays permanently non-promotable. The design assumed 'TRUNCATED => provably below HWM', which is FALSE for a torn TAIL above the committed prefix. When all survivors are in this state (after multiple rolling failovers), none can promote => election/availability WEDGE. Observed: ctrl-1 recovery_truncated=1 durable_floor=13 cluster_hwm=13, PreVote-DENY peers reason=catch_up, indefinitely.", + "fix": "DATA-LOSS dimension RESOLVED by the bug-778 fix (commit 64ab210): the cold-restart loss was the CRC-chain clobber; with the chain intact, cold restart recovers all committed jobs (soak 5/5, joblist 6/6 after cold restart). The AVAILABILITY-WEDGE dimension was a CONSEQUENCE of the spurious RECOVER_TRUNCATED from the broken chain — no longer triggered (nodes now recover CLEAN; round-3 re-elects normally). RESIDUAL HARDENING (latent, not currently triggered): the promotion fence keeps recovery_truncated sticky even when durable_floor>=HWM, so a GENUINE torn-tail that leaves a node RECOVER_TRUNCATED while holding all committed data could still over-fence. Optional follow-up: clear the truncated fence when durable_floor>=cluster_hwm (the torn tail is provably uncommitted). Not blocking — the systematic trigger is gone.", "tags": [ "ha", "convergence", @@ -12742,7 +12742,7 @@ "timestamp": "2026-06-19T13:59:32.538Z", "error_message": "Missing error handling in unknown", "file": "src/controller/testsuite/test_commit_durability.c", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -12759,7 +12759,7 @@ "error_message": "Significant refactor of ", "file": "src/include/persist.h", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 5\u219212 lines (3 removed)", + "fix": "Rewrote 5→12 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -12775,7 +12775,7 @@ "error_message": "Wrong reference: hdr should be payload_len", "file": "src/controller/persist.c", "root_cause": "Used \"hdr\" instead of \"payload_len\"", - "fix": "Changed hdr \u2192 payload_len", + "fix": "Changed hdr → payload_len", "tags": [ "auto-detected", "wrong-reference", @@ -12791,7 +12791,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/persist.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 3\u21925 lines (2 removed)", + "fix": "Rewrote 3→5 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -12807,7 +12807,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ha.c", "root_cause": "8 lines replaced/restructured", - "fix": "Rewrote 9\u21927 lines (8 removed)", + "fix": "Rewrote 9→7 lines (8 removed)", "tags": [ "auto-detected", "refactor", @@ -12823,7 +12823,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_repl_apply.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 11\u219213 lines (4 removed)", + "fix": "Rewrote 11→13 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -12839,7 +12839,7 @@ "error_message": "Wrong reference: NULL should be type", "file": "src/controller/testsuite/test_ha_role_lifecycle.c", "root_cause": "Used \"NULL\" instead of \"type\"", - "fix": "Changed NULL \u2192 type", + "fix": "Changed NULL → type", "tags": [ "auto-detected", "wrong-reference", @@ -12855,7 +12855,7 @@ "error_message": "xce_test jobsub prints 'jobsubmit ok: JobId: 63-15--1-1015808' (uninitialized/garbage jobid) on repeated/mid-failover submits; controller then logs 'handle_jobsubmit fail to lock job 63-15--1-...'. Client claims success with a malformed id it never got a clean ack for.", "file": "src/controller/testsuite/test.c (cmd_jobsub)", "root_cause": "On the 2nd+ submit (and during failover windows) the jobsub reply/jobid is not cleanly populated, leaving an uninitialized jobid struct; cmd_jobsub still prints 'jobsubmit ok' with the garbage id instead of verifying the controller reply status and failing. Forces E2E scripts (test-commit-durability.sh) to strict-regex-filter the 4-field id to avoid chasing phantom jobs.", - "fix": "PENDING \u2014 jobsub must report success ONLY on a clean ack with a valid jobid; otherwise print a typed error (REDIRECT/NOMASTER/NOT_READY/TIMEOUT) and set non-zero status. Part of the xce-test-e2e-primitives work (exit-code-as-verdict + $jobid auto-var). | STRENGTHENED 2026-06-19 (commit 8609976): `jobid != 0` gate was insufficient \u2014 a stale non-zero reply id (63-15--1-1015808) from a non-jobid reply during failover slipped through. New gate requires the echoed jobid == the locally idgen-generated jobsubmit.jobid (controller commits/echoes exactly that id, job.c:616). Surfaced by the live durability soak (task 7.2).", + "fix": "PENDING — jobsub must report success ONLY on a clean ack with a valid jobid; otherwise print a typed error (REDIRECT/NOMASTER/NOT_READY/TIMEOUT) and set non-zero status. Part of the xce-test-e2e-primitives work (exit-code-as-verdict + $jobid auto-var). | STRENGTHENED 2026-06-19 (commit 8609976): `jobid != 0` gate was insufficient — a stale non-zero reply id (63-15--1-1015808) from a non-jobid reply during failover slipped through. New gate requires the echoed jobid == the locally idgen-generated jobsubmit.jobid (controller commits/echoes exactly that id, job.c:616). Surfaced by the live durability soak (task 7.2).", "tags": [ "xce_test", "jobsub", @@ -12877,7 +12877,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test.c", "root_cause": "13 lines replaced/restructured", - "fix": "Rewrote 27\u219211 lines (13 removed) | Also: *; * XCE_TEST_VERDICT_FLAG gates the new exit-code/ab", + "fix": "Rewrote 27→11 lines (13 removed) | Also: *; * XCE_TEST_VERDICT_FLAG gates the new exit-code/ab", "tags": [ "auto-detected", "refactor", @@ -12909,7 +12909,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test.c", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 9\u21923 lines (5 removed)", + "fix": "Rewrote 9→3 lines (5 removed)", "tags": [ "auto-detected", "refactor", @@ -12925,7 +12925,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test.c", "root_cause": "5 lines replaced/restructured", - "fix": "Rewrote 7\u21923 lines (5 removed) | Also: #if JOBSUB_RED_FORCE_SUCCESS; /* RED only: simulate a non-clean ack that surface", + "fix": "Rewrote 7→3 lines (5 removed) | Also: #if JOBSUB_RED_FORCE_SUCCESS; /* RED only: simulate a non-clean ack that surface", "tags": [ "auto-detected", "refactor", @@ -12941,7 +12941,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test.c", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 38\u219285 lines (2 removed) | Also: * g_last_jobcount most recent job count; -1 means ; * cmd_joblist (task 5.1).", + "fix": "Rewrote 38→85 lines (2 removed) | Also: * g_last_jobcount most recent job count; -1 means ; * cmd_joblist (task 5.1).", "tags": [ "auto-detected", "refactor", @@ -12973,7 +12973,7 @@ "error_message": "Significant refactor of ", "file": "src/tools/test-commit-durability.sh", "root_cause": "31 lines replaced/restructured", - "fix": "Rewrote 63\u2192115 lines (31 removed) | Also: if ! wait_for_primary 40; then; fail \"cluster did not elect an initial primary wit | Also: # Wait until some live node is CTRL_PRIMARY and th; wait_for_primary() { | Also: script=$(printf 'set masters %s\\nclient\\n' \"$MASTE; script+=$(printf 'expect-job %s\\n' \"$jid\") | Also: # format-agnostically, instead of re-imposing a br; # submit did not cleanly ack, jobsub fails -> sess", + "fix": "Rewrote 63→115 lines (31 removed) | Also: if ! wait_for_primary 40; then; fail \"cluster did not elect an initial primary wit | Also: # Wait until some live node is CTRL_PRIMARY and th; wait_for_primary() { | Also: script=$(printf 'set masters %s\\nclient\\n' \"$MASTE; script+=$(printf 'expect-job %s\\n' \"$jid\") | Also: # format-agnostically, instead of re-imposing a br; # submit did not cleanly ack, jobsub fails -> sess", "tags": [ "auto-detected", "refactor", @@ -12989,7 +12989,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test.c", "root_cause": "7 lines replaced/restructured", - "fix": "Rewrote 8\u219220 lines (7 removed) | Also: if (jobid == 0) {; /* Non-negative rc but no JobId was written: not a", + "fix": "Rewrote 8→20 lines (7 removed) | Also: if (jobid == 0) {; /* Non-negative rc but no JobId was written: not a", "tags": [ "auto-detected", "refactor", @@ -13003,9 +13003,9 @@ "id": "bug-795", "timestamp": "2026-06-19", "error_message": "Cold-restart-all (kill ALL 3 controllers + restart) non-deterministically loses a committed job: a job that returned 'jobsubmit ok' and survived every rolling single-node failover round vanishes after a full-cluster restart (expect-jobs expected N, actual N-2; expect-job not found). Recovery logs show WAL truncate.", - "file": "src/controller/{persist.c recovery/truncate, commit.c commit-gate, ha.c promotion-fence/election} \u2014 UPSTREAM ha-commit-durability Phase 6.2", - "root_cause": "Commit-durability race under TOTAL-cluster-outage recovery. Acked (committed) entry not durable on a fault-tolerant majority before all nodes die, OR recovery truncates committed entries on cold restart. Non-deterministic (passed 1 run, failed next). Surfaced on overlayfs sandbox (NOT a real fsync-able fs) \u2014 must be reconfirmed on real hardware. UPDATE: with the hardened (trustworthy) soak, the loss ALSO reproduces in ROLLING single-node failover with the cluster confirmed serving (~1/3 runs) \u2014 broader than cold-restart-all; bug-661 not fully closed for rolling failover. Separate finding: >60s post-kill election (liveness) every run, always recovers. CORRECTION: NOT an overlay-fdatasync artifact \u2014 reproduces OUTSIDE the container (user's real host) AND on an in-sandbox real ext2/3 fs with ZERO fdatasync EIO. Real controller durability bug + a coupled liveness stall (>60s / stuck election after killing ctrl-1). Phase 6.1 answered (REAL); now at 6.2 root-cause.", - "fix": "NOT FIXED HERE \u2014 out of boundary for xce-test-e2e-primitives (this spec changes no controller/replication code). Routed to spec ha-commit-durability owner. The converted durability soak (src/tools/test-commit-durability.sh, new run_cold_restart_recovery phase) correctly detects it via the exit-code verdict.", + "file": "src/controller/{persist.c recovery/truncate, commit.c commit-gate, ha.c promotion-fence/election} — UPSTREAM ha-commit-durability Phase 6.2", + "root_cause": "Commit-durability race under TOTAL-cluster-outage recovery. Acked (committed) entry not durable on a fault-tolerant majority before all nodes die, OR recovery truncates committed entries on cold restart. Non-deterministic (passed 1 run, failed next). Surfaced on overlayfs sandbox (NOT a real fsync-able fs) — must be reconfirmed on real hardware. UPDATE: with the hardened (trustworthy) soak, the loss ALSO reproduces in ROLLING single-node failover with the cluster confirmed serving (~1/3 runs) — broader than cold-restart-all; bug-661 not fully closed for rolling failover. Separate finding: >60s post-kill election (liveness) every run, always recovers. CORRECTION: NOT an overlay-fdatasync artifact — reproduces OUTSIDE the container (user's real host) AND on an in-sandbox real ext2/3 fs with ZERO fdatasync EIO. Real controller durability bug + a coupled liveness stall (>60s / stuck election after killing ctrl-1). Phase 6.1 answered (REAL); now at 6.2 root-cause.", + "fix": "NOT FIXED HERE — out of boundary for xce-test-e2e-primitives (this spec changes no controller/replication code). Routed to spec ha-commit-durability owner. The converted durability soak (src/tools/test-commit-durability.sh, new run_cold_restart_recovery phase) correctly detects it via the exit-code verdict.", "tags": [ "ha-commit-durability", "commit-durability", @@ -13049,7 +13049,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/xce-test-e2e-primitives/spec.json", "root_cause": "2 lines replaced/restructured", - "fix": "Rewrote 3\u21928 lines (2 removed)", + "fix": "Rewrote 3→8 lines (2 removed)", "tags": [ "auto-detected", "refactor", @@ -13064,7 +13064,7 @@ "timestamp": "2026-06-20T01:05:04.547Z", "error_message": "Missing error handling in unknown", "file": "src/tools/test-commit-durability.sh", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -13080,8 +13080,8 @@ "timestamp": "2026-06-20T01:05:25.091Z", "error_message": "Incorrect value in code", "file": "src/tools/test-commit-durability.sh", - "root_cause": "Had \"$label: DATA LOSS \u2014 a committed job vanished (bug", - "fix": "Changed to \"$label: DATA LOSS \u2014 a committed job vanished from", + "root_cause": "Had \"$label: DATA LOSS — a committed job vanished (bug", + "fix": "Changed to \"$label: DATA LOSS — a committed job vanished from", "tags": [ "auto-detected", "wrong-value", @@ -13097,7 +13097,7 @@ "error_message": "Significant refactor of ", "file": "src/tools/test-commit-durability.sh", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 8\u219212 lines (4 removed)", + "fix": "Rewrote 8→12 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -13113,7 +13113,7 @@ "error_message": "Significant refactor of ", "file": "src/tools/test-commit-durability.sh", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 8\u219210 lines (4 removed)", + "fix": "Rewrote 8→10 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -13128,7 +13128,7 @@ "timestamp": "2026-06-20T01:21:21.789Z", "error_message": "Missing error handling in unknown", "file": "src/tools/test-commit-durability.sh", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -13144,7 +13144,7 @@ "timestamp": "2026-06-20T01:27:31.000Z", "error_message": "Missing error handling in unknown", "file": ".kiro/specs/xce-test-e2e-primitives/tasks.md", - "root_cause": "Code path had no error handling \u2014 exceptions would propagate uncaught", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", "fix": "Added try/catch block", "tags": [ "auto-detected", @@ -13193,7 +13193,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/controller-lsid/requirements.md", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 6\u21928 lines (3 removed) | Also: ### Requirement 5: Committed configuration version; **Objective:** As an operator, I want the configur", + "fix": "Rewrote 6→8 lines (3 removed) | Also: ### Requirement 5: Committed configuration version; **Objective:** As an operator, I want the configur", "tags": [ "auto-detected", "refactor", @@ -13209,7 +13209,7 @@ "error_message": "Significant refactor of ", "file": ".kiro/specs/controller-status/design.md", "root_cause": "12 lines replaced/restructured", - "fix": "Rewrote 16\u219213 lines (12 removed) | Also: clusterName @0: Text;; primaryName @1: Text; # empty \u21d2 \"unknown\" | Also: /* failover-aware read (is_write=false) */; - Postconditions: on success `*status` owns a deco | Also: - `configChecksum`/`configVersion` equal across tw; - `ctrlstatus` against a healthy 3-node cluster pr | Also: - Answerable on **any** node and across failover \u2014; - Report a content checksum of the committed confi", + "fix": "Rewrote 16→13 lines (12 removed) | Also: clusterName @0: Text;; primaryName @1: Text; # empty ⇒ \"unknown\" | Also: /* failover-aware read (is_write=false) */; - Postconditions: on success `*status` owns a deco | Also: - `configChecksum`/`configVersion` equal across tw; - `ctrlstatus` against a healthy 3-node cluster pr | Also: - Answerable on **any** node and across failover —; - Report a content checksum of the committed confi", "tags": [ "auto-detected", "refactor", @@ -13257,7 +13257,7 @@ "error_message": "Wrong reference: Data should be Text", "file": "src/schema/controller.capnp", "root_cause": "Used \"Data\" instead of \"Text\"", - "fix": "Changed Data \u2192 Text", + "fix": "Changed Data → Text", "tags": [ "auto-detected", "wrong-reference", @@ -13273,7 +13273,7 @@ "error_message": "Wrong reference: xce_data_t should be char", "file": "src/include/typemap.h", "root_cause": "Used \"xce_data_t\" instead of \"char\"", - "fix": "Changed xce_data_t \u2192 char", + "fix": "Changed xce_data_t → char", "tags": [ "auto-detected", "wrong-reference", @@ -13305,7 +13305,7 @@ "error_message": "Wrong operator: >= should be ==", "file": "src/controller/testsuite/test.c", "root_cause": "Used \">=\" instead of \"==\"", - "fix": "Changed operator >= \u2192 ==", + "fix": "Changed operator >= → ==", "tags": [ "auto-detected", "operator-fix", @@ -13321,7 +13321,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ctrlstatus.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 23\u219273 lines (3 removed)", + "fix": "Rewrote 23→73 lines (3 removed)", "tags": [ "auto-detected", "refactor", @@ -13337,7 +13337,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/ctrlstatus.c", "root_cause": "3 lines replaced/restructured", - "fix": "Rewrote 3\u219213 lines (3 removed) | Also: /* primary identity \u2014 guarded by ha->lock (Req 2.2; st->primaryName = (ha->primary_host[0] != '\\0') ? ", + "fix": "Rewrote 3→13 lines (3 removed) | Also: /* primary identity — guarded by ha->lock (Req 2.2; st->primaryName = (ha->primary_host[0] != '\\0') ? ", "tags": [ "auto-detected", "refactor", @@ -13353,7 +13353,7 @@ "error_message": "Significant refactor of ", "file": "src/controller/testsuite/test_ctrlstatus.c", "root_cause": "4 lines replaced/restructured", - "fix": "Rewrote 13\u219216 lines (4 removed)", + "fix": "Rewrote 13→16 lines (4 removed)", "tags": [ "auto-detected", "refactor", @@ -13367,7 +13367,7 @@ "id": "bug-817", "timestamp": "2026-06-19", "error_message": "client op (joblist, ctrlstatus, wait-primary) returns EINVAL / 'Invalid argument' and fails to reach the live primary when a DEAD controller is FIRST in the masters list; with a live-only masters list the same op succeeds in ~46ms.", - "file": "src/lib/client_failover.c / client.c failover-rotation (libxce) \u2014 UPSTREAM, not controller-status", + "file": "src/lib/client_failover.c / client.c failover-rotation (libxce) — UPSTREAM, not controller-status", "root_cause": "libxce client failover does not cleanly advance past a dead FIRST master to the live nodes; surfaces a typed EINVAL instead of rotating. Pre-existing (controller-status consumes failover unchanged). Surfaced by controller-status task 5.1 E2E (authoritative wait-primary timed out after a primary kill while a dead node led the masters list, even though a new primary was serving).", "fix": "NOT fixed in controller-status (out of boundary). Route to libxce-failover. Confirmed affects plain joblist identically. Workaround: ensure the masters list does not lead with a known-dead node, or restart killed nodes (as the durability soak does).", "tags": [ @@ -13384,6 +13384,371 @@ ], "occurrences": 1, "last_seen": "2026-06-19" + }, + { + "id": "bug-818", + "timestamp": "2026-06-20T06:11:49.084Z", + "error_message": "Wrong condition in logic", + "file": "src/controller/testsuite/test.c", + "root_cause": "Condition was: if (!g_continue_on_error)", + "fix": "Changed to: if (!g_continue_on_error && !g_interactive)", + "tags": [ + "auto-detected", + "logic-fix", + "c" + ], + "related_bugs": [], + "occurrences": 2, + "last_seen": "2026-06-20T06:14:41.073Z" + }, + { + "id": "bug-819", + "timestamp": "2026-06-20T06:14:51.644Z", + "error_message": "Significant refactor of ", + "file": "src/controller/testsuite/test.c", + "root_cause": "4 lines replaced/restructured", + "fix": "Rewrote 4→8 lines (4 removed) | Also: /* Session verdict toggle (xce-test-e2e-primitives; * `set continue-on-error` keeps the read loop runn", + "tags": [ + "auto-detected", + "refactor", + "c" + ], + "related_bugs": [], + "occurrences": 2, + "last_seen": "2026-06-20T06:15:17.616Z" + }, + { + "id": "bug-820", + "timestamp": "2026-06-20T06:16:34.685Z", + "error_message": "Incorrect value in code", + "file": ".kiro/specs/xce-test-e2e-primitives/tasks.md", + "root_cause": "Had `g_interactive = isatty(STDIN_FILENO)`", + "fix": "Changed to `-e`", + "tags": [ + "auto-detected", + "wrong-value", + "md" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T06:16:34.685Z" + }, + { + "id": "bug-821", + "timestamp": "2026-06-20T08:14:43.577Z", + "error_message": "Incorrect value in code", + "file": ".kiro/specs/controller-status/requirements.md", + "root_cause": "Had 's identity and role, the configured controller to", + "fix": "Changed to 's identity and role, the configured controller to", + "tags": [ + "auto-detected", + "wrong-value", + "md" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T08:14:43.577Z" + }, + { + "id": "bug-822", + "timestamp": "2026-06-20T08:15:28.319Z", + "error_message": "Significant refactor of ", + "file": ".kiro/specs/controller-status/spec.json", + "root_cause": "6 lines replaced/restructured", + "fix": "Rewrote 24→30 lines (6 removed)", + "tags": [ + "auto-detected", + "refactor", + "json" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T08:15:28.319Z" + }, + { + "id": "bug-823", + "timestamp": "2026-06-20T09:03:03.345Z", + "error_message": "Significant refactor of ", + "file": "src/controller/ctrlstatus.c", + "root_cause": "4 lines replaced/restructured", + "fix": "Rewrote 4→12 lines (4 removed)", + "tags": [ + "auto-detected", + "refactor", + "c" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T09:03:03.345Z" + }, + { + "id": "bug-824", + "timestamp": "2026-06-20T09:30:23.040Z", + "error_message": "Function not marked async", + "file": ".kiro/specs/controller-status/tasks.md", + "root_cause": "Function uses await but wasn't declared async", + "fix": "Added async modifier", + "tags": [ + "auto-detected", + "async-fix", + "md" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T09:30:23.040Z" + }, + { + "id": "bug-825", + "timestamp": "2026-06-20T09:31:24.076Z", + "error_message": "Function not marked async", + "file": ".kiro/specs/controller-status/spec.json", + "root_cause": "Function uses await but wasn't declared async", + "fix": "Added async modifier", + "tags": [ + "auto-detected", + "async-fix", + "json" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T09:31:24.076Z" + }, + { + "id": "bug-826", + "timestamp": "2026-06-20T10:10:14.423Z", + "error_message": "Significant refactor of ", + "file": "src/lib/utils.c", + "root_cause": "13 lines replaced/restructured", + "fix": "Rewrote 26→50 lines (13 removed)", + "tags": [ + "auto-detected", + "refactor", + "c" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T10:10:14.423Z" + }, + { + "id": "bug-827", + "timestamp": "2026-06-20T18:30:00.000Z", + "error_message": "AddressSanitizer: global-buffer-overflow READ in xce_strerror_r (lib/utils.c:335), triggered from cmd_ctrlstatus (test.c:1273) when ctrlstatus failover exhausts and returns E_XCE_NOT_READY (-10013) on a primary-less cluster", + "file": "src/lib/utils.c", + "root_cause": "Enum/table drift: errnox.h E_XCE_* enum gained NOMASTER(10011)/REDIRECT(10012)/NOT_READY(10013)/STANDBY(10014), bumping E_XCE_MAX to 10015, but xce_err_str[] still had only 10 strings + a NULL (11 slots, 88 bytes). xce_strerror_r's only upper guard was errnum>=E_XCE_MAX, so 10011-10014 passed the gate and indexed past the table: NOMASTER->xce_err_str[10]==NULL (strncpy NULL deref); NOT_READY->xce_err_str[12] = 8 bytes past the 88-byte array (the ASAN OOB). Pre-existing libxce foundation bug; controller-status ctrlstatus was the first caller to format these HA codes on a failing cluster.", + "fix": "Filled xce_err_str[] with strings for all 14 codes (enum order); hardened xce_strerror_r to bound against the ACTUAL array size (idx>=nstr) + reject NULL slots (degrade to 'unknown error', never OOB) + null-terminate; added a _Static_assert tying sizeof(xce_err_str) to (E_XCE_MAX-XCE_ERRNO_BASE) so future enum additions without a string fail the build. RED: new test_strerror_all_xce_codes SEGV'd at utils.c:335 under ASAN; GREEN: lib suite Ok:1 Fail:0. Commit 1aa0761.", + "tags": [ + "global-buffer-overflow", + "asan", + "xce_strerror_r", + "errnox", + "enum-table-drift", + "libxce", + "controller-status", + "ctrlstatus", + "E_XCE_NOT_READY" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T18:30:00.000Z" + }, + { + "id": "bug-828", + "timestamp": "2026-06-20T10:19:56.678Z", + "error_message": "Function not marked async", + "file": ".kiro/specs/ha-convergence-recovery/research.md", + "root_cause": "Function uses await but wasn't declared async", + "fix": "Added async modifier", + "tags": [ + "auto-detected", + "async-fix", + "md" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T10:19:56.678Z" + }, + { + "id": "bug-829", + "timestamp": "2026-06-20T11:44:25.496Z", + "error_message": "Incorrect value in code", + "file": "src/controller/meson.build", + "root_cause": "Had 'ctrlstatus.c'", + "fix": "Changed to 'testsuite/test_ha_heartbeat_committed.c'", + "tags": [ + "auto-detected", + "wrong-value", + "build" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T11:44:25.496Z" + }, + { + "id": "bug-830", + "timestamp": "2026-06-20T11:45:26.943Z", + "error_message": "Missing guard clause", + "file": "src/controller/ha.c", + "root_cause": "No early return/throw for edge case: !c", + "fix": "Added guard clause: if (!c)", + "tags": [ + "auto-detected", + "guard-clause", + "c" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T11:45:26.943Z" + }, + { + "id": "bug-831", + "timestamp": "2026-06-20T11:54:30.060Z", + "error_message": "Incorrect value in code", + "file": "src/controller/testsuite/test_ha_heartbeat_committed.c", + "root_cause": "Had \"xce/cluster_def.h\"", + "fix": "Changed to \"typemap.h\"", + "tags": [ + "auto-detected", + "wrong-value", + "c" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T11:54:30.060Z" + }, + { + "id": "bug-832", + "timestamp": "2026-06-20T11:56:28.240Z", + "error_message": "Missing guard clause", + "file": "src/controller/ha.c", + "root_cause": "No early return/throw for edge case: !c || !hb", + "fix": "Added guard clause: if (!c || !hb)", + "tags": [ + "auto-detected", + "guard-clause", + "c" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T11:56:28.240Z" + }, + { + "id": "bug-833", + "timestamp": "2026-06-20T12:09:25.213Z", + "error_message": "Significant refactor of ", + "file": "src/controller/ha.c", + "root_cause": "2 lines replaced/restructured", + "fix": "Rewrote 12→26 lines (2 removed)", + "tags": [ + "auto-detected", + "refactor", + "c" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T12:09:25.213Z" + }, + { + "id": "bug-834", + "timestamp": "2026-06-20T12:22:39.011Z", + "error_message": "Incorrect value in code", + "file": ".kiro/specs/ha-heartbeat-committed-propagation/tasks.md", + "root_cause": "Had `heartbeatmsg`", + "fix": "Changed to `xce-build.sh controller`", + "tags": [ + "auto-detected", + "wrong-value", + "md" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T12:22:39.011Z" + }, + { + "id": "bug-835", + "timestamp": "2026-06-20T12:26:15.999Z", + "error_message": "Incorrect value in code", + "file": "src/controller/meson.build", + "root_cause": "Had 'ctrlstatus.c'", + "fix": "Changed to 'testsuite/test_ha_vote_floor_heartbeat.c'", + "tags": [ + "auto-detected", + "wrong-value", + "build" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T12:26:15.999Z" + }, + { + "id": "bug-836", + "timestamp": "2026-06-20T12:31:51.359Z", + "error_message": "Function not marked async", + "file": ".kiro/specs/ha-heartbeat-committed-propagation/tasks.md", + "root_cause": "Function uses await but wasn't declared async", + "fix": "Added async modifier", + "tags": [ + "auto-detected", + "async-fix", + "md" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T12:31:51.359Z" + }, + { + "id": "bug-837", + "timestamp": "2026-06-20T12:32:25.549Z", + "error_message": "Function not marked async", + "file": ".kiro/specs/ha-heartbeat-committed-propagation/spec.json", + "root_cause": "Function uses await but wasn't declared async", + "fix": "Added async modifier", + "tags": [ + "auto-detected", + "async-fix", + "json" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T12:32:25.549Z" + }, + { + "id": "bug-828", + "timestamp": "2026-06-20T21:30:00.000Z", + "error_message": "Controller failover test loses committed jobs: ACKed jobs (jobsubmit ok, visible in joblist) disappear after kill/restart cycles. Predates libxce-failover (long-standing, unfixed).", + "file": "src/controller/persist.c / config_repl.c (CtrlHello catch-up)", + "root_cause": "REAL data loss (not commit-index). Decoded WAL bytes at /tmp/xce-ha-deploy: committed prefix entries (LSN 2-12, the lost job submits) are PHYSICALLY ABSENT from the live nodes' segments (ctrl-2 WAL starts at LSN 16, ctrl-3 at 13) and exist only on the down node (ctrl-1, LSN 1-15). snap/ dirs empty. Mechanism: a node (re)joining with an empty WAL is brought up to the cluster's CURRENT committed watermark via WAL streaming [watermark..committed], NOT from LSN 1, and no snapshot backfills [1..watermark-1] -> permanent prefix gap. When the only full-log node is down while others rejoined high, committed jobs are gone. replay_wal (persist.c:1422) is correct (read-only recovery); the gap is in catch-up start-point + missing entity-bearing snapshots.", + "fix": "NOT YET FIXED. Full findings: .kiro/specs/ha-commit-durability/INVESTIGATION-failover-job-loss.md. Open forks: (1) test clean-vs-preserve data on restart; (2) CtrlHello advertise real applied_lsn vs learned committed (config_repl.c handle_ctrlhello); (3) is snapshot creation implemented or a stub (XCESNAP header carries only an lsn, no entities). Likely architectural: catch-up assumes a snapshot safety net that does not exist. Secondary: WAL hdr.term always 0 (p->term never set) -> no term-based divergence detection.", + "tags": [ + "data-loss", + "failover", + "committed-job-loss", + "wal", + "catch-up", + "snapshot", + "ha-commit-durability", + "bug-795", + "architecture", + "NOT-FIXED" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T21:30:00.000Z" + }, + { + "id": "bug-839", + "timestamp": "2026-06-20T13:51:29.352Z", + "error_message": "Missing error handling in unknown", + "file": "../../../../../claude/.claude/projects/-home-rshen-projects-xce-xce/memory/MEMORY.md", + "root_cause": "Code path had no error handling — exceptions would propagate uncaught", + "fix": "Added try/catch block", + "tags": [ + "auto-detected", + "error-handling", + "md" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-06-20T13:51:29.352Z" } ] } \ No newline at end of file