docs(ha-term-safety): record bug-828 FIXED (premature pre-election truncate) + bug-854 residual

.kiro/specs/ha-term-safety/tasks.md

@@ -130,6 +130,8 @@
   - _NEW EXPOSED — bug-828/795 (pre-existing, DOCUMENTED, unfixed; NOT ha-term-safety; NOT introduced by this work): the live STRICT failover now FAILS at walcompare because the FAILOVER LOSES COMMITTED JOBS. Sequence: committed=5 (valid, durable majority), kill primary, a standby promotes at term 3 computing `safe_committed_lsn=1 (from 1 granted voter)` and TRUNCATES the committed jobs (LSN 2-5) → ctrl-2/ctrl-3 end at LSN 1 + term-3 NOOP while ctrl-1 (restarted) holds the committed jobs. This is a Leader-Completeness violation in the ELECTION/safe-commit-from-votes path (`ha_compute_safe_committed_lsn`), which this spec does NOT touch. It was INVISIBLE before because the commit-stall cascade (bug-990/991/2.4-ceiling) kept committed pinned at 1, so the failover never had committed jobs (LSN>1) to lose. Matches the project-memory "Failover job-loss root cause (bug-828/795, root-caused 2026-06-20, NOT fixed)". Evidence: /home/claude/tmp/xce-ha-deploy/final (kept). OWNER: ha-commit-durability / a failover-safety spec, NOT ha-term-safety. 5.2/5.4 cannot be GREEN until bug-828 is fixed._
   - _bug-828 PARTIAL FIX landed (commit 91b9a44, reviewer-verified SAFE, 177 unit tests): (part 1) `vote_reply_dispatch` was discarding the granting voter's decoded peerLastLogLsn (`(void)peer_lsn`), so `ha_compute_safe_committed_lsn` saw only the self-vote → truncated to self_lsn (the "from 1 granted voters" fingerprint). Now routes it into `ctx->votes[]` via a record-only `ha_record_vote_reply` (no double-promotion; raft_on_vote_reply stays the sole promotion driver). (part 2) candidate/voter advertise the real DURABLE (fsync'd) tip; `persist_wal_write_replicated` advances `durable_lsn` after its fail-loud inline fdatasync. Safe-commit is bounded by the durable majority (preserves more committed entries, never marks a non-majority-durable LSN safe). Fingerprint moved "from 1" → "from 2 granted voters"._
   - _REMAINING SLICE (precisely characterized, NOT yet fixed — the documented architectural core): the live clean-failover STILL loses committed jobs because the ELECTION WATERMARK TANGLE persists. A standby that caught up to applied_lsn/durable_lsn=5 during the run still resolves its OWN election position to the STALE BOOT `durable_floor=1` in the vote/PreVote freshness gate + advertise, so (a) it fails to out-rank a job-LESS candidate (freshness gate: job-less candidate's lastLogIndex=1 >= the job-having voter's mis-computed 1 → grants), and (b) a job-less node wins term 3, computes safe_committed=1, and `persist_truncate_wal_above(1)` destroys committed LSN 2-5. FIX DIRECTION: source the voter/candidate election position from the LIVE durable/applied tip (max(persist_durable_lsn/applied_lsn, last_known_committed_lsn)) NOT the boot durable_floor, consistently across ALL FOUR election paths — candidate self-advertise (`ha_candidate_advertise_lsn`), voter floor (cluster.c ~403/495 + what the controller passes as the voter's own last_log_index into `raft_on_vote_request`), the freshness comparison, and safe-commit. SAFETY GUARD: advertise/compare only DURABLE (fsync'd) positions — never a merely-received tip — or a node could win/keep an entry it isn't durable for. This is safety-critical consensus code (a wrong fix = data loss); needs a dedicated, careful, fresh-context effort. Owner: ha-commit-durability / failover-safety. Live evidence: /home/claude/tmp/xce-ha-deploy/{final,fix828,fix828b_r1} (kept)._
+  - _**bug-828 FIXED (commit b77d412, reviewer-verified SAFE).** The earlier "watermark tangle" framing was WRONG — instrumentation proved the standbys DID fsync LSN 2-5 (durable_lsn=5; not a durability violation, not an advertisement-staleness bug). ACTUAL ROOT CAUSE: `ha_enter_real_election` (ha.c ~5187) did `persist_truncate_wal_above(&c->persist, ha->last_known_committed_lsn)` BEFORE the vote — a defensive heuristic copied from the RDHUP path that VIOLATES the documented rule "Never use last_known_committed_lsn for truncation — it may be stale". last_known_committed_lsn lags the true commit point (primary committed 2-5 on a durable majority then died before a heartbeat carried committed=5), so the cutoff was 1; the truncate's clamp_durable_lsn lowered durable_lsn 5→1, PHYSICALLY DESTROYING committed entries before the election computed the safe point → candidate advertised 1 → safe_committed=1 → post-election truncate kept only LSN 1. FIX: delete the premature truncate. Only truncates now are the authoritative votes-derived post-election persist_truncate_wal_above(safe_committed_lsn) + the library on_truncate_from (term,lsn) reconcile — both before any commit under the new leader. SAFE because a divergent uncommitted tail can only exist at a stale/lower term (single-writer), and the library's TERM-PRIMARY (lastLogTerm,lastLogIndex) freshness gate denies such a candidate — correct ONLY because task 3.4 fixed the term axis (the whole ha-term-safety cascade was the prerequisite that makes this fix safe). 177 unit tests + RED-proven regression test (`test_repl_catchup_durable_floor_survives_failover`). **LIVE: clean failover preserves committed jobs — walcompare committed-prefix IDENTICAL across all 3 members, both jobs on disk, across multiple runs.**_
+  - _RESIDUAL (separate, pre-existing, NOT data-safety): the live harness still EXIT-1s at the POST-walcompare client `wait-primary`/`expect-job` step due to **bug-854** (libxce positional primary-discovery / dead-first-master: the client can't reach the live primary when the killed node is FIRST in the masters list — `ctrl-1,ctrl-2,ctrl-3` with ctrl-1 dead). This is a CLIENT-side discovery defect (libxce), orthogonal to the server-side committed-prefix safety which now PASSES. To get the harness fully GREEN: fix bug-854 (libxce client failover discovery) OR have the harness target a known-live node post-failover. 5.2's DATA-SAFETY assertion (walcompare identical + jobs on disk) now passes; the EXIT-0 is gated on bug-854._
 
 - [x] 5.3 Cross-member `walcompare` divergence tool
   - Build a tool that diffs member WALs by `(lsn → term, pcrc)` and exits non-zero on any committed-prefix divergence, reporting the diverging LSN and per-member entry identities; reuse the existing WAL decoder