---
id: "bug-1023"
status: "fixed"
status_updated: "2026-09-24"
status_note: ""
component: "controller"
severity: "unknown"
timestamp: "2026-06-23T03:45:00Z"
last_seen: "2026-09-19T09:00:00Z"
occurrences: 1
files: ["src/controller/ha.c"]
tags: ["ha", "raft", "failover", "leader-completeness", "bug-828", "bug-795", "durability", "truncate", "election", "durable_lsn"]
related: ["bug-661", "bug-795", "bug-828", "bug-851", "bug-854", "bug-339"]
legacy: true
---

# bug-1023: Clean single failover (STRICT_TERM) loses committed jobs: safe_committed_lsn=1 from votes despite…

## Description
Clean single failover (STRICT_TERM) loses committed jobs: safe_committed_lsn=1 from votes despite a standby being genuinely durable at LSN 5; walcompare reports committed-prefix divergence at LSN 2 (jobs gone on the new leader).

## Environment
_Not recorded (migrated from buglog.json)._

## Reproduction
_Not recorded (migrated from buglog.json)._

## Root cause
ha_enter_real_election() PRE-TRUNCATED the candidate's WAL above ha->last_known_committed_lsn BEFORE the vote. last_known_committed_lsn is the LOCALLY-LEARNED commit watermark, which LAGS the true cluster commit point (the primary commits an entry on a durable majority, then dies before a heartbeat carries that committed LSN to the standby). persist_truncate_wal_above clamps durable_lsn down to the cutoff (clamp_durable_lsn_locked). So a standby durably holding committed entries [committed+1..tip] discarded them, then advertised the stale-low cutoff as its durable floor (ha_candidate_advertise_lsn / ha_vote_self_freshness both read persist_durable_floor). Every voter doing the same collapsed safe-committed-from-votes to the stale watermark; the AUTHORITATIVE post-election truncate (raft_cb_become_leader -> persist_truncate_wal_above(safe_committed_lsn)) then kept only that stale prefix, losing committed jobs. Live trace (DBG828) PROVED the standbys wrote+fdatasync'd LSN 2-5 (durable_lsn=5) and only LOST them at the pre-election truncate clamp — NOT a durability/ack-before-fsync violation. Final bug-828/795 slice.

## Fix
Removed the pre-election persist_truncate_wal_above(last_known_committed_lsn) in ha_enter_real_election (src/controller/ha.c). Raft model: a candidate does NOT truncate its log before campaigning — it keeps its full durable log and advertises its real durable tip so voters holding more committed entries deny a colder candidate and the winner has every committed entry (Leader Completeness). Divergent UNCOMMITTED tails are removed only by the authoritative post-election safe-commit truncate (above the majority-held point) and by the library's per-entry (term,lsn) reconcile (on_truncate_from) during AppendEntries. RED: live STRICT_TERM harness safe=1 + walcompare divergence at LSN 2. GREEN: every run now logs safe_committed_lsn=5 (from 2 granted voters) and walcompare 'committed prefixes identical'. Unit: new test_repl_catchup_durable_floor_survives_failover (test_repl_apply.c) + 177/177 unit GREEN. NOTE: live EXIT still 1 due to a PRE-EXISTING libxce client-failover bug (bug: 'client op fails to reach live primary when a DEAD controller is FIRST in the masters list') surfacing now that the durability bug no longer masks the post-walcompare expect-job/wait-primary step — committed jobs are physically present on disk (walcompare proves it).

## Verification
_Not recorded (migrated from buglog.json)._

## References
- Related: bug-661, bug-795, bug-828, bug-851, bug-854, bug-339