---
id: "bug-1431"
status: "fixed"
status_updated: "2026-09-24"
status_note: ""
component: "controller"
severity: "data-loss"
timestamp: "2026-07-10T10:02:00Z"
last_seen: "2026-09-19T09:00:00Z"
occurrences: 3
files: ["src/controller/ha.c", "src/controller/persist.c", "src/include/persist.h"]
tags: ["raft", "promotion", "committed_lsn", "chaotic-soak", "apply-decoupling", "durability"]
related: ["bug-1432"]
legacy: true
---

# bug-1431: chaotic HA soak: committed_job_loss=5-8 across 10 rounds (VERDICT result=FAIL) after…

## Description
chaotic HA soak: committed_job_loss=5-8 across 10 rounds (VERDICT result=FAIL) after raft-apply-decoupling task 16 gated standby apply strictly on commit; committed_divergence stayed 0 and all jobs eventually reappeared at final settlement -- a visibility-latency gap, not true data loss

## Environment
_Not recorded (migrated from buglog.json)._

## Reproduction
_Not recorded (migrated from buglog.json)._

## Root cause
Two stacked gaps in raft_cb_become_leader promotion, both masked pre-task-16 by eager apply-on-WAL-receipt: (1) persist_truncate_wal_above only sets p->committed_lsn on branches of truncate_wal_above_locked that perform a REAL truncation; its common no-op early-return (leader own log already covers safe_lsn) left committed_lsn stuck at its pre-election value, so newly-computed safe_committed_lsn was never applied to the actual commit watermark. (2) Even after fixing (1) with a durable_lsn-capped store, a newly-promoted leader whose own fsync had not yet caught up to its own WAL tip (group-commit batches fsync on a timer) stayed capped below safe_lsn indefinitely -- nothing re-drove the promotion advance once the flusher eventually synced, since fsync_worker on_durable_advance only re-evaluates the commit engines PENDING per-entry continuations, and a standby has none for replicated entries.

## Fix
New function persist_promotion_advance_committed(p, safe_lsn) (persist.c, declared in persist.h) called from raft_cb_become_leader right after persist_truncate_wal_above: unconditionally advances p->committed_lsn toward safe_lsn (monotonic), forcing a synchronous flusher_flush_once(p) first if durable_lsn < safe_lsn (safe/bounded -- by Raft Leader Completeness the new leaders own WAL tip is always >= safe_lsn, so the bytes are already on disk and only need fdatasync). Reduced but did not fully close the soak failures alone -- see bug-1432 for the second half of the fix.

## Verification
_Not recorded (migrated from buglog.json)._

## References
- Related: bug-1432