---
id: "bug-1128"
status: "fixed"
status_updated: "2026-09-24"
status_note: ""
component: "controller"
severity: "unknown"
timestamp: "2026-06-25T03:20:00.000Z"
last_seen: "2026-09-19T09:00:00Z"
occurrences: 2
files: ["src/controller/testsuite/test_demotion_convergence.c"]
tags: ["test-flake", "demotion-convergence", "persist", "overlayfs", "ha-convergence-recovery", "quiescence", "wal-truncate", "sandbox", "commit-engine-race", "deterministic", "c"]
related: ["bug-343", "bug-704", "bug-795", "bug-828", "bug-9799", "bug-1060"]
legacy: true
---

# bug-1128: test_demotion_replay_yields_committed_only flakes ~16% on overlayfs: boot recovery over surviving…

## Description
test_demotion_replay_yields_committed_only flakes ~16% on overlayfs: boot recovery over surviving WAL must yield the committed prefix ONLY

## Environment
_Not recorded (migrated from buglog.json)._

## Reproduction
_Not recorded (migrated from buglog.json)._

## Root cause
TWO compounding non-determinism sources, both fixture artifacts not product bugs: (1) overlayfs metadata incoherence — a freshly-opened persist_t fd over a just-ftruncate'd WAL intermittently reads the STALE pre-truncate size, so the cross-context fresh recovery replays the already-removed suffix; (2) DEEPER, fs-independent: the fixture wires a LIVE commit engine and on a single node (n_peers=1) the persist durable-advance callback (fsync worker -> on_durable_advance) commits the whole durable log, so persist_flush asynchronously raced committed_lsn UP from the test-set committed_boundary (5) to the appended tip (8) AFTER the test pinned it; the demotion seam reads committed_lsn at task time -> truncated to 8 not 5 -> current_lsn=9 > committed_boundary+1. DIAG proof: committed_boundary=5 live_committed=8 live_tip=9 on failing runs.

## Fix
Test-only, two parts. (a) Detach the durable-advance commit wake (g_dc.persist.on_durable_advance = NULL) and re-pin committed_lsn = committed_boundary right before driving the demotion task, killing the commit-engine race deterministically (helps BOTH fs paths). (b) Hybrid gate by fs type via static dc_path_is_overlayfs() (statfs f_type == OVERLAYFS_SUPER_MAGIC 0x794c7630, defaults to real-fs path on statfs error): on a real fs keep the strong cross-context fresh boot-recovery assertion (fresh->current_lsn <= committed_boundary+1); on overlayfs skip that fd-incoherent step and assert the LIVE-context truncate effect (g_dc.persist.current_lsn <= committed_boundary+1, coherent under p->lock). Both branches prove the same property. Production persist.c truncate (ftruncate+fdatasync) untouched.

## Verification
Full controller suite 20/20 runs: 0 demotion_replay failures, 0 total failures. Group-only 60/60: 0 failures. Baseline (pre-fix) was 4/20. /tmp confirmed overlayfs (statfs magic 0x794c7630), so the overlayfs branch is the one exercised here.

## References
- Related: bug-343, bug-704, bug-795, bug-828, bug-9799, bug-1060