bug-1562: ha_on_primary_ctrl_fd_rdhup's 5.10b debounce branch (ha.c) returned
without re-arming the liveness watchdog when a debounced RDHUP landed after
the in-flight PreVote round had already concluded (e.g. a lone cold-booting
node whose first round loses in milliseconds with no peers up yet). That was
the last timer ever scheduled -- the node froze permanently in CTRL_STANDBY,
so a 3-node cluster with a long inter-node start stagger could never elect a
primary. Root-caused via a live local reproduction matching the reporting
user's exact timing (byte-for-byte match of their hung ctrl-1.log), fixed by
re-arming the watchdog in that branch (mirroring the function's own 5.2
branch), and confirmed end-to-end: the same scenario now reaches CTRL_PRIMARY
and full 3-node config convergence.
bug-1563: fsync_worker's retry loop (persist.c) only backs off when no work
is pending, but flusher_work_pending_locked() can't distinguish "fresh work
just landed" from "still stuck from the last failed fdatasync" -- a
sustained sync failure left durable_lsn permanently behind tip, so the loop
skipped its wait every iteration and spun fdatasync as fast as the CPU/lock
allowed (a 40-round chaotic /xce-test-ha soak hit 300k+ failed attempts on
one node in under a minute). Fixed by waiting out the normal wal_sync_ms
cadence when a flush was pending but made no progress, leaving the
legitimate idle and just-succeeded fast paths untouched.
Both fixed via TDD (RED test before the fix, GREEN after) and live-validated
by re-running 40-round sequential and chaotic HA soaks end-to-end: both pass
cleanly, and the chaotic run's excluded fdatasync-error count dropped from
304,689 to 0.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Rongsong Shen
846713ab92
