feat(raft-apply-decoupling): ordered live apply pump activates apply_committed_entry (task 6.1)

Build persist_apply_pump_run()/persist_apply_pump_kick() -- the
ordered walk from last_applied_lsn+1 through committed_lsn, applying
every entry via apply_committed_entry() regardless of commit mode.
This is the first live (non-replay) consumer of that dispatch
function. Kick is wired into commit_resolve_pending() (the single
choke point all 6 internal committed_lsn-advancing paths funnel
through, discovered in task 1.2), dispatched via
vtask_enq_classified(VTASK_LIVE_BLOCK) with a CAS-coalesced
fire-and-forget trigger.

Fixes the two hazards flagged since task 4.1 as required before
going live:

1. Double-append: a thread-local call-provenance guard
   (t_apply_seam_notifying) prevents persist_job_hook_fun/
   persist_queue_hook_fun/persist_worker_hook_fun from re-appending
   entries the apply pump itself just applied, while preserving
   job_ctrl's legitimate live command-path appends (a naive
   detail-based filter would have broken SIGCONT/SIGSTOP). Also
   closes an undocumented instance of the same hazard in the
   queue/worker hook functions.

2. Merge-staleness: record_job_fail_fact/postdone's fact recording
   previously baked a receipt-time-computed final job state into the
   WAL entry, which goes stale under deferred apply -- a FINISH
   followed shortly by an ordinary post-exec FAIL could silently
   clobber a correct DONE completion with a stale RUN|EXIT|PERR, no
   guard catching it. Fixed by repurposing wal_jobstate_t's unused
   _pad field to carry just enough data (FAIL's stage) to recompute
   against the job's actual current state at apply time via the real
   apply_job_fail_transition()/new apply_job_postdone_transition()
   guard cores, instead of overwriting with a stale absolute value.
   Backward compatible: _pad==0 (all prior data) routes to the same
   legacy overwrite behavior.

Also fixes bug-1409 (unbounded busy-loop in apply_pump_drain when
committed_lsn advances without a corresponding on-disk entry) and a
genuine test-flakiness hazard (a second, independent apply-pump
kicker from the fsync_worker thread that existing tests didn't
account for).

Two pre-existing tests in test_commit_barrier_reply.c now fail as
expected -- their own assertions state "no job may be constructed
until task 6's apply pump exists," now correctly invalidated by this
task landing; updating them is task 7.3's explicit job.
This commit is contained in:
Rongsong Shen 2026-07-04 19:00:43 +00:00
parent d3da3ca8bd
commit 2fa955b92e
11 changed files with 1687 additions and 135 deletions

View file

@ -815,6 +815,23 @@ static void commit_resolve_pending(commit_engine_t* ce)
for (int i = 0; i < n; i++) {
commit_resolve(ce, ready[i], COMMIT_OUTCOME_COMMITTED, 0);
}
/* raft-apply-decoupling task 6.1: this function is the single choke
* point every one of the six committed_lsn-advancing paths already
* funnels through (durable-advance, self-quorum commit_submit, ack-
* driven advance, the writable-lsn gate, the raft-library commit-index
* feed via commit_set_raft_commit_index, and membership change) --
* kicking the ordered apply pump here, rather than only from ha.c's
* on_commit_advanced callback as design.md's wiring text names, is a
* deliberate deviation: on_commit_advanced is fed by exactly ONE of
* these six paths, so wiring the pump there alone would silently starve
* it for any cluster (e.g. single-node/self-quorum) that never
* exercises that specific raft-library callback. Called unconditionally
* -- persist_apply_pump_kick is itself a documented no-op whenever
* ce->p->cluster is NULL, so this has zero effect on any commit-engine
* unit fixture that never wires a cluster_t (the overwhelming majority
* of this file's own test suite). */
persist_apply_pump_kick(ce->p, ce->pool);
}
/* raft-apply-decoupling task 1.4: the FAILED-outcome sibling of