feat(raft-apply-decoupling): ordered live apply pump activates apply_committed_entry (task 6.1)

Build persist_apply_pump_run()/persist_apply_pump_kick() -- the
ordered walk from last_applied_lsn+1 through committed_lsn, applying
every entry via apply_committed_entry() regardless of commit mode.
This is the first live (non-replay) consumer of that dispatch
function. Kick is wired into commit_resolve_pending() (the single
choke point all 6 internal committed_lsn-advancing paths funnel
through, discovered in task 1.2), dispatched via
vtask_enq_classified(VTASK_LIVE_BLOCK) with a CAS-coalesced
fire-and-forget trigger.

Fixes the two hazards flagged since task 4.1 as required before
going live:

1. Double-append: a thread-local call-provenance guard
   (t_apply_seam_notifying) prevents persist_job_hook_fun/
   persist_queue_hook_fun/persist_worker_hook_fun from re-appending
   entries the apply pump itself just applied, while preserving
   job_ctrl's legitimate live command-path appends (a naive
   detail-based filter would have broken SIGCONT/SIGSTOP). Also
   closes an undocumented instance of the same hazard in the
   queue/worker hook functions.

2. Merge-staleness: record_job_fail_fact/postdone's fact recording
   previously baked a receipt-time-computed final job state into the
   WAL entry, which goes stale under deferred apply -- a FINISH
   followed shortly by an ordinary post-exec FAIL could silently
   clobber a correct DONE completion with a stale RUN|EXIT|PERR, no
   guard catching it. Fixed by repurposing wal_jobstate_t's unused
   _pad field to carry just enough data (FAIL's stage) to recompute
   against the job's actual current state at apply time via the real
   apply_job_fail_transition()/new apply_job_postdone_transition()
   guard cores, instead of overwriting with a stale absolute value.
   Backward compatible: _pad==0 (all prior data) routes to the same
   legacy overwrite behavior.

Also fixes bug-1409 (unbounded busy-loop in apply_pump_drain when
committed_lsn advances without a corresponding on-disk entry) and a
genuine test-flakiness hazard (a second, independent apply-pump
kicker from the fsync_worker thread that existing tests didn't
account for).

Two pre-existing tests in test_commit_barrier_reply.c now fail as
expected -- their own assertions state "no job may be constructed
until task 6's apply pump exists," now correctly invalidated by this
task landing; updating them is task 7.3's explicit job.
This commit is contained in:
Rongsong Shen 2026-07-04 19:00:43 +00:00
parent d3da3ca8bd
commit 2fa955b92e
11 changed files with 1687 additions and 135 deletions

View file

@ -136,6 +136,17 @@ extern job_transition_result_t apply_job_fail_transition(job_t*,
job_failinfo_t*);
extern job_transition_result_t apply_job_update_transition(job_t*, int);
/* Guarded state-transition core for a postdone fact (raft-apply-decoupling
* task 6.1 round 2, Req 3.3/7.1-7.3), extracted from on_job_postdone's old
* inline mutation. Unlike a generic UPDATE, postdone's formula (conditional
* EXIT backstop, then OR in JOB_STAT_PDONE) is a pure function of the job's
* CURRENT state — so, like the other cores above, it is meant to be invoked
* at apply time against whatever state actually holds then, never against a
* receipt-time snapshot. No-op (JOB_TRANSITION_NOOP_DUPLICATE) when PDONE is
* already set and no backstop is needed. Caller must hold `job` under
* RTTI_LOCK_WRITE. */
extern job_transition_result_t apply_job_postdone_transition(job_t*);
extern int on_job_start(cluster_t*, job_pidinfo_t*);
extern int on_job_finish(cluster_t*, xce_jobfinish_t*);
extern int on_job_fail(cluster_t*, job_failinfo_t*);