lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(raft-apply-decoupling): fire command continuation post-apply (task 6.2)

Browse source 
Wire persist_apply_pump_run() to call persist_take_pending_apply()
immediately after each entry's apply_committed_entry() call,
invoking the registered continuation strictly after the mutation.
Adds jobsubmit_reply_success_once() (job.c) so a successful job
submission finally sends a client reply once every array element
applies -- closes the standing gap from task 3.2 where committed
submissions never got a response.

Fixes bug-1415: commit_resolve_pending() dispatches the commit
continuation and the apply-pump kick as two independent
VTASK_LIVE_BLOCK vtasks with no completion-order guarantee. On any
real multi-vcpu pool (the production default, n_vcpus=8), if the
apply pump won the race and advanced last_applied_lsn past an LSN
before its continuation registered, the registration was permanently
orphaned -- the job would be constructed and visible, but the client
would hang forever. Untested by every existing fixture in this spec
(all use pool=NULL, which trivially serializes and can't reproduce
the race).

Fixed with a self-healing check in persist_register_pending_apply():
if the LSN is already <= last_applied_lsn, invoke the callback
immediately instead of registering for later pickup. Verified by the
spec's first genuine multi-vcpu regression test (real vtaskmgr_t,
n_vcpus=4), independently RED->GREEN reproduced by two review rounds.
This commit is contained in:
Rongsong Shen 2026-07-05 00:38:02 +00:00
parent 2fa955b92e
commit 88577f4658
6 changed files with 719 additions and 25 deletions
Download patch file Download diff file Expand all files Collapse all files

23
src/include/persist.h
View file

@ -939,6 +939,29 @@ int persist_submit_job_payloads_async(persist_t* p, xlist_t* payloads,
* during quorum loss (Req 6.1), not a leak — task 6.1 is expected to drain
* and free each node as it walks committed LSNs.
*
* bug-1415 self-healing (round 2): the caller (e.g. jobsubmit_commit_
* continuation, dispatched off-thread via commit.c's dispatch_continuation)
* and the ordered apply pump's walk (persist_apply_pump_run(), dispatched
* off-thread via persist_apply_pump_kick()) are two INDEPENDENT vtasks
* enqueued back-to-back onto the same pool with no completion-order
* guarantee between them — with n_vcpus > 1 (the production default), the
* walk can apply `lsn` and advance last_applied_lsn PAST it before this
* function ever runs for that same `lsn`. A registration arriving that late
* would otherwise be permanently orphaned: the walk visits each lsn exactly
* once, so nothing would ever call persist_take_pending_apply() for it
* again, silently hanging the caller forever. This function guards against
* that by checking `lsn <= persist_last_applied_lsn(p)` BEFORE linking the
* node onto the list; if the walk has already passed `lsn`, `on_applied` is
* invoked immediately and synchronously, right here, on the CALLER's thread
* (never the apply-pump thread) instead of being linked for a walk pass
* that will never revisit it. This is race-safe in both interleavings:
* when the walk has NOT reached `lsn` yet, the check is false and
* registration proceeds exactly as before (unchanged behavior for the
* common, non-racing case); when the walk already passed `lsn`, the walk's
* own persist_take_pending_apply() call for that lsn already ran and found
* nothing (that is precisely how we know this registration is late), so
* invoking `on_applied` here can never double-fire alongside it.
*
* NOTE for whoever builds task 6.1: apply_committed_entry()'s WAL_JOB_SUBMIT
* case (persist.c) already constructs the job (via restore_job()) AND fires
* the HOOK_DETAIL_JOB_NEW AFTER_EVENT notification, per entry, independently