lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(raft-apply-decoupling): ordered live apply pump activates apply_committed_entry (task 6.1)

Browse source 
Build persist_apply_pump_run()/persist_apply_pump_kick() -- the
ordered walk from last_applied_lsn+1 through committed_lsn, applying
every entry via apply_committed_entry() regardless of commit mode.
This is the first live (non-replay) consumer of that dispatch
function. Kick is wired into commit_resolve_pending() (the single
choke point all 6 internal committed_lsn-advancing paths funnel
through, discovered in task 1.2), dispatched via
vtask_enq_classified(VTASK_LIVE_BLOCK) with a CAS-coalesced
fire-and-forget trigger.

Fixes the two hazards flagged since task 4.1 as required before
going live:

1. Double-append: a thread-local call-provenance guard
   (t_apply_seam_notifying) prevents persist_job_hook_fun/
   persist_queue_hook_fun/persist_worker_hook_fun from re-appending
   entries the apply pump itself just applied, while preserving
   job_ctrl's legitimate live command-path appends (a naive
   detail-based filter would have broken SIGCONT/SIGSTOP). Also
   closes an undocumented instance of the same hazard in the
   queue/worker hook functions.

2. Merge-staleness: record_job_fail_fact/postdone's fact recording
   previously baked a receipt-time-computed final job state into the
   WAL entry, which goes stale under deferred apply -- a FINISH
   followed shortly by an ordinary post-exec FAIL could silently
   clobber a correct DONE completion with a stale RUN|EXIT|PERR, no
   guard catching it. Fixed by repurposing wal_jobstate_t's unused
   _pad field to carry just enough data (FAIL's stage) to recompute
   against the job's actual current state at apply time via the real
   apply_job_fail_transition()/new apply_job_postdone_transition()
   guard cores, instead of overwriting with a stale absolute value.
   Backward compatible: _pad==0 (all prior data) routes to the same
   legacy overwrite behavior.

Also fixes bug-1409 (unbounded busy-loop in apply_pump_drain when
committed_lsn advances without a corresponding on-disk entry) and a
genuine test-flakiness hazard (a second, independent apply-pump
kicker from the fsync_worker thread that existing tests didn't
account for).

Two pre-existing tests in test_commit_barrier_reply.c now fail as
expected -- their own assertions state "no job may be constructed
until task 6's apply pump exists," now correctly invalidated by this
task landing; updating them is task 7.3's explicit job.
This commit is contained in:
Rongsong Shen 2026-07-04 19:00:43 +00:00
parent d3da3ca8bd
commit 2fa955b92e
11 changed files with 1687 additions and 135 deletions
Download patch file Download diff file Expand all files Collapse all files

565
src/controller/persist.c
View file

@ -643,6 +643,11 @@ int persist_init(persist_t* p, persist_config_t* cfg, cluster_t* c)
p->pending_applies = NULL;
pthread_mutex_init(&p->pending_applies_lock, NULL);
/* Ordered apply pump starts idle (raft-apply-decoupling task 6.1): no
* walk scheduled or running yet. Freed/destroyed in persist_fini. */
pthread_mutex_init(&p->apply_pump_lock, NULL);
atomic_store(&p->apply_pump_scheduled, 0);
/* Create WAL and snapshot directories */
if (mkdir_p(p->wal_path, 0700) != 0) return -errno;
if (mkdir_p(p->snap_path, 0700) != 0) return -errno;
@ -766,6 +771,12 @@ void persist_fini(persist_t* p)
}
p->pending_applies = NULL;
pthread_mutex_destroy(&p->pending_applies_lock);
/* raft-apply-decoupling task 6.1: no walk can still be in flight by the
* time persist_fini runs (the caller owns quiescing any vtask pool
* first, same precondition every other persist_t resource here already
* assumes). */
pthread_mutex_destroy(&p->apply_pump_lock);
}
/* Forward declaration — defined after checkpoint internals */
@ -2625,6 +2636,37 @@ int apply_wal_entry(persist_t* p, uint32_t type, uint64_t lsn,
* (routing table, the job-state-family hook-mapping deviation, and what this
* function deliberately does NOT own). Not yet wired into any live call
* site — replay (2.3) and the live apply pump (6.1) will call this later. */
/* raft-apply-decoupling task 6.1 (Hazard 1 fix): thread-local "this call
* stack is inside apply_committed_entry's own post-mutation notification"
* guard. persist_job_hook_fun / persist_queue_hook_fun / persist_worker_hook_fun
* are AFTER_EVENT subscribers that (for RETAIN/fact-shaped entry types)
* durably append a NEW WAL entry every time they fire — the correct behavior
* for their ORIGINAL, still-live purpose (job_ctrl's mark_job_update/
* mark_job_fail synchronous command path; the legacy queue/worker control
* command handlers) which mutate live state and rely on this exact hook
* firing to durably record that mutation.
*
* apply_committed_entry() now ALSO fires these same AFTER_EVENT hooks (job
* facts always as the generic HOOK_DETAIL_JOB_UPDATE, per task 2.2's
* documented wire-format deviation; queue/worker control at their native
* detail) to deliver its own post-mutation NOTIFICATION — but that entry is
* by definition already durably committed (it just got applied FROM the
* WAL), so re-appending it would silently duplicate the entry on every live
* apply. This is fixable, correctly, ONLY by provenance (which caller fired
* the hook), NOT by hook `detail` value: HOOK_DETAIL_JOB_UPDATE in
* particular is genuinely dual-origin (job_ctrl's SIGCONT/SIGSTOP path uses
* the identical detail for its own, still-legitimate live append) — an
* unconditional detail-based filter would silently break job_ctrl's suspend/
* resume durability instead of fixing anything. A shared persist_t-wide flag
* would ALSO be wrong: it would race across threads (e.g. job_ctrl mutating
* a DIFFERENT job on the iomgr thread while this thread's apply walk is
* mid-hook for an unrelated fact) and could wrongly suppress or wrongly
* permit an append depending on scheduling. A thread-local flag, scoped
* narrowly around each xce_run_hooks() call this function makes, is race-free
* by construction: it is only ever observed by hook subscribers invoked
* SYNCHRONOUSLY on this exact call stack, from this exact function. */
static _Thread_local bool t_apply_seam_notifying = false;
int apply_committed_entry(persist_t* p, uint32_t wal_type, uint64_t lsn,
const void* payload, uint32_t len)
{
@ -2666,25 +2708,30 @@ int apply_committed_entry(persist_t* p, uint32_t wal_type, uint64_t lsn,
void* jptr = j;
xlist_t jlist = { .count = 1, .caps = 1, .delta = 1, .data = &jptr };
t_apply_seam_notifying = true;
xce_run_hooks(&c->hooksys, XCE_CONTROLLER_HOOK_JOB, HOOK_AFTER_EVENT,
HOOK_DETAIL_JOB_NEW, c, &jlist);
t_apply_seam_notifying = false;
return 0;
}
case WAL_JOB_START:
case WAL_JOB_FINISH:
case WAL_JOB_FAIL:
case WAL_JOB_STOP:
case WAL_JOB_RESUME:
case WAL_JOB_UPDATE:
case WAL_JOB_RUSAGE:
case WAL_JOB_CKPT:
case WAL_JOB_PREASON: {
/* Every job-state fact type shares one wire shape — wal_jobstate_t
* {jobid, state} — and therefore one guarded core. See persist.h's
* doc comment on apply_committed_entry for why this does not
* dispatch to the type-specific apply_job_{start,finish,fail}_
* transition() cores (their richer parameter structs are not
/* Every job-state fact type in this group shares one wire shape —
* wal_jobstate_t {jobid, state} — and therefore one guarded core.
* Their payloads are deterministic/state-independent (a START is
* always JOB_STAT_RUN, a FINISH's outcome depends only on
* exit_code), so an absolute `state` baked at receipt time never
* goes stale — unlike WAL_JOB_FAIL/WAL_JOB_UPDATE below, which have
* their own dedicated cases. See persist.h's doc comment on
* apply_committed_entry for the full routing table and why this
* does not dispatch to the richer type-specific apply_job_{start,
* finish}_transition() cores (their parameter structs are not
* present on this wire format). */
if (len < sizeof(wal_jobstate_t))
return 0; /* undersized payload — skip, mirrors apply_wal_entry */
@ -2705,8 +2752,111 @@ int apply_committed_entry(persist_t* p, uint32_t wal_type, uint64_t lsn,
if (result != JOB_TRANSITION_APPLIED)
return 0; /* rejected-backward / no-op-duplicate — no hook */
/* Hazard 1 guard (see t_apply_seam_notifying's doc comment above):
* HOOK_DETAIL_JOB_UPDATE is dual-origin — job_ctrl's mark_job_update
* (SIGCONT/SIGSTOP) fires the identical detail for its own live,
* still-legitimate append via persist_job_hook_fun. Without this
* guard, EVERY fact this apply pump applies would re-append itself
* as a duplicate, mis-typed WAL_JOB_UPDATE entry the moment the live
* pump (task 6.1) starts calling this function. */
t_apply_seam_notifying = true;
xce_run_hooks(&c->hooksys, XCE_CONTROLLER_HOOK_JOB, HOOK_AFTER_EVENT,
HOOK_DETAIL_JOB_UPDATE, c, j);
t_apply_seam_notifying = false;
return 0;
}
case WAL_JOB_FAIL: {
/* task 6.1 round 2 (Req 3.3/7.1-7.3, 11.3) fix for a job-state
* corruption bug: unlike the deterministic group above, a FAIL's
* resulting state depends on the job's CURRENT state (PERR merges
* onto whatever terminal state already holds). Dispatch on _pad
* (see wal_types.h's doc comment on wal_jobstate_t._pad):
* - WAL_JOB_FAIL_STAGE_FLAG set: a deferred fail-fact
* (record_job_fail_fact, job.c) — reconstruct a minimal
* job_failinfo_t from {jobid, stage} and call the REAL
* apply_job_fail_transition() against the job's ACTUAL CURRENT
* state at apply time, never a receipt-time snapshot.
* - _pad == 0: the legacy live path (job_ctrl's mark_job_fail) —
* `state` is already the correct, current, absolute value,
* computed synchronously under the same lock as the mutation
* (no staleness window) — an absolute overwrite is safe and
* unchanged. */
if (len < sizeof(wal_jobstate_t))
return 0; /* undersized payload — skip, mirrors apply_wal_entry */
const wal_jobstate_t* js = (const wal_jobstate_t*)payload;
job_t* j = NULL;
lock(c->jobmgr, RTTI_LOCK_READ);
key_find(c->jobmgr, (void*)js->jobid, (void**)&j);
unlock(c->jobmgr);
if (!j)
return -ENOENT;
lock(j, RTTI_LOCK_WRITE);
job_transition_result_t result;
if (js->_pad & WAL_JOB_FAIL_STAGE_FLAG) {
job_failinfo_t failinfo = {
.jobid = js->jobid,
.stage = js->_pad & ~WAL_JOB_FAIL_STAGE_FLAG,
.code = 0,
.detail = NULL,
};
result = apply_job_fail_transition(j, &failinfo);
} else {
result = apply_job_update_transition(j, (int)js->state);
}
unlock(j);
if (result != JOB_TRANSITION_APPLIED)
return 0; /* rejected-backward / no-op-duplicate — no hook */
/* Hazard 1 guard — see the identical guard above. */
t_apply_seam_notifying = true;
xce_run_hooks(&c->hooksys, XCE_CONTROLLER_HOOK_JOB, HOOK_AFTER_EVENT,
HOOK_DETAIL_JOB_UPDATE, c, j);
t_apply_seam_notifying = false;
return 0;
}
case WAL_JOB_UPDATE: {
/* task 6.1 round 2 (Req 3.3/7.1-7.3, 11.3): same class of fix as
* WAL_JOB_FAIL above. Dispatch on _pad:
* - WAL_JOB_UPDATE_POSTDONE_MARK: a deferred postdone fact
* (record_job_postdone_fact, job.c) — call
* apply_job_postdone_transition(), which re-derives the
* postdone merge (conditional EXIT backstop + OR JOB_STAT_PDONE)
* fresh against the job's ACTUAL CURRENT state at apply time.
* - _pad == 0: the legacy live path (job_ctrl's mark_job_update,
* SIGCONT/SIGSTOP) — `state` is already correct/absolute,
* computed synchronously under the same lock as the mutation —
* an absolute overwrite is safe and unchanged. */
if (len < sizeof(wal_jobstate_t))
return 0; /* undersized payload — skip, mirrors apply_wal_entry */
const wal_jobstate_t* js = (const wal_jobstate_t*)payload;
job_t* j = NULL;
lock(c->jobmgr, RTTI_LOCK_READ);
key_find(c->jobmgr, (void*)js->jobid, (void**)&j);
unlock(c->jobmgr);
if (!j)
return -ENOENT;
lock(j, RTTI_LOCK_WRITE);
job_transition_result_t result =
(js->_pad == WAL_JOB_UPDATE_POSTDONE_MARK)
? apply_job_postdone_transition(j)
: apply_job_update_transition(j, (int)js->state);
unlock(j);
if (result != JOB_TRANSITION_APPLIED)
return 0; /* rejected-backward / no-op-duplicate — no hook */
/* Hazard 1 guard — see the identical guard above. */
t_apply_seam_notifying = true;
xce_run_hooks(&c->hooksys, XCE_CONTROLLER_HOOK_JOB, HOOK_AFTER_EVENT,
HOOK_DETAIL_JOB_UPDATE, c, j);
t_apply_seam_notifying = false;
return 0;
}
@ -2732,8 +2882,17 @@ int apply_committed_entry(persist_t* p, uint32_t wal_type, uint64_t lsn,
if (!q)
return 0;
/* Same double-append class as the job-fact guard above: the legacy
* queue-control command handler still mutates then relies on this
* exact AFTER_EVENT firing to durably append via
* persist_queue_hook_fun (WAL_QUEUE_CTRL is unconditionally
* appended on AFTER_EVENT there, not gated on WAL_MODE_WAIT like the
* job-submit case is) — guard so this live-pump re-notify does not
* re-append an already-committed entry. */
t_apply_seam_notifying = true;
xce_run_hooks(&c->hooksys, XCE_CONTROLLER_HOOK_QUEUE, HOOK_AFTER_EVENT,
HOOK_DETAIL_QUEUE_UPDATE, c, q);
t_apply_seam_notifying = false;
return 0;
}
@ -2772,6 +2931,13 @@ int apply_committed_entry(persist_t* p, uint32_t wal_type, uint64_t lsn,
if (!w)
return 0; /* defensive: should not happen after a successful restore */
/* Same double-append class as the job-fact/queue guards above:
* persist_worker_hook_fun unconditionally appends on AFTER_EVENT for
* every worker detail (no WAL_MODE_WAIT gate), and the legacy worker
* join/update/dead command paths still rely on that firing for their
* own live durability — guard the whole re-notify switch so this
* live-pump call never re-appends an already-committed entry. */
t_apply_seam_notifying = true;
switch (hook_detail) {
case HOOK_DETAIL_WORKER_JOIN:
xce_run_hooks(&c->hooksys, XCE_CONTROLLER_HOOK_WORKER,
@ -2790,6 +2956,7 @@ int apply_committed_entry(persist_t* p, uint32_t wal_type, uint64_t lsn,
break; /* unrecognized discriminator — mutation already applied,
* notification skipped rather than guessed at */
}
t_apply_seam_notifying = false;
return 0;
}
@ -2801,6 +2968,292 @@ int apply_committed_entry(persist_t* p, uint32_t wal_type, uint64_t lsn,
}
}
/* ============================================================
* Ordered apply pump (raft-apply-decoupling task 6.1)
* ============================================================ */
/* Locate and read back the WAL entry at exactly `lsn` from the on-disk
* segment files, for the live apply pump to hand to apply_committed_entry()
* (mirrors read_entry_meta_locked's segment-scan idiom above, but returns
* the entry's type and a malloc'd copy of its payload instead of just
* metadata, and re-verifies the entry CRC — this is live-path plumbing, not
* a metadata probe).
*
* Does not take p->lock: every LSN this is ever called with is already
* majority-committed (durable and replicated), strictly below the physical
* append tip a concurrent persist_wal_async could be extending — so this
* never races the single-writer append path for the bytes it actually
* reads.
*
* Returns 0 and populates *out_type/*out_payload (malloc'd if payload_len >
* 0, caller frees; NULL when payload_len is 0)/*out_len on success.
* Returns -ENOENT if no segment contains this LSN (should not happen for a
* genuinely committed entry) or a negative errno on a read/CRC failure. */
static int apply_pump_read_entry(persist_t* p, uint64_t lsn,
uint32_t* out_type, uint8_t** out_payload,
uint32_t* out_len)
{
if (!p || lsn == 0 || !out_type || !out_payload || !out_len)
return -EINVAL;
DIR* d = opendir(p->wal_path);
if (!d)
return -ENOENT;
uint64_t seg_lsns[4096];
int nseg = 0;
struct dirent* e;
while ((e = readdir(d)) != NULL &&
nseg < (int)(sizeof(seg_lsns) / sizeof(seg_lsns[0]))) {
if (strncmp(e->d_name, "seg-", 4) != 0)
continue;
uint64_t sl = 0;
if (sscanf(e->d_name, "seg-%" SCNx64 ".wal", &sl) != 1)
continue;
seg_lsns[nseg++] = sl;
}
closedir(d);
if (nseg == 0)
return -ENOENT;
/* Ascending sort by start LSN (segment count is tiny in practice). */
for (int i = 1; i < nseg; i++) {
uint64_t key = seg_lsns[i];
int j = i - 1;
while (j >= 0 && seg_lsns[j] > key) {
seg_lsns[j + 1] = seg_lsns[j];
j--;
}
seg_lsns[j + 1] = key;
}
int rc = -ENOENT;
for (int si = 0; si < nseg; si++) {
/* Segments are LSN-ordered; one whose start LSN already exceeds the
* target cannot contain it, nor can any later one. */
if (seg_lsns[si] > lsn)
break;
char seg_path[4096];
snprintf(seg_path, sizeof(seg_path), WAL_SEG_FMT, p->wal_path,
seg_lsns[si]);
int fd = open(seg_path, O_RDONLY);
if (fd < 0)
continue;
uint32_t off = 0;
for (;;) {
wal_entry_hdr_t hdr;
ssize_t hn = pread(fd, &hdr, sizeof(hdr), (off_t)off);
if (hn < (ssize_t)sizeof(hdr))
break; /* short read / EOF / pre-allocated zero tail */
if (hdr.lsn == 0 && hdr.type == 0 && hdr.payload_len == 0)
break;
if (hdr.lsn == lsn) {
uint8_t* payload = NULL;
uint32_t expected_crc =
crc32(0L, (const Bytef*)&hdr, WAL_HDR_CRC_SIZE);
if (hdr.payload_len > 0) {
payload = malloc(hdr.payload_len);
if (!payload) {
rc = -ENOMEM;
break;
}
ssize_t pn = pread(fd, payload, hdr.payload_len,
(off_t)(off + sizeof(hdr)));
if (pn != (ssize_t)hdr.payload_len) {
free(payload);
rc = -EIO;
break;
}
expected_crc = crc32(expected_crc, (const Bytef*)payload,
hdr.payload_len);
}
if (hdr.crc != expected_crc) {
free(payload);
rc = -EIO;
break;
}
*out_type = hdr.type;
*out_payload = payload;
*out_len = hdr.payload_len;
rc = 0;
break;
}
off += (uint32_t)(sizeof(hdr) + hdr.payload_len);
}
close(fd);
if (rc == 0 || rc == -ENOMEM || rc == -EIO)
break; /* found it, or hit a genuine error inside the segment
* that should contain it */
}
return rc;
}
int persist_apply_pump_run(persist_t* p)
{
if (!p)
return -EINVAL;
pthread_mutex_lock(&p->apply_pump_lock);
int applied = 0;
for (;;) {
uint64_t committed = atomic_load_explicit(&p->committed_lsn,
memory_order_acquire);
uint64_t next = atomic_load_explicit(&p->last_applied_lsn,
memory_order_relaxed) + 1;
if (next > committed)
break; /* caught up as of this instant */
uint32_t wal_type = 0;
uint8_t* payload = NULL;
uint32_t len = 0;
int rc = apply_pump_read_entry(p, next, &wal_type, &payload, &len);
if (rc < 0) {
ERROR("persist_apply_pump_run: failed to read committed entry "
"lsn=%" PRIu64 " (rc=%d) -- stopping this pass; the entry "
"stays committed and will be retried, from the same "
"position, on the next kick",
next, rc);
pthread_mutex_unlock(&p->apply_pump_lock);
return rc;
}
/* Strict order, no skipping or reordering: this is the single
* dispatch call every newly committed entry passes through here,
* regardless of its commit mode (WAIT command or RETAIN fact) — the
* loop only ever advances last_applied_lsn one entry at a time, in
* ascending order, so a burst of commands and facts that committed
* out of their original submission order still applies in strict
* commit (LSN) order (Req 3.3, 7.3). */
(void)apply_committed_entry(p, wal_type, next, payload, len);
free(payload);
atomic_store_explicit(&p->last_applied_lsn, next, memory_order_relaxed);
applied++;
}
pthread_mutex_unlock(&p->apply_pump_lock);
return applied;
}
/* Heap payload for one dispatched apply-pump walk invocation; frees itself
* after use, mirroring commit.c's commit_resolve_dispatch_t convention. */
typedef struct {
persist_t* p;
} apply_pump_dispatch_t;
/* Run walk passes until genuinely caught up, using the same recheck-then-
* clear idiom commit.c's sweep thread uses (task 1.3) to avoid a lost-wakeup
* race between "persist_apply_pump_run just decided it is caught up" and "a
* new commit lands right after". Runs on a pool worker thread (or inline as
* persist_apply_pump_kick's documented fallback) — never on the calling
* (potentially replication-ACK/iomgr) thread when dispatched normally.
*
* Does NOT immediately retry a failed pass: persist_apply_pump_run() already
* logs loudly and leaves last_applied_lsn unadvanced past a read failure, so
* an immediate self-retry here would busy-loop forever on the exact same
* unresolved head-of-line entry (a real, reproduced hazard — a persist_t
* whose committed_lsn is ever advanced without a corresponding real WAL
* entry on disk, e.g. a test fixture exercising commit-engine resolve logic
* in isolation, spun this into a multi-million-iteration tight loop before
* this guard existed). A future, genuinely NEW kick — triggered by a later
* real commit-advance event — retries then, at that event's natural cadence,
* never in a busy spin. */
static void apply_pump_drain(persist_t* p)
{
for (;;) {
int rc = persist_apply_pump_run(p);
atomic_store_explicit(&p->apply_pump_scheduled, 0, memory_order_release);
if (rc < 0)
return; /* already logged loudly inside persist_apply_pump_run;
* do not busy-retry the same unresolved entry */
uint64_t committed = atomic_load_explicit(&p->committed_lsn,
memory_order_acquire);
uint64_t next = atomic_load_explicit(&p->last_applied_lsn,
memory_order_relaxed) + 1;
if (next > committed)
return; /* genuinely caught up: no race window survived open */
int expected = 0;
if (!atomic_compare_exchange_strong_explicit(
&p->apply_pump_scheduled, &expected, 1, memory_order_acq_rel,
memory_order_relaxed))
return; /* a concurrent kick already reclaimed the flag; it will
* run its own pass and pick up this same new work */
/* else: we reclaimed the flag ourselves -- loop and run another pass */
}
}
static int apply_pump_vtask_entry(vtask_t* task)
{
apply_pump_dispatch_t* d = (apply_pump_dispatch_t*)task->data->u.generic;
apply_pump_drain(d->p);
free(d);
return 0;
}
void persist_apply_pump_kick(persist_t* p, void* pool_arg)
{
if (!p || !p->cluster)
return;
int expected = 0;
if (!atomic_compare_exchange_strong_explicit(
&p->apply_pump_scheduled, &expected, 1, memory_order_acq_rel,
memory_order_relaxed))
return; /* already scheduled/running -- it will pick up any new work */
vtaskmgr_t* pool = (vtaskmgr_t*)pool_arg;
vtask_data_t* data = calloc(1, sizeof(*data));
vtask_t* task = vtask_alloc();
apply_pump_dispatch_t* d = calloc(1, sizeof(*d));
/* Mirrors commit.c's dispatch_continuation fallback exactly: never leave
* a scheduled walk silently unrun. pool == NULL is the expected shape of
* a commit-only / bare-persist-cluster unit fixture that never calls
* commit_set_pool — a real cluster always wires the pool before any
* commit activity (cluster_init_phase2), so this fallback path is the
* normal, deterministic behavior for that class of test, not a rare
* production condition. */
if (pool == NULL || data == NULL || task == NULL || d == NULL) {
free(data);
free(task);
free(d);
apply_pump_drain(p);
return;
}
d->p = p;
data->vtype = VTASK_GENERIC;
data->u.generic = d;
vtask_init(task, (vtask_func_t)apply_pump_vtask_entry, data);
/* VTASK_LIVE_BLOCK is never admission-gated (same delivery guarantee as
* commit.c's own continuation dispatch) — this cannot fast-reject the
* way VTASK_REQ_BLOCK could. */
if (vtask_enq_classified(pool, task, VTASK_LIVE_BLOCK) < 0) {
WARN("persist_apply_pump_kick: vtask enqueue failed -- running "
"inline (stay alive, never leave a scheduled walk unrun)");
free(data);
free(task);
free(d);
apply_pump_drain(p);
}
}
/* Read WAL segment files in LSN order; replay entries with lsn > from_lsn.
*
* Segment filename format: seg-<start_lsn_hex16>.wal
@ -4172,22 +4625,77 @@ int persist_has_pending_apply(persist_t* p, uint64_t lsn)
return found;
}
int persist_take_pending_apply(persist_t* p, uint64_t lsn,
pending_apply_notify_fn* out_fn, void** out_ctx)
{
pending_command_apply_t* prev = NULL;
pending_command_apply_t* n;
int found = 0;
if (p == NULL)
return 0;
pthread_mutex_lock(&p->pending_applies_lock);
for (n = (pending_command_apply_t*)p->pending_applies; n != NULL;
n = n->next) {
if (n->lsn == lsn) {
if (prev == NULL)
p->pending_applies = n->next;
else
prev->next = n->next;
if (out_fn)
*out_fn = n->on_applied;
if (out_ctx)
*out_ctx = n->ctx;
found = 1;
break;
}
prev = n;
}
pthread_mutex_unlock(&p->pending_applies_lock);
free(n); /* NULL when not found -- free(NULL) is a no-op */
return found;
}
/* raft-apply-decoupling task 4.1 (Req 5.1): the receipt-time durable-append
* primitive for a worker-reported job-state fact. Unlike persist_job_hook_fun
* below (an AFTER_EVENT hook that fires only once a mutation has already
* happened), this is a direct, hook-fan-out-free call: job.c's fact receipt
* handlers (record_job_start_fact/record_job_finish_fact/record_job_fail_fact/
* record_job_update_fact) call this INSTEAD of mutating and firing
* AFTER_EVENT, so no notification (scheduler dispatch, slot release,
* apiserver push) can ever fire for an uncommitted fact (Req 5.3). `state`
* is always WAL_MODE_RETAIN for every job-state fact type (verified against
* wal_classify's table) — hardcoded here rather than routed through
* wal_classify since none of these types are detail-disambiguated the way
* WAL_WORKER_CTRL is. */
* record_job_postdone_fact) call this (or persist_submit_job_fact_ex below)
* INSTEAD of mutating and firing AFTER_EVENT, so no notification (scheduler
* dispatch, slot release, apiserver push) can ever fire for an uncommitted
* fact (Req 5.3). `state` is always WAL_MODE_RETAIN for every job-state fact
* type (verified against wal_classify's table) — hardcoded here rather than
* routed through wal_classify since none of these types are detail-
* disambiguated the way WAL_WORKER_CTRL is.
*
* Thin wrapper over persist_submit_job_fact_ex() with pad = 0 — the "legacy
* absolute state" discriminator (see wal_types.h's doc comment on
* wal_jobstate_t._pad). Unchanged contract/signature for every existing
* caller. */
int persist_submit_job_fact(persist_t* p, uint32_t wal_type, uint64_t jobid,
int state)
{
wal_jobstate_t js = { .jobid = jobid, .state = (uint32_t)state };
return persist_submit_job_fact_ex(p, wal_type, jobid, state, 0);
}
/* raft-apply-decoupling task 6.1 round 2 (Req 3.3/7.1-7.3): full-control
* sibling of persist_submit_job_fact() above, letting the caller set
* wal_jobstate_t's `_pad` discriminator explicitly. Used by
* record_job_fail_fact (WAL_JOB_FAIL_STAGE_FLAG | stage) and
* record_job_postdone_fact (WAL_JOB_UPDATE_POSTDONE_MARK) to mark a fact
* whose real mutation must be re-derived at apply time against the job's
* CURRENT state, rather than trusting a receipt-time snapshot baked into
* `state` — see wal_types.h's doc comment on wal_jobstate_t._pad for the
* full rationale. */
int persist_submit_job_fact_ex(persist_t* p, uint32_t wal_type,
uint64_t jobid, int state, uint32_t pad)
{
wal_jobstate_t js = { .jobid = jobid, .state = (uint32_t)state,
._pad = pad };
return wal_hook_append_classified(p, wal_type, WAL_MODE_RETAIN, &js,
sizeof(js));
@ -4205,6 +4713,17 @@ int persist_job_hook_fun(int type, int situation, int detail,
if (situation != HOOK_AFTER_EVENT)
return 0;
/* raft-apply-decoupling task 6.1 (Hazard 1): this AFTER_EVENT firing
* originated from apply_committed_entry()'s own post-mutation notify —
* the entry is already durably committed (that is why it is being
* applied at all), so appending it again here would duplicate it. See
* t_apply_seam_notifying's doc comment (above apply_committed_entry) for
* why this MUST be a call-provenance guard, not a `detail` filter:
* job_ctrl's mark_job_update/mark_job_fail still fire this exact same
* hook for their own live, still-legitimate append. */
if (t_apply_seam_notifying)
return 0;
persist_t* p = cls;
uint32_t wal_type = detail_to_wal_type(detail);
wal_commit_mode_t mode;
@ -4272,6 +4791,17 @@ int persist_queue_hook_fun(int type, int situation, int detail,
if (situation != HOOK_AFTER_EVENT)
return 0;
/* raft-apply-decoupling task 6.1 (Hazard 1, queue analog): see
* persist_job_hook_fun's identical guard above for the full rationale —
* apply_committed_entry()'s WAL_QUEUE_CTRL branch fires this same
* AFTER_EVENT to notify, and this hook appends unconditionally
* (WAL_QUEUE_CTRL is WAL_MODE_WAIT with no early-return here), so
* without this guard a live-applied queue entry would re-append and
* re-trigger a nested majority-commit wait every time the pump walks
* it. */
if (t_apply_seam_notifying)
return 0;
persist_t* p = cls;
uint32_t wal_type = detail_to_wal_type(detail);
wal_commit_mode_t mode;
@ -4310,6 +4840,11 @@ int persist_worker_hook_fun(int type, int situation, int detail,
if (situation != HOOK_AFTER_EVENT)
return 0;
/* raft-apply-decoupling task 6.1 (Hazard 1, worker analog): see
* persist_job_hook_fun's identical guard above for the full rationale. */
if (t_apply_seam_notifying)
return 0;
persist_t* p = cls;
uint32_t wal_type = detail_to_wal_type(detail);
wal_commit_mode_t mode;