lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(job-array-single-wal-record): make apply pump respect apply failures

Browse source 
persist_apply_pump_run() no longer discards apply_committed_entry()'s
return value. On a negative rc for any WAL type (not just the new
array type), the walk halts at that LSN — no last_applied_lsn advance,
no pending-apply hand-off — logs a loud error, and retries from the
same position on the next kick, mirroring the existing
apply_pump_read_entry failure branch's shape.

Investigated whether this newly turns the pre-existing job-fact
-ENOENT path into a permanent-wedge hazard: it does not, since
c->jobmgr never deletes entries and facts are only ever applied after
their causally-prior submit already succeeded, so a live -ENOENT there
can only mean genuine divergence — halting is the correct behavior.

Task 4.1 of job-array-single-wal-record (Requirement 9.4).
This commit is contained in:
Rongsong Shen 2026-07-15 03:24:28 +00:00
parent 107e3839d9
commit cc7e99c2f7
3 changed files with 236 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

38
src/controller/persist.c
View file

@ -3343,7 +3343,43 @@ int persist_apply_pump_run(persist_t* p)
* ascending order, so a burst of commands and facts that committed
* out of their original submission order still applies in strict
* commit (LSN) order (Req 3.3, 7.3). */
(void)apply_committed_entry(p, wal_type, next, payload, len);
int apply_rc = apply_committed_entry(p, wal_type, next, payload, len);
if (apply_rc < 0) {
/* job-array-single-wal-record task 4.1 (design.md "Live Apply
* Resource Fence"): this return value was previously discarded
* -- (void)apply_committed_entry(...) -- with last_applied_lsn
* advancing unconditionally, silently and permanently losing
* any apply failure. A negative rc here is never a content-
* derived failure (Req 9.1-9.3's pre-commit validation already
* made those impossible post-commit); it is a node-local
* resource-exhaustion failure (e.g. OOM mid array expansion,
* Req 9.4) that must not be skipped. Mirror the
* apply_pump_read_entry failure branch immediately above: stop
* the walk exactly at this LSN, do not advance
* last_applied_lsn, and do not take this LSN's pending-apply
* hand-off (persist_take_pending_apply/on_applied) -- the entry
* never actually applied, so that hand-off must not fire for
* it. The entry stays committed and will be retried, from the
* same position, on the next kick. This is a GENERAL fix: every
* WAL type's apply outcome is now respected, not just
* WAL_JOB_ARRAY_SUBMIT's. Under NORMAL operation every other
* existing type's apply case returns 0 or positive, so this
* branch is a behavior no-op for them in practice; the
* fact-type cases' pre-existing "-ENOENT: referenced job not
* found" branch (persist.h's dispatch table) is one other real,
* pre-existing source of a negative return here, and now also
* correctly halts the walk here instead of being silently
* discarded, exactly as this fix intends. */
ERROR("persist_apply_pump_run: apply_committed_entry failed for "
"committed entry lsn=%" PRIu64 " wal_type=%" PRIu32
" (rc=%d) -- stopping this pass at this position; the "
"entry stays committed and will be retried, from the same "
"position, on the next kick",
next, wal_type, apply_rc);
free(payload);
pthread_mutex_unlock(&p->apply_pump_lock);
return apply_rc;
}
free(payload);
atomic_store_explicit(&p->last_applied_lsn, next, memory_order_relaxed);