fix(job-array-single-wal-record): close bug-1443's bulk-insert partial-visibility gap

restore_job_array()'s bulk-insert pass could fail partway through
(key_add() on realloc growth exhaustion), leaving an already-inserted
prefix permanently visible in c->jobmgr but excluded from
out_new_jobs -- so those elements existed and were individually
addressable but never reached HOOK_DETAIL_JOB_NEW, never scheduled or
pushed, not on this attempt or any later successful retry. A literal
violation of Requirements 2.3 and 9.4, tracked as bug-1443.

Adds kvmgr_reserve()/xzarray_reserve(), new purely-additive primitives
in shared kvmgr.c/zarray.c (following the existing kvmgr_capture_keys()
template -- bare extern function, no RTTI vtable slot, caller-holds-
write-lock contract), that pre-grow c->jobmgr's backing store before
the bulk-insert pass begins, closing the dominant N-scaling failure
trigger structurally.

Found a genuine off-by-one in the existing, unmodified
xzarray_ensure_items(): its strict less-than sufficiency check means
reserving exactly array_size and then performing array_size real
stores still triggers one more growth cycle on the last store.
Reserving array_size + 1 is the exact minimal correct compensation,
independently re-derived and confirmed by review with concrete
arithmetic.

For the now-much-narrower residual case (kvtab_add()'s own small
per-entry allocation, or a content collision -- both non-N-scaling,
honestly documented as still theoretically possible): restore_job_array()
now populates out_new_jobs with whatever prefix was genuinely inserted
before a failure, and apply_committed_entry() fires HOOK_DETAIL_JOB_NEW
for that non-empty partial prefix even when the overall return is
negative, while still propagating the negative rc so the LSN correctly
halts and the Live Apply Resource Fence still engages for the
remainder. A genuinely-inserted element is never left un-pushed again.

Task 7.1 of job-array-single-wal-record (Requirements 2.3, 9.4, 9.6).
This commit is contained in:
Rongsong Shen 2026-07-16 15:08:02 +00:00
parent 575f928bbc
commit dcf9f39c22
13 changed files with 910 additions and 36 deletions

View file

@ -780,3 +780,72 @@ typeinfo_t xzarray_type = {
.lock = &(intf_lock_t) { .lock = (lock_fn_t)xzarray_lock,
.unlock = (unlock_fn_t)xzarray_unlock } }
};
/* ------------------------------------------------------------------ */
/* xzarray_reserve -- job-array-single-wal-record task 7.1 (bug-1443) */
/* ------------------------------------------------------------------ */
/* NOTE on why these test-support symbols are NOT wrapped in
* `#ifdef XCE_TESTING` here, unlike job.c's analogous seams: job.c is
* recompiled from source directly into each test binary's own source list
* (controller_unit_test_src, meson.build) with `-DXCE_TESTING` supplied as
* that binary's own c_args, so an `#ifdef XCE_TESTING` guard in job.c is
* evaluated per-binary. zarray.c, by contrast, is compiled EXACTLY ONCE, as
* part of the shared `libxce` static_library() target (src/lib/meson.build)
* -- WITHOUT `-DXCE_TESTING` -- and that single libxce.a is then linked,
* unmodified, into every consumer, test binaries included. Guarding these
* definitions behind `#ifdef XCE_TESTING` here would compile them out of
* libxce.a entirely, in every build, making them permanently unresolvable
* link errors for the test binaries that need them (confirmed empirically:
* this was the very shape of the link failure this fix corrects). The
* header-side declarations (zarray.h) remain guarded, so these symbols stay
* un-prototyped -- and therefore uncallable without a compile error -- from
* any production translation unit; only the symbol bytes are unconditionally
* present in libxce.a, not a callable, declared API. The force-fail flag
* below defaults to false and only ever affects xzarray_reserve() itself
* (never any other function), so its unconditional presence is inert unless
* a test explicitly engages it. File-scope, not per-instance -- mirrors
* job.c's own s_job_array_bulk_insert_force_fail_at convention otherwise. */
static _Atomic _Bool s_xzarray_reserve_force_fail = false;
void xzarray_reserve_test_set_force_fail(void)
{
atomic_store_explicit(&s_xzarray_reserve_force_fail, true,
memory_order_relaxed);
}
void xzarray_reserve_test_clear_force_fail(void)
{
atomic_store_explicit(&s_xzarray_reserve_force_fail, false,
memory_order_relaxed);
}
int xzarray_test_get_caps(var array)
{
if (array == NULL) {
return -EINVAL;
}
meta_obj_t* obj = RTTI_DATA_TO_OBJ(array);
xzarray_t* a = RTTI_OBJ_TO_DATA(xzarray_t, obj);
return (int)a->caps;
}
int xzarray_reserve(var array, int n)
{
if ((array == NULL) || (n < 0)) {
return -EINVAL;
}
if (atomic_load_explicit(&s_xzarray_reserve_force_fail,
memory_order_relaxed)) {
errno = ENOMEM; /* simulated resource-exhaustion failure */
return -ENOMEM;
}
meta_obj_t* obj = RTTI_DATA_TO_OBJ(array);
xzarray_t* a = RTTI_OBJ_TO_DATA(xzarray_t, obj);
return xzarray_ensure_items(a, a->count + n);
}