wdgzyjx/myxce
1
0
Fork 0
forked from lei.chen/myxce
Code Pull requests Activity

fix(job-array-single-wal-record): close bug-1443's bulk-insert partial-visibility gap

Browse source 
restore_job_array()'s bulk-insert pass could fail partway through
(key_add() on realloc growth exhaustion), leaving an already-inserted
prefix permanently visible in c->jobmgr but excluded from
out_new_jobs -- so those elements existed and were individually
addressable but never reached HOOK_DETAIL_JOB_NEW, never scheduled or
pushed, not on this attempt or any later successful retry. A literal
violation of Requirements 2.3 and 9.4, tracked as bug-1443.

Adds kvmgr_reserve()/xzarray_reserve(), new purely-additive primitives
in shared kvmgr.c/zarray.c (following the existing kvmgr_capture_keys()
template -- bare extern function, no RTTI vtable slot, caller-holds-
write-lock contract), that pre-grow c->jobmgr's backing store before
the bulk-insert pass begins, closing the dominant N-scaling failure
trigger structurally.

Found a genuine off-by-one in the existing, unmodified
xzarray_ensure_items(): its strict less-than sufficiency check means
reserving exactly array_size and then performing array_size real
stores still triggers one more growth cycle on the last store.
Reserving array_size + 1 is the exact minimal correct compensation,
independently re-derived and confirmed by review with concrete
arithmetic.

For the now-much-narrower residual case (kvtab_add()'s own small
per-entry allocation, or a content collision -- both non-N-scaling,
honestly documented as still theoretically possible): restore_job_array()
now populates out_new_jobs with whatever prefix was genuinely inserted
before a failure, and apply_committed_entry() fires HOOK_DETAIL_JOB_NEW
for that non-empty partial prefix even when the overall return is
negative, while still propagating the negative rc so the LSN correctly
halts and the Live Apply Resource Fence still engages for the
remainder. A genuinely-inserted element is never left un-pushed again.

Task 7.1 of job-array-single-wal-record (Requirements 2.3, 9.4, 9.6).
This commit is contained in:
Rongsong Shen 2026-07-16 15:08:02 +00:00
parent 575f928bbc
commit dcf9f39c22
13 changed files with 910 additions and 36 deletions
Download patch file Download diff file Expand all files Collapse all files

65
src/controller/testsuite/test_apply_committed_entry.c
View file

@ -662,6 +662,70 @@ static void test_ace_job_array_submit_forced_failure_fires_no_hook(void)
tace_teardown();
printf("test_ace_job_array_submit_forced_failure_fires_no_hook: PASS\n");
}
/* ── job-array-single-wal-record task 7.1 (bug-1443 partial-prefix fix):
* a bulk-insert-phase failure (the narrower residual gap, distinct from the
* scratch-build-phase failure above) must STILL fire HOOK_DETAIL_JOB_NEW --
* for exactly the genuinely-inserted prefix -- while still returning the
* negative rc so persist_apply_pump_run() halts/fences for the remainder.
* Before this task, this exact scenario fired NO hook at all (see
* test_ace_job_array_submit_forced_failure_fires_no_hook above, which forces
* the OTHER, scratch-build-phase seam and correctly still expects no hook --
* that one never inserts anything real, so no hook is correct there; this
* test's bulk-insert-phase seam DOES insert a real, permanent prefix, so a
* hook is now required). */
static void test_ace_job_array_submit_bulk_insert_failure_fires_hook_for_prefix(void)
{
tace_setup();
const uint64_t base_jobid = 5400;
const uint64_t array_size = 5;
job_submit_data_t spec = tace_array_make_spec();
uint8_t* payload = NULL;
size_t plen = 0;
TEST_ASSERT_EQUAL_INT(0, tace_encode_job_submit_array(base_jobid,
array_size, &spec,
&payload, &plen));
xce_add_hook(&g_tace_c.hooksys, XCE_CONTROLLER_HOOK_JOB,
tace_array_spy_hook, NULL);
/* Force the bulk-insert pass (NOT the scratch-build pass) to fail at
* index 2 -- indices 0 and 1 are genuinely, permanently inserted into
* c->jobmgr via restore_job_array()'s real key_add() calls before this
* fires. */
job_array_apply_test_set_bulk_insert_force_fail_at(2);
int rc = apply_committed_entry(&g_tace_p, WAL_JOB_ARRAY_SUBMIT, 63,
payload, (uint32_t)plen);
TEST_ASSERT_TRUE_MESSAGE(rc < 0,
"a bulk-insert-phase failure must still propagate as a negative"
" errno so the caller halts/fences the LSN");
TEST_ASSERT_EQUAL_INT_MESSAGE(1, g_tace_hook_fires,
"the genuinely-inserted prefix must still get exactly ONE"
" HOOK_DETAIL_JOB_NEW fire -- task 7.1's bug-1443 partial-prefix fix"
" -- even though the overall call reports failure");
TEST_ASSERT_EQUAL_INT_MESSAGE(2, g_tace_last_new_list_count,
"the fire must carry exactly the 2 genuinely-inserted elements, not"
" the full array_size and not zero");
for (uint64_t i = 0; i < 2; i++) {
TEST_ASSERT_NOT_NULL_MESSAGE(tace_find_job(base_jobid + i),
"the genuinely-inserted prefix must be visible in c->jobmgr");
}
for (uint64_t i = 2; i < array_size; i++) {
TEST_ASSERT_NULL_MESSAGE(tace_find_job(base_jobid + i),
"indices never reached by key_add() must not be visible");
}
job_array_apply_test_clear_bulk_insert_force_fail();
free(payload);
tace_teardown();
printf("test_ace_job_array_submit_bulk_insert_failure_fires_hook_for_"
"prefix: PASS\n");
}
#endif /* XCE_TESTING */
/* ── Defensive / boundary ─────────────────────────────────────────────── */
@ -688,6 +752,7 @@ void test_apply_committed_entry(void)
RUN_TEST(test_ace_job_array_submit_idempotent_replay_no_extra_notify);
#ifdef XCE_TESTING
RUN_TEST(test_ace_job_array_submit_forced_failure_fires_no_hook);
RUN_TEST(test_ace_job_array_submit_bulk_insert_failure_fires_hook_for_prefix);
#endif
RUN_TEST(test_ace_null_persist_returns_error);
UNITY_END();