forked from lei.chen/myxce
feat(job-array-single-wal-record): implement restore_job_array primitive
Adds the shared N-way array expansion primitive: decodes a committed WAL_JOB_ARRAY_SUBMIT payload, scratch-builds every element via a throwaway kv-table (never touching c->jobmgr during construction), then makes the whole set visible in one bulk pass under a single write-lock acquisition -- no window where a concurrent reader can observe a partially-constructed array. Idempotent per index (skips already-existing jobids; whole-record no-op if all exist). Includes an XCE_TESTING-guarded fault-injection seam. Not yet wired into any live dispatch path (tasks 3.2/3.3). Two review rounds: round 1 found the bulk-insert pass silently claimed success even when key_add() itself could fail on allocation exhaustion (c->jobmgr grows via realloc, no reservation primitive exists anywhere in kv.c/kvmgr.c/zarray.c to pre-check this). Fixed by checking key_add()'s return and reporting failure honestly instead of a false success. The underlying table has no key_del, so an already-inserted prefix on this specific failure mode can remain genuinely visible while the call reports failure -- documented as a residual gap in design.md and tasks.md for tasks 3.2/4.x to account for (that prefix is never pushed to scheduler/API server, even on a later successful retry, since it's never included in out_new_jobs). Task 3.1 of job-array-single-wal-record.
This commit is contained in:
parent
78eb983a74
commit
7c755dc4d7
7 changed files with 1346 additions and 1 deletions
|
|
@ -50,7 +50,7 @@
|
|||
- _Boundary: WAL Array Submission_
|
||||
|
||||
- [ ] 3. Array Apply Expansion: shared N-way expansion and dispatch wiring
|
||||
- [ ] 3.1 Implement the shared array expansion primitive
|
||||
- [x] 3.1 Implement the shared array expansion primitive
|
||||
- Decode the committed array payload and, for each index in the array's range, build a job entry without yet making it visible to any reader
|
||||
- Skip building an entry for any index that already exists (idempotent re-apply); if every index already exists, treat the whole record as an already-applied no-op
|
||||
- Once every index has successfully built, make the whole set visible in one atomic step; if any index fails to build, discard everything built in this attempt without ever making any of it visible — a retry starts over from the first index
|
||||
|
|
@ -173,3 +173,4 @@
|
|||
- **Task 1.3**: `meson test --suite xce`'s `libxce test` target SIGABRTs under ASAN leak detection whenever any code path exercises a `job_init()` failure — root cause is the vendored `rtti` library's `makeInstance()` not freeing its allocation when a type's `.init()` returns negative (no `.c` source available, headers-only in the nix store). This is pre-existing and independently reproduced against an unmodified pre-task `jobj.c`, not introduced by this feature. It will fire again on any future task whose tests exercise a `create_job()`/`makeJob()` failure path (e.g. task 3.1/3.4's fault-injection tests for `restore_job_array()`) — expect the same SIGABRT-not-assertion-failure signature (`grep ":FAIL"` on the Unity log will be empty; the failure is LeakSanitizer-only) and do not treat it as a new regression without the same stash-and-reproduce check task 1.3's review used.
|
||||
- **Task 1.3**: `job_obj_init()`/`job_obj_fini()` are now idempotent (atomic_flag-guarded) — safe to call more than once without an intervening `fini()`. Any future test construction can rely on this instead of coordinating a single call site.
|
||||
- **Task 1.3**: `makeJobsub()`/`makeJobstartinfo()` now return `-ENODEV` (not a generic `-EINVAL`) when the interning table isn't initialized — if a later task's error handling branches on `job_init()`'s return value, `-ENODEV` is a distinct, checkable signal for "table not ready" vs. other construction failures.
|
||||
- **Task 3.1**: `restore_job_array()`'s atomicity has one narrow, documented exception (design.md updated to match): the final bulk-insert-into-`c->jobmgr` pass can itself fail on allocation/growth exhaustion (`c->jobmgr` grows via `realloc()`; no capacity-reservation primitive exists anywhere in `kv.c`/`kvmgr.c`/`zarray.c` to pre-check this before mutating). On this specific failure, the function never silently claims success — it returns a negative errno — but an already-inserted prefix `[0, inserted)` can be left genuinely, permanently visible in `c->jobmgr` (no `key_del` exists to undo it), and that prefix is NEVER included in `out_new_jobs`, so `HOOK_DETAIL_JOB_NEW` never fires for it, not on this attempt or any later successful retry of the same LSN. **Tasks 3.2/4.x must account for this**: those elements are individually addressable (status/cancel/completion work normally) but permanently un-pushed to the scheduler/API server unless something later re-derives and re-notifies them — this is a real gap in the current design, not yet solved, and task 4.x's Live Apply Resource Fence design should explicitly decide whether/how to handle it (e.g. detecting this residual state and re-notifying on self-heal) rather than only handling the construction-phase failure case.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue