feat(job-array-single-wal-record): scheduler-side local expansion of the compact push
Adds sched_on_job_array() (handler.c), a sibling case to Data_data_schedjobs decoding the new Data.jobArray variant and expanding it locally into N ordinary schedulable tasks via base_jobid + i, routed through the existing unmodified per-job pipeline (sched_on_jobs()). Each element gets its own deep clone (clone_job_submit_data/clone_job_startinfo) of the shared submit spec, since per-element teardown (free_Job_ptr) frees it independently and a shared pointer would double-free. Dedup reuses sched_get_job() against the same kvtab sched_add_job() guards, rather than a new mechanism. A confirmation log line fires once per successful expansion (skipped on a fully-duplicate redelivery). Adds a new task (5.3b) discovered during this task's review: the shared wire dispatcher extract_data() (src/lib/request.c) has no case for Data_data_jobArray and silently drops every real array push before any receiver's handler runs -- outside every current task's boundary, confirmed to affect the future API-server receive path (task 5.4) identically. design.md's File Structure Plan corrected to record this. User confirmed sequencing: fix it now, before 5.4. Task 5.3 of job-array-single-wal-record (Requirements 6.3, 6.4, 6.5, 8.4).
This commit is contained in:
parent
e0cec75284
commit
d16847185e
6 changed files with 526 additions and 6 deletions
|
|
@ -133,6 +133,7 @@ graph TB
|
|||
- `src/scheduler/src/handler.c` — sibling case to the existing `Data_data_schedjobs` (line 112) decoding the new array variant and expanding it locally into N schedulable tasks via the same `base_jobid + i` arithmetic used by the controller, logging one confirmation line per array (Requirement 6.4).
|
||||
- `src/frontend/apiserver/backend/` — sibling decode path to `cJobToModel` (`convert.go:31`) for the new array variant, expanding N `models.JobInfo` entries locally, logging one confirmation line per array (Requirement 6.4).
|
||||
- `src/lib/jobj.c` — `job_init()`: check `makeJobsub()`'s (and `makeString(src->ckptinfo)`'s) return value; propagate failure instead of storing NULL with a success return (Requirement 10.1). Root-cause and fix the `jobsub_table` initialization-ordering gap behind the previously observed NULL-with-EINVAL failure (Requirement 10.2). `job_destroy()`: fix the `accounts` leak (currently `/* FIXME??? */`, jobj.c:584) by adding the missing free call, following the existing pattern used for `allocs`/`pidinfo`/`run_usage` (Requirement 10.4).
|
||||
- **`src/lib/request.c` (added during implementation, task 5.3b, discovered 2026-07-15):** `extract_data()` — the shared inbound-push reassembly dispatcher every receiver (scheduler, API server) passes through inside `vtask_entry()` (`src/lib/xcectx.c`) before its own handler ever runs. Missed by this component's original architecture analysis, which examined `sched.c`/`apiserver.c`/`handler.c` but not the transport dispatch layer beneath them. Its switch has no case for `Data_data_jobArray` and falls into `default: rc = -EINVAL`; because `vtask_entry()`'s `if (callnext)` treats any nonzero return (including a negative errno) as "reassembly complete, proceed," this silently frees the real payload and hands the receiver an empty `dataset_t`, with no log anywhere in the chain. Add a `case Data_data_jobArray:` passing the single, unchunked descriptor straight through — no pagination loop needed, unlike the existing LIST-shaped cases.
|
||||
|
||||
### New Test Files
|
||||
- `src/controller/testsuite/test_job_array_apply.c` — unit tests: single-record N-way expansion, idempotent re-apply (no duplicate elements), all-or-nothing scratch-discard on a simulated construction failure (never touches `c->jobmgr`), identical results from `apply_committed_entry()` vs. `apply_wal_entry()` for the same record.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue