test(job-array-single-wal-record): end-to-end integration test for the compact push
Cross-daemon integration test, test-only, spanning the scheduler (C) and API-server (Go/CGO) receive paths built in tasks 5.1-5.4/5.3b. Drives real xce_encode()/xce_decode() wire bytes through the actual production dispatch chain on both daemons -- extract_data() -> scheduler_req_handler() -> sched_handle_dataset() -> sched_on_job_array() and go_push_handler() -> handleDataset() -> cJobArrayToModels()/dispatchJobArray() -- skipping only vtask_entry()'s ~10 lines of static glue, mirrored verbatim in test code (the if (callnext) treats-any-nonzero-as-proceed behavior that caused 5.3b's bug is reproduced exactly). Redelivery is tested via genuinely independent fresh decodes of separately-encoded bytes, not a decode-once-call-twice shortcut. Bug-catching capability proven by temporarily reverting task 5.3b's fix and confirming both the C and Go tests fail meaningfully, then restoring to green -- confirmed independently by both the implementer and reviewer. Task 5.5 of job-array-single-wal-record (Requirements 6.1-6.5, 8.4). This completes Phase 5 (Compact Scheduler/API-Server Push).
This commit is contained in:
parent
9e0d87fe50
commit
b7b90e9d21
5 changed files with 769 additions and 1 deletions
|
|
@ -149,7 +149,7 @@
|
|||
- _Boundary: Compact Scheduler/API-Server Push (apiserver receive path)_
|
||||
- _Depends: 5.2_
|
||||
|
||||
- [ ] 5.5 Integration test: end-to-end compact push
|
||||
- [x] 5.5 Integration test: end-to-end compact push
|
||||
- Commit one array record end-to-end and confirm the scheduler and the API server each receive exactly one push and each expand it into N correctly-addressable entries
|
||||
- Confirm a simulated redelivery (reconnect) to either receiver does not double-expand
|
||||
- Observable: a passing integration test covering both receivers' one-push/N-entries/no-double-expand behavior, including both receivers' confirmation log lines
|
||||
|
|
@ -193,3 +193,4 @@
|
|||
**CRITICAL discovery during this task's review (2026-07-15), NOT a defect in task 5.3 itself**: the shared wire dispatcher `extract_data()` (`src/lib/request.c`) has no case for `Data_data_jobArray` and silently drops every real array push before ANY receiver's handler runs (see new task 5.3b, inserted immediately after this task per explicit user decision). Confirmed to affect task 5.4's future API-server receive path identically (both share the same `xcecli_ctx_init()` → `vtask_entry()` → `extract_data()` pipeline). Task 5.3's own tests are unaffected by this gap since they use the same established white-box convention as every sibling `handler.c` test (calling `sched_handle_dataset()` directly, bypassing the transport layer) — this is not a new testing shortcut, it just means task 5.3's passing tests do not, and were never intended to, prove real-wire delivery.
|
||||
- **Task 5.3b**: Added `case Data_data_jobArray:` to `extract_data()` (`request.c`) — sets `dataset->rows`/`nrows=1` directly (no pagination, matching the always-one-shot wire shape) and nulls `getdata->data.job_array` after transferring it. The null is NOT cosmetic — independently verified `free_Request()`'s `Request_body_pushmsg` case calls `free_Data()`, which has a live `Data_data_jobArray` branch that would double-free/UAF without it. Proven end-to-end with a real `xce_encode()`/`xce_decode()` round-trip through `extract_data()` → `sched_handle_dataset()` → `sched_on_job_array()` (task 5.3, unmodified), not just a unit-level check. Also independently confirmed the alternate hypothesis some earlier notes floated — that `scheduler_req_handler()` itself has a `pushmsg`-rejecting branch — is dead code on the live wire path, since `vtask_entry()` always converts `body_which` to `datamsg` before invoking any handler. **New residual gap, self-flagged, NOT fixed (pre-existing since task 5.3, not introduced here, genuinely out of this task's `request.c`-only boundary)**: `dataset->rows` for the jobArray case is never deep-freed downstream — `free_Dataset()` is an empty no-op for all table types, and `sched_handle_dataset()`'s jobArray case (task 5.3) doesn't null it after consuming it. Net effect: one bounded `xce_job_submit_t` + embedded spec leaks per successfully-delivered array push commit (not per element, not unbounded) — real but small; worth a dedicated follow-up before this spec is considered fully closed, alongside the other self-flagged residual gaps (task 3.1's bulk-insert orphan prefix, task 4.2's boot-replay fence gap).
|
||||
- **Task 5.4**: Go/CGO receive path — `rawJobArrayToModels()` (pure-Go N-way expansion), `cJobArrayToModels()` (CGO adapter sibling to `cJobToModel`), `dispatchJobArray()` (dedup via the SAME `store.Jobs.Has()` the ordinary `Data_data_jobs` path writes through, one confirmation log only on ≥1 newly-dispatched element), new `case C.Data_data_jobArray:` in `handleDataset`. Elements route through `OnJobUpdated` (not `OnJobCreated`), matching the ordinary single-job push's own routing — verified consistent, not a divergence. CGO boundary needed zero changes — `C.xce_job_submit_t` and `C.Data_data_jobArray` were already exposed/generated from prior tasks. A structural TOCTOU exists between the `Has()` dedup check and the eventual store write (two separate lock acquisitions, no compare-and-set primitive) but is narrower than task 5.3's accepted C-side gap (no intervening async hop) and strictly safer than the pre-existing baseline (the ordinary `Data_data_jobs` path has NO dedup check at all today) — not fixed, judged acceptable. `cJobArrayToModels()` has zero direct unit coverage at the CGO-struct level, but this exactly mirrors the pre-existing, repo-wide precedent for `cJobToModel()` itself (also never unit-tested with a real C struct) — reviewer independently reconstructed a full `CGO_ENABLED=1` build+link+test run against the real generated headers to confirm correctness beyond a compile-only check. `store.ArrayJobs` (array-level rollup aggregate) deliberately left unpopulated — confirmed genuinely out of this spec's scope per requirements.md's Boundary Context, which explicitly defers whole-array bulk query/cancel to a future spec; zero code anywhere in the codebase populates it today, including this diff.
|
||||
- **Task 5.5**: Cross-daemon integration test, test-only (zero net production diff, confirmed on `request.c` and every task 5.1-5.4 file). Drives real `xce_encode()`/`xce_decode()` wire bytes through the REAL production dispatch chain on both daemons — `extract_data()` → `scheduler_req_handler()` → `sched_handle_dataset()` → `sched_on_job_array()` (C, via `#include "../src/handler.c"` in the test) and `go_push_handler()` → `handleDataset()` → `cJobArrayToModels()`/`dispatchJobArray()` (Go/CGO) — skipping only `vtask_entry()`'s ~10 lines of `static` glue, which the test mirrors verbatim (independently verified line-by-line accurate on the one behavior that mattered: `if (callnext)` treating any nonzero `extract_data()` return, including a negative errno, as "proceed"). Redelivery is tested via genuinely independent fresh decodes of separately-encoded bytes, not a cheaper decode-once-call-twice shortcut. Bug-catching capability independently proven twice (implementer, then reviewer): temporarily reverting task 5.3b's fix makes both the C and Go tests fail meaningfully; restoring returns both to green. Go/cgo toolchain constraint discovered: `import "C"` is disallowed in any `_test.go` file, forcing a two-file split (`wire_dispatch_support.go` for cgo-preamble helpers, `wire_dispatch_test.go` for the actual tests) — logged as bug-1440/1441/1442. One low-severity, non-blocking nit (not fixed): the C test file's own header comment misdescribes which assertion fails first under the RED-phase revert (claims the `callnext != 0` check; actually a downstream `>= 0` check) — cosmetic only, the test's real bug-catching property is independently confirmed accurate.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue