lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(job-array-single-wal-record): wire dispatcher was silently dropping compact array pushes

Browse source 
extract_data() (src/lib/request.c), the shared reassembly step every
inbound push passes through inside vtask_entry() before any receiver's
handler runs, had no case for Data_data_jobArray -- it hit
default: rc=-EINVAL. Because vtask_entry()'s callnext check treats any
nonzero return (including a negative errno) as "reassembly complete,
proceed," this never surfaced as an error: the real payload was freed
and the receiver silently handed an empty, zeroed dataset_t, with no
log anywhere in the chain. Tasks 5.1-5.3 were each individually
correct and tested in isolation, but the feature was dead on the real
wire end-to-end.

Adds case Data_data_jobArray: passing the single, unchunked
descriptor straight through (no pagination needed, unlike the
existing LIST-shaped cases) and nulls the source pointer after
transfer -- verified this is not cosmetic, since free_Request()'s
pushmsg case would otherwise double-free it via free_Data()'s live
Data_data_jobArray branch.

Proven with a real xce_encode()/xce_decode() round-trip through the
fixed extract_data() -> sched_handle_dataset() -> sched_on_job_array()
(task 5.3, unmodified) chain, not just a unit-level check on
extract_data() alone. Also confirms task 5.4's future API-server
receive path shares this exact fix, since both go through the same
xcecli_ctx_init() -> vtask_entry() -> extract_data() pipeline.

One self-flagged, non-blocking residual gap (pre-existing since task
5.3, out of this task's request.c-only boundary): dataset->rows for
the jobArray case is never deep-freed downstream, leaking one bounded
xce_job_submit_t per successfully-delivered array push commit.

Task 5.3b of job-array-single-wal-record (Requirements 6.1, 6.2, 6.4, 8.4).
Inserted mid-run after independent review confirmed the gap; user
confirmed sequencing (fix now, before task 5.4).
This commit is contained in:
Rongsong Shen 2026-07-15 10:59:39 +00:00
parent d16847185e
commit 1cbd7df1a8
4 changed files with 216 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

96
src/lib/testsuite/test_request.c
View file

@ -176,6 +176,99 @@ static void test_init_reply_by_req_zeroes_reply(void)
TEST_ASSERT_EQUAL_INT(0, reply.data_which);
}
/* ── job-array-single-wal-record task 5.3b: extract_data() wire dispatch for
* Data_data_jobArray ─────────────────────────────────────────────────────
* extract_data() is the shared inbound-push reassembly step every receiver
* (scheduler, API server) passes through inside vtask_entry() (xcectx.c)
* before its own handler ever runs. Its switch had no case for
* Data_data_jobArray -- it fell into `default: rc = -EINVAL`, and
* vtask_entry()'s `if (callnext)` treats ANY nonzero return (including a
* negative errno) as "reassembly complete, proceed" -- so the real payload
* was silently freed and the receiver was handed an empty, zeroed
* dataset_t{rows=NULL, nrows=0}. This test drives the REAL wire encode/
* decode path (mirrors xcesched_push_jobarray()'s exact construction,
* sched.c, and apiserver_push_jobarray()'s identical shape, apiserver.c) so
* the payload extract_data() sees here is genuinely wire-decoded, not a
* hand-built struct. */
static void test_extract_data_job_array_passes_through_intact(void)
{
job_submit_data_t submit;
xce_job_submit_t job_array;
xce_data_t data;
xce_request_t request;
xce_request_t decoded;
io_packet_t *pkt;
dataset_t dataset;
int rc;
xce_job_submit_t *got;
/* Arrange: build the exact wire shape xcesched_push_jobarray()/
* apiserver_push_jobarray() send: OPCode_push / Request_body_pushmsg
* carrying one Data.jobArray descriptor (base jobid, array_size, one
* shared submit spec) -- never chunked, exactly one per commit. */
memset(&submit, 0, sizeof(submit));
submit.queue = "normal";
submit.user = "tester";
submit.mintasks = 1;
submit.maxtasks = 1;
submit.resreq = "type == local";
memset(&job_array, 0, sizeof(job_array));
job_array.jobid = 70000;
job_array.array_size = 12;
job_array.submit = &submit;
memset(&data, 0, sizeof(data));
data.table = Data_data_jobArray;
data.data.job_array = &job_array;
memset(&request, 0, sizeof(request));
request.opcode = OPCode_push;
request.body_which = Request_body_pushmsg;
request.body.pushmsg = &data;
pkt = (io_packet_t *)calloc(1, sizeof(*pkt));
TEST_ASSERT_NOT_NULL(pkt);
TEST_ASSERT_EQUAL_INT(1, xce_encode(&request, pkt));
memset(&decoded, 0, sizeof(decoded));
TEST_ASSERT_EQUAL_INT(0, xce_decode(pkt, &decoded)); /* consumes/frees pkt */
TEST_ASSERT_EQUAL_INT(Request_body_pushmsg, decoded.body_which);
TEST_ASSERT_NOT_NULL(decoded.body.pushmsg);
TEST_ASSERT_EQUAL_INT(Data_data_jobArray, decoded.body.pushmsg->table);
/* Act: this is exactly what vtask_entry() calls on every inbound push. */
memset(&dataset, 0, sizeof(dataset));
dataset.table = decoded.body.pushmsg->table;
rc = extract_data(&dataset, decoded.body.pushmsg);
/* Assert: nonzero/"reassembly complete" (matches vtask_entry's
* `if (callnext)` truthy contract -- a single, unchunked descriptor never
* waits for more) AND the genuine payload, not an empty dataset. Before
* the fix, rc was -EINVAL (still truthy, so vtask_entry proceeded
* anyway) but dataset.rows stayed NULL/nrows stayed 0 -- exactly the
* silent-drop bug this test pins down. */
TEST_ASSERT_EQUAL_INT(1, rc);
TEST_ASSERT_NOT_NULL(dataset.rows);
TEST_ASSERT_EQUAL_INT(1, dataset.nrows);
got = (xce_job_submit_t *)dataset.rows;
TEST_ASSERT_EQUAL_UINT64(70000, got->jobid);
TEST_ASSERT_EQUAL_UINT64(12, got->array_size);
TEST_ASSERT_NOT_NULL(got->submit);
TEST_ASSERT_EQUAL_STRING("normal", got->submit->queue);
TEST_ASSERT_EQUAL_STRING("tester", got->submit->user);
/* Cleanup: extract_data() transfers ownership of the decoded
* xce_job_submit_t out of decoded.body.pushmsg->data.job_array into
* dataset.rows (nulling the source pointer) -- free the now-hollow
* container and the transferred descriptor separately. */
xce_free_request(&decoded);
free_JobSubmit_ptr((xce_job_submit_t **)&dataset.rows);
printf("test_extract_data_job_array_passes_through_intact: PASS\n");
}
void test_request(void)
{
printf("\n>> host control request validation tests\n\n");
@ -193,4 +286,7 @@ void test_request(void)
RUN_TEST(test_ha_heartbeat_req_null_ep);
RUN_TEST(test_ha_heartbeat_req_roundtrip_fields);
RUN_TEST(test_nop_req_remains_bodyless);
printf("\n>> extract_data() Data_data_jobArray wire dispatch tests\n\n");
RUN_TEST(test_extract_data_job_array_passes_through_intact);
}