wdgzyjx/myxce
1
0
Fork 0
forked from lei.chen/myxce
Code Pull requests Activity

feat(job-array-single-wal-record): wire array submissions into handle_jobsubmit

Browse source 
Branches handle_jobsubmit() on array_size: ==1 continues through
today's exact unmodified single-job path; >1 routes through a new
handle_jobsubmit_array() helper that reuses jobsubmit_async_ctx_t
with pending=1 and calls task 2.1's persist_submit_job_array_async()
instead of looping N single-element appends. Range validation
(jobid_range_exists/inflight_range_try_add) unchanged, same order,
strictly before any WAL append, for both branches.

Retires two pre-existing tests whose premise (N independent
per-element WAL entries with mixed outcomes) is now structurally
impossible with one entry per array; restores array-scoped
job-non-existence coverage for the failed-commit case that an
earlier draft of this change had dropped (Req 2.2/9.2).

Task 2.2 of job-array-single-wal-record.
This commit is contained in:
Rongsong Shen 2026-07-14 10:32:59 +00:00
parent 7c4afbbb83
commit d38ca72e76
3 changed files with 174 additions and 119 deletions
Download patch file Download diff file Expand all files Collapse all files

74
src/controller/job.c
View file

@ -923,6 +923,72 @@ static void jobsubmit_commit_continuation(void* ctx, uint64_t lsn, int rc)
jobsubmit_apply_complete, actx);
}
/* job-array-single-wal-record task 2.2 (design.md §WAL Array Submission):
* the array-shaped sibling of handle_jobsubmit()'s single-job append +
* continuation-registration block below, taken for submit->array_size > 1.
* Called strictly AFTER handle_jobsubmit() has already run the SAME
* pre-commit range validation (jobid_range_exists + inflight_range_try_add)
* the single-job path uses — this function never repeats or reorders that
* validation (Req 9.1, 9.2); it only takes over at the point the two paths
* diverge.
*
* jobsubmit_async_ctx_t / jobsubmit_commit_continuation / jobsubmit_apply_
* complete / jobsubmit_reply_success_once / jobsubmit_reply_failure_once are
* reused completely unmodified — task 2.1's persist_submit_job_array_async()
* plugs into the exact same commit-continuation contract as persist_submit_
* job_payloads_async() does for the single-job path (persist.h). The only
* difference from the single-job construction below is actx->pending = 1
* instead of one entry per array element (Req 1.1-1.3): exactly one
* WAL_JOB_ARRAY_SUBMIT record describes the whole array, so exactly one
* outcome must resolve before the batch's single reply fires (Req 9.3). No
* job_t/xce_job_t payload is built here — array expansion into individual
* job entries happens later, in the shared apply seam (task 3.x), not on
* this call path. */
static int handle_jobsubmit_array(cluster_t* c, endpoint_t* ep,
xce_request_t* request, uint64_t jobid,
xce_job_submit_t* submit)
{
xce_reply_t reply;
uint64_t array_lsn = 0;
jobsubmit_async_ctx_t* actx;
init_reply_by_req(&reply, request);
actx = calloc(1, sizeof(*actx));
if (actx == NULL) {
inflight_range_remove(jobid, submit->array_size);
reply.status = -ENOMEM;
reply.data_which = RequestReply_data_nodata;
xce_reply(ep, c->myhost, &reply);
return -ENOMEM;
}
actx->c = c;
actx->ep = ep;
actx->seq = request->seq;
actx->opcode = request->opcode;
actx->jobid = jobid;
actx->array_size = submit->array_size;
actx->pending = 1; /* one WAL_JOB_ARRAY_SUBMIT record, one outcome */
actx->replied = 0;
pthread_mutex_init(&actx->lock, NULL);
endpoint_ref(ep);
/* Mirrors handle_jobsubmit's own call below: the return value carries
* no information this function still needs to act on — fn (jobsubmit_
* commit_continuation) has already been invoked, or is already
* guaranteed to be invoked later, exactly once, by the time this call
* returns (persist_submit_job_array_async's own documented contract,
* persist.h). */
(void)persist_submit_job_array_async(&c->persist, jobid,
submit->array_size, submit,
&array_lsn,
jobsubmit_commit_continuation, actx);
return 0;
}
int handle_jobsubmit(cluster_t* c, endpoint_t* ep, xce_request_t* request)
{
uint64_t jobid;
@ -980,6 +1046,14 @@ int handle_jobsubmit(cluster_t* c, endpoint_t* ep, xce_request_t* request)
return -errno;
}
/* job-array-single-wal-record task 2.2 (Req 1.1-1.3, 8.1, 9.1, 9.2):
* array_size == 1 falls through to today's single-job path below,
* completely unmodified (Req 8.1) — array_size > 1 diverges here,
* strictly AFTER the same pre-commit range validation above, into the
* single-record append path task 2.1 built. */
if (submit->array_size > 1)
return handle_jobsubmit_array(c, ep, request, jobid, submit);
/* Build only the WAL payload data for each array element — no job
* object is constructed or inserted into c->jobmgr until the commit
* below succeeds (raft-apply-decoupling task 3.1 / Req 2.1, 2.3): a