lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(job-array-single-wal-record): build the compact scheduler push

Browse source 
Adds xcesched_push_jobarray() (sched.c), encoding one compact
(base jobid, array_size, one spec) Data.jobArray descriptor -- reusing
the JobSubmit capnp struct already added to the Data union in task 1.2
-- instead of N individually-encoded job structs. sched_job_new_fun()
branches on l->count > 1 to call it; the existing single-job path
(l->count == 1 -> xcesched_push_joblist()) is byte-for-byte unchanged.

Sends via direct xce_request()/OPCode_push/Request_body_pushmsg rather
than the existing xce_push_data_req() helper, which hard-rejects any
payload outside jobs/workers/resources/resvals and cannot carry a
single non-list descriptor. design.md corrected to record this
transport deviation for task 5.2 to follow.

New standalone sched_push_unit_test binary (registered in
meson.build): sched.c/dataex.c/egress.c were never linked into
controller_unit_test, and two existing test files there ship local
fakes of on_scheduler_join()/notify()/collect_confvars() that would
collide with the real definitions at link time.

Task 5.1 of job-array-single-wal-record (Requirements 6.1, 6.3, 6.4).
This commit is contained in:
Rongsong Shen 2026-07-15 08:03:04 +00:00
parent 1888498401
commit 5e7a4aa392
5 changed files with 385 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

68
src/controller/sched.c
View file

@ -11,6 +11,62 @@
extern void collect_shared_resvals(cluster_t*, collect_check_fn_t, xlist_t*);
/* job-array-single-wal-record task 5.1 (design.md "Compact Scheduler/
* API-Server Push", File Structure Plan sched.c bullet; Req 6.1, 6.3, 6.4):
* encodes ONE compact (base jobid, array size, one shared spec) Data.jobArray
* push -- reusing the JobSubmit capnp struct (already added to the Data
* union in task 1.2, controller.capnp), never N individually-encoded Job
* structs. (base_jobid, array_size) are derived from the already-constructed
* xlist_t the HOOK_DETAIL_JOB_NEW fire carries (apply_committed_entry's
* WAL_JOB_ARRAY_SUBMIT case, task 3.2): the first element's jobid is the
* array base and l->count is the array size; every element shares the same
* submit spec by construction (restore_job_array's own construction
* discipline, persist.c), so any one element's submit pointer is the one
* shared spec to encode.
*
* Sent via xce_request() directly (OPCode_push / Request_body_pushmsg)
* rather than xce_push_data_req(): that helper's pagination loop exists to
* chunk large LIST payloads (jobs/workers) across multiple packets and has
* no case for this single, non-chunkable descriptor -- adding one would mean
* editing shared transport code outside this task's sched.c-only boundary
* (task 5.2's apiserver push is a separate, possibly-concurrent task that
* would collide on the same edit). The wire mechanism itself (OPCode_push,
* Request_body_pushmsg, the xce_data_t/Data union envelope) is unchanged --
* this mirrors xce_push_req()'s (request.c, static) exact shape for a single
* non-list xce_data_t payload. */
static int xcesched_push_jobarray(cluster_t* c, xlist_t* l)
{
xce_job_t* first;
xce_job_submit_t job_array;
xce_data_t data;
xce_request_t request;
if ((c == NULL) || (l == NULL) || (l->count <= 0)) {
return -EINVAL;
}
first = (xce_job_t*)xlist_get(l, 0);
if ((first == NULL) || (first->submit == NULL)) {
return -EINVAL;
}
memset(&job_array, 0x0, sizeof(job_array));
job_array.jobid = first->jobid;
job_array.array_size = (uint64_t)l->count;
job_array.submit = first->submit;
memset(&data, 0x0, sizeof(data));
data.table = Data_data_jobArray;
data.data.job_array = &job_array;
memset(&request, 0x0, sizeof(request));
request.opcode = OPCode_push;
request.body_which = Request_body_pushmsg;
request.body.pushmsg = &data;
return xce_request(c->sched.ep, &request);
}
static int xcesched_push_joblist(cluster_t* c, xlist_t* l)
{
xce_data_t data;
@ -280,7 +336,17 @@ static int sched_job_new_fun(int type, int situation, void* data, va_list ap)
case HOOK_AFTER_EVENT:
rc = -EINVAL;
if (l != NULL) {
rc = xcesched_push_joblist(c, l);
/* job-array-single-wal-record task 5.1 (Req 6.1, 6.3, 6.4):
* apply_committed_entry()'s WAL_JOB_ARRAY_SUBMIT case is the
* ONLY way this hook is ever fired with more than one
* element (Req 3.4 — one HOOK_DETAIL_JOB_NEW fire per
* committed array record, carrying every constructed
* element). Route that case through the compact descriptor
* push instead of the per-job push. The ordinary
* single-job branch (l->count == 1) below is the exact,
* unmodified call this function always made (Req 8.1). */
rc = (l->count > 1) ? xcesched_push_jobarray(c, l)
: xcesched_push_joblist(c, l);
}
break;
default: