lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(scheduler-dispatch-commit): controller commits alloc+RUN transition together on successful dispatch (task 3.1)

Browse source 
dispatch_a_job_()'s previously-unconditional job.allocs commit is now
gated on xce_startjob_req() actually succeeding (rc >= 0). In that same
successful branch, under a newly-acquired lock(j, RTTI_LOCK_WRITE) (a
real pre-existing locking gap this call site had -- it took no lock on
the job object itself before this), the job transitions to JOB_STAT_RUN,
dispatch_fail_count resets to 0, and a WAL_JOB_DISPATCH fact is appended
for durability. The failure path is otherwise unchanged -- classification
and resolution of the six failure sites is tasks 3.2/3.3's job, not this
one's.

Adds an XCE_TESTING-only dispatch_a_job_test() wrapper (dispatch_a_job_()
is static) and test_dispatch_commit.c, proving: state->RUN, allocs
committed correctly, counter resets, and durability survives a real
persist_recover() replay with no WAL_JOB_START present.

Independently reviewed and approved. Two pre-existing, out-of-boundary
findings surfaced and logged for later tasks: test_commit_barrier_reply.c's
test_jobdispatch_receipt_ack_no_wal_append will need updating by task 5.4
(Req 5.3 anticipates this), and bug-1429's hang was reconfirmed a 3rd time
(unrelated to this change).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cxa1PmqAB5PyXfJaNvj8TN
This commit is contained in:
Rongsong Shen 2026-08-23 15:24:55 +00:00
parent d87f4bb333
commit d9aedc4b34
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
5 changed files with 699 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

67
src/controller/job.c
View file

@ -1035,24 +1035,71 @@ static int dispatch_a_job_(cluster_t* c, endpoint_t* ep, xce_job_alloc_t* alloc)
free_launchinfo(&launchinfo);
if (j->job.allocs != NULL) {
free_JobAlloc_ptr(&(j->job.allocs));
}
j->job.allocs = (xce_job_alloc_t*)calloc(1, sizeof(xce_job_alloc_t));
j->job.allocs->jobid = alloc->jobid;
j->job.allocs->nhosts = nc;
MOVE_PTR(j->job.allocs->allocs, allocs);
alloc->allocs = NULL;
/* scheduler-dispatch-commit task 3.1 (Req 2.1-2.4, 5.1): the allocation
* commit -- previously unconditional regardless of xce_startjob_req()'s
* outcome -- is now gated on the dispatch to the worker actually
* succeeding, and the job's transition to a running state happens in
* that same successful outcome. Protected under this component's own
* lock(j, RTTI_LOCK_WRITE): dispatch_a_job_() previously took no lock on
* `j` itself here (only on c->jobmgr/c->workermgr/w), a real
* pre-existing gap this restructuring corrects for this call site
* (design.md's Concurrency Strategy note -- not fixed anywhere else in
* this file). On failure, state is left untouched (still PEND) and
* nothing is committed, so there is nothing to release (Req 4.4/4.5
* satisfied vacuously by construction for this path). The worker's own
* later confirmation that the job actually started
* (record_job_start_fact()/WAL_JOB_START) continues to fire exactly as
* it does today, unaffected by this change. */
if (rc >= 0) {
lock(j, RTTI_LOCK_WRITE);
if (rc < 0) {
if (j->job.allocs != NULL) {
free_JobAlloc_ptr(&(j->job.allocs));
}
j->job.allocs = (xce_job_alloc_t*)calloc(1, sizeof(xce_job_alloc_t));
j->job.allocs->jobid = alloc->jobid;
j->job.allocs->nhosts = nc;
MOVE_PTR(j->job.allocs->allocs, allocs);
alloc->allocs = NULL;
j->job.state = JOB_STAT_RUN;
/* Reset the consecutive-retryable-failure counter (task 1.3): a
* fresh successful dispatch clears any prior failure history, so a
* job that later fails again is evaluated against the bound as if
* it had only failed once (Req 4.3; the bound itself is exercised
* end-to-end by task 5.2, not here). */
j->dispatch_fail_count = 0;
/* Durable record of the dispatch-time RUN transition (task 1.2),
* mirroring record_job_start_fact()'s persist_submit_job_fact()
* call shape exactly -- _pad stays 0 (the "legacy: state is
* absolute value" convention, matching every other job-state fact
* committed synchronously here rather than via deferred apply). */
(void)persist_submit_job_fact(&c->persist, WAL_JOB_DISPATCH,
alloc->jobid, JOB_STAT_RUN);
unlock(j);
} else {
SET_ERROR(-rc);
PERROR("xce_startjob_req");
return rc;
}
return rc;
}
#ifdef XCE_TESTING
/* Test-only accessor: dispatch_a_job_() is static/file-local (its only
* caller, handle_jobdispatch(), is unmodified by this task) -- mirroring
* xce_classify_dispatch_fail_test()'s existing XCE_TESTING-only-export
* pattern earlier in this file (xce_controller itself is built without
* -DXCE_TESTING, so this never ships). */
int dispatch_a_job_test(cluster_t* c, endpoint_t* ep, xce_job_alloc_t* alloc)
{
return dispatch_a_job_(c, ep, alloc);
}
#endif /* XCE_TESTING */
int job_ctrl(cluster_t* c, xce_jobctrl_t* ctrinfo)
{
uint64_t jobid = ctrinfo->jobid;