lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(controller): send job control to every host of a multi-host job (bug-1942)

Browse source 
Cause: job_ctrl (job.c) sent kill/stop/resume only to allocs[0]->host.
The tasks on the other hosts are started through runtasks (by the head
worker for a parallel job, or by xrun as the MPI launcher's rsh agent)
and run under their own worker entry on each host; the head worker
signals only its local process group and forwards nothing, so they kept
running.

Fix: job_ctrl sends the request to every allocated host through
job_ctrl_to_host. The head decides the result as before; an unconnected
remote host is logged and skipped (its tasks belong to host failure
handling). Sends are non-blocking enqueues. Test:
testsuite/test_job_ctrl_hosts.c.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-10 08:36:05 +00:00
parent f244661b14
commit 8568c8ab80
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
2 changed files with 106 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

20
src/controller/job.c
View file

@ -2175,11 +2175,25 @@ int job_ctrl(cluster_t* c, xce_jobctrl_t* ctrinfo)
lock(j, RTTI_LOCK_WRITE);
if (j->job.allocs != NULL) {
xce_host_alloc_t** allocs = j->job.allocs->allocs;
int i;
/* job control command will be sent to first node
* for parallel jobs
*/
/* bug-1942: the control goes to every host of the job, not only
* the head (allocs[0]). The tasks on the other hosts are started
* through runtasks (task.c) -- by the head worker for a plain
* parallel job, or by xrun as the MPI launcher's rsh agent -- and
* each runs under that host's own worker entry for this jobid; the
* head worker only signals its local process group and does not
* forward. The head decides the result, as before; an unconnected
* remote host is logged by job_ctrl_to_host and skipped. Each send
* only enqueues on the worker endpoint. */
rc = job_ctrl_to_host(c, allocs[0]->host, ctrinfo);
for (i = 1; i < j->job.allocs->nhosts; i++) {
int rc1 = job_ctrl_to_host(c, allocs[i]->host, ctrinfo);
if ((rc1 < 0) && (rc1 != -E_XCE_BAD_WORKER) && (rc == 0)) {
rc = rc1;
}
}
} else {
/* job has not been started yet */
job_failinfo_t failinfo = {