2.9 KiB
| id | status | status_updated | status_note | component | severity | timestamp | last_seen | occurrences | files | tags | related | fix_commits | gitea_issue | gitea_synced_status | gitea_synced_hash | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| bug-1942 | fixed | 2026-10-10 | controller | wrong-result | 2026-10-10T08:14:51Z | 2026-10-10T08:14:51Z | 1 |
|
|
|
373 | fixed | 4eed5d6f82ea |
bug-1942: xce_controller sends job control of a multi-host job only to its first host
Description
Observed: (source reading, .kiro/reviews/project-gap-review-2026-10-10.md (P0); to be confirmed by a failing test before the fix)
Job control is routed to allocs[0]->host only (src/controller/job.c:2126-2131).
Expected: Signals reach every host of the job (or the job's launcher is documented to fan them out and does so).
Impact: Suspending or killing a multi-host job can leave its tasks on other hosts running.
Found by: review: .kiro/reviews/project-gap-review-2026-10-10.md (P0)
Environment
- OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
- libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2)
- Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
- Build: asan (xce.build)
- Code: scheduler-framework @ 94472ee3-dirty
- Deployment: any (found by source reading; the failing test that confirms it is named in Verification)
Reproduction
- Run a 2-host job (e.g.
/xce-test-multihost), stop it. - Tasks on the second host keep running.
- Frequency: deterministic (by code reading; confirmed by the RED test of the fix)
- Evidence: the code sites quoted in Description; no logs kept.
Root cause
job_ctrl() (src/controller/job.c) routed control to allocs[0]->host only. Remote tasks (started via runtasks by the head worker or by xrun) run under their own host's worker entry, and the head worker signals only its own process group, so signals never reached other hosts.
Fix
job.c:job_ctrl: sends the request to every allocated host through job_ctrl_to_host (queue-only, non-blocking); the head host decides the result; an unconnected non-head host is logged and skipped. A remaining worker-side window (signal while a remote task starts) was bug-1955.
Verification
- Test:
test_jc_multi_host_job_reaches_every_hostand two related tests (test_job_ctrl_hosts.c): RED:305: FAIL: Expected 1 Was 0 (host jch-h1 of a 3-host job must get the control), GREEN after. - Suites:
xce-build.sh all asan xce.build --incrementalon the merged branch, 2026-10-10: all C suites Ok 155 / Fail 0, frontend Go unit tests ok, xceadmin tests ok; apiserverCGO_ENABLED=0 go test ./...(incl. middleware, router, cmd) ok - Live: not run: a live
/xce-test-multihoststop/kill check is a follow-up
References
- Fix commit:
8568c8ab(onscheduler-framework; worker-branch original in the merge) - Review:
.kiro/reviews/project-gap-review-2026-10-10.md