myxce/.wolf/bugs/bug-1955.md

id	status	status_updated	status_note	component	severity	timestamp	last_seen	occurrences	files	tags	related	fix_commits	gitea_issue	gitea_synced_status	gitea_synced_hash
bug-1955	partial	2026-10-10	covers a signal arriving after this host accepted the runtasks request; a control that reaches a remote host before its runtasks request arrives is still dropped there (needs a controller-side or pre-start record)	worker	wrong-result	2026-10-10T08:46:10Z	2026-10-10T08:46:10Z	1	src/worker/src/controller_connector.c	worker job-control parallel signal	bug-1942 bug-1908	d3da3140	386	partial	095558c6c67a

bug-1955: xcecd drops a signal for a remote task of a multi-host job that is still starting

Description

Observed: (source reading, 2026-10-10; found while fixing bug-1942) Only the head host's job start opens the pending-signal window (pending_signal_start(d->jobid), src/worker/src/controller_connector.c:1062, bug-1908). The runtasks handler for remote tasks (handle_runtasks(), :1147) never does, so a signal that arrives before the remote task's worker entry is registered is logged and dropped:

XCECD_LOG(XCE_LOG_ERROR, "Failed to find worker for job ID: %s", jobidStr);
FREEUP(task_ctx);
return -1;                               /* controller_connector.c:1606 */

Since bug-1942 every host of a multi-host job receives job control, so this window is now reachable.

Expected: like the head host (bug-1908), a signal for a remote task that is starting is kept and delivered once the task runs. Impact: wrong result: a kill or stop sent right after a multi-host job starts can miss some of its remote tasks, which keep running. Found by: review of the bug-1940/1942 fixes, 2026-10-10.

Environment

• OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
• libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2)
• Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
• Build: asan (xce.build)
• Code: scheduler-framework @ 8568c8ab-dirty
• Deployment: real xcecd worker (systemd/dbus); multi-host for the remote-task case

Reproduction

1. Submit a multi-host job (e.g. /xce-test-multihost topology) and kill it immediately after dispatch.
2. A remote host logs "Failed to find worker for job ID" and its task keeps running.

• Frequency: deterministic by code reading (not yet run)
• Evidence: the code quoted in Description; no logs kept.

Root cause

Only the head-host start path opened the pending-signal window (pending_signal_start, controller_connector.c, bug-1908); handle_runtasks() queued remote tasks without it, so a signal arriving before the task's worker entry registered hit "Failed to find worker" and was dropped.

Fix

handle_runtasks() calls pending_signal_start(jobid) before submit_fork_task() and takes it back if queuing fails; the existing RUNNING transition (xcecd_worker_utils.c, bug-1908) delivers the kept signal to the task.

Verification

• Test: test_jobcontrol_signal_for_queued_remote_task_is_kept (src/worker/testsuite/test_controller_connector.c): RED :1631: FAIL: Expected 0 Was -1. the kill must not be dropped, GREEN after (32 tests, 0 failures).
• Suites: xce-build.sh all asan xce.build --incremental on the merged branch, 2026-10-10: all C suites Ok 155 / Fail 0, frontend Go unit tests ok, xceadmin tests ok; apiserver CGO_ENABLED=0 go test ./... (incl. middleware, router, cmd) ok
• Live: n/a: unit/fixture-level verification only; no live deployment was run (follow-up)

References

• Fix commit: d3da3140 (on scheduler-framework; worker-branch original in the merge)
• Review: .kiro/reviews/project-gap-review-2026-10-10.md