74 lines
3.6 KiB
Markdown
74 lines
3.6 KiB
Markdown
---
|
|
id: "bug-1954"
|
|
status: "fixed"
|
|
status_updated: "2026-10-10"
|
|
status_note: ""
|
|
component: "worker"
|
|
severity: "wrong-result"
|
|
timestamp: "2026-10-10T08:46:07Z"
|
|
last_seen: "2026-10-10T08:46:07Z"
|
|
occurrences: 1
|
|
files: ["src/worker/src/xcecd_worker.c"]
|
|
tags: ["worker", "job-control", "signal"]
|
|
related: ["bug-1940"]
|
|
fix_commits: ["148557a5"]
|
|
gitea_issue: 385
|
|
gitea_synced_status: "fixed"
|
|
gitea_synced_hash: "c0260515d036"
|
|
---
|
|
|
|
# bug-1954: xcecd_worker sends SIGTERM for a KILL request without force and SIGSTOP for a TSTP request
|
|
|
|
## Description
|
|
**Observed:** (source reading, 2026-10-10; found while fixing bug-1940)
|
|
`worker_signal_callback()` (`src/worker/src/xcecd_worker.c:1115-1130`)
|
|
replaces the requested signal:
|
|
```c
|
|
case SIGTERM:
|
|
case SIGKILL:
|
|
if (force) { success = signal_gid_except_self(SIGKILL); }
|
|
else { success = signal_gid_except_self(SIGTERM); }
|
|
...
|
|
case SIGSTOP:
|
|
case SIGTSTP:
|
|
success = signal_gid_except_self(SIGSTOP);
|
|
```
|
|
Since bug-1940 the requested signal reaches the worker, but an explicit
|
|
`kill -s KILL` without force still sends SIGTERM, and an explicit TSTP is sent
|
|
as the uncatchable SIGSTOP.
|
|
|
|
**Expected:** the job receives the signal the user asked for; force kill
|
|
(SIGKILL) stays available for a default kill.
|
|
**Impact:** wrong result: a job that ignores SIGTERM survives an explicit
|
|
SIGKILL request; a job that handles SIGTSTP to checkpoint or clean up cannot
|
|
do so.
|
|
**Found by:** review of the bug-1940/1942 fixes, 2026-10-10.
|
|
## Environment
|
|
- OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
|
|
- libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2)
|
|
- Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
|
|
- Build: asan (xce.build)
|
|
- Code: scheduler-framework @ 8568c8ab-dirty
|
|
- Deployment: real `xcecd` worker (systemd/dbus); multi-host for the remote-task case
|
|
|
|
## Reproduction
|
|
1. Run a job that traps TERM (`trap '' TERM; sleep 600`).
|
|
2. `POST /xce/v1/jobs/<id>/control` with action kill, signal `SIGKILL`,
|
|
force_kill false.
|
|
3. The job keeps running (it received SIGTERM).
|
|
- Frequency: deterministic by code reading (not yet run)
|
|
- Evidence: the code quoted in Description; no logs kept.
|
|
## Root cause
|
|
`worker_signal_callback()` (`src/worker/src/xcecd_worker.c`) mapped TERM/KILL to `force ? SIGKILL : SIGTERM` and STOP/TSTP to SIGSTOP; the apiserver's default stop signal was TSTP, relying on that substitution.
|
|
|
|
## Fix
|
|
New `job_signal_to_deliver()` (`src/worker/src/job_signal.c`, `include/job_signal.h`) delivers the requested signal; only a forced TERM escalates to SIGKILL; the callback uses it. The apiserver's default stop signal becomes SIGSTOP (`backend/convert_pure.go` `defaultActionSignal`, `models.DefaultActionSignal`), so a plain stop still suspends jobs that ignore SIGTSTP; the bug-1940 test pinning stop=20 now expects 19.
|
|
|
|
## Verification
|
|
- Test: `test_requested_signal_is_delivered` (`src/worker/testsuite/test_job_signal.c`): RED `test_job_signal.c:14: FAIL: Expected 9 Was 15` (KILL without force), GREEN after; `TestJobCtrlParams_PlainStopSendsSigstop` (`backend/jobctrl_params_test.go`): RED `DefaultActionSignal[stop] = "TSTP", want STOP`, GREEN after.
|
|
- Suites: `xce-build.sh all asan xce.build --incremental` on the merged branch, 2026-10-10: all C suites Ok 155 / Fail 0, frontend Go unit tests ok, xceadmin tests ok; apiserver `CGO_ENABLED=0 go test ./...` (incl. middleware, router, cmd) ok
|
|
- Live: n/a: unit/fixture-level verification only; no live deployment was run (follow-up)
|
|
|
|
## References
|
|
- Fix commit: `148557a5` (on `scheduler-framework`; worker-branch original in the merge)
|
|
- Review: `.kiro/reviews/project-gap-review-2026-10-10.md`
|