lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs(buglog): bug-1908 fixed in 8b7173bb

Browse source 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-02 02:50:49 +00:00
parent 8b7173bb09
commit 167c854673
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
2 changed files with 15 additions and 12 deletions
Download patch file Download diff file Expand all files Collapse all files

15
.wolf/bugs/bug-1908.md
View file

@ -1,8 +1,8 @@
---
id: "bug-1908"
status: "investigating"
status: "fixed"
status_updated: "2026-10-02"
status_note: "worker drops a kill that arrives before the job is in its worker list; controller-side handling of the worker's error (retry or not) not yet traced"
status_note: ""
component: "worker"
severity: "wrong-result"
timestamp: "2026-10-02T02:27:01Z"
@ -11,6 +11,7 @@ occurrences: 1
files: ["src/worker/src/controller_connector.c"]
tags: ["job-control", "kill", "race", "dispatch"]
related: ["bug-1905"]
fix_commits: ["8b7173bb"]
---
# bug-1908: a kill sent right after dispatch is lost: the worker drops it before registering the job
@ -45,15 +46,15 @@ related: ["bug-1905"]
- Evidence: /home/claude/tmp/kill-race/worker.log
## Root cause
TODO: required for fixed/partial
The controller forwards a job-control signal as soon as the job has an allocation (`src/controller/job.c` job-control handler -> `xce_jobctrl_req`). The worker answers the start request by queuing it on its thread pool (`controller_connector.c` start handler -> `submit_fork_task`) and only adds the job to its worker list when `xcecd_process_command` runs (`xcecd.c`), which for the first jobs after a worker starts was ~3 s later; the real worker can also only signal a job once the job's IPC client has connected. `handle_jobcontrol` (`controller_connector.c`) dropped a signal for a job not in the list ("Failed to find worker", -1) and also for a listed job still PENDING/STARTING, so the kill was lost while the API had already reported it accepted.
## Fix
TODO: required for fixed/partial
New `src/worker/src/pending_signal.c` (`include/pending_signal.h`): a thread-safe table of jobs whose start the worker has accepted. The start handler calls `pending_signal_start()` before queuing the start; `handle_jobcontrol` keeps a signal for a queued or PENDING/STARTING job with `pending_signal_record()` and replies success; `update_worker_status()` takes it on the STARTING -> RUNNING transition and delivers it (virtual: `worker->killed_signal`; real: `ipc_master_send_job_signal`, whose client is connected once the job reports RUNNING). `xcecd_job_start_error()` and the DONE/EXITED transitions drop the entry. A signal for a job the worker knows nothing about still fails as before.
## Verification
- Test: TODO: name + file; RED before the fix, GREEN after
- Suites: TODO: suite results and build profile
- Live: TODO: soak/deploy run, or n/a and why
- Test: `test_jobcontrol_signal_for_queued_start_is_kept` (RED before the fix: `Expected 0 Was -1`) and `test_jobcontrol_signal_for_unknown_job_fails` in `src/worker/testsuite/test_controller_connector.c`; table semantics in the new `src/worker/testsuite/test_pending_signal.c` (6 tests); GREEN after
- Suites: worker meson suite 15/15 OK (asan, src/build.jai)
- Live: fresh xce-deploy-ha + xcecli, kills sent as soon as the jobs listed RUN: the worker logged `has not started yet; signal 15 will be delivered when it runs` for `plain` and `arr[2]` 2.3 s before they started, then `delivering signal 15 kept while it was starting`; both ended EXIT while `arr[1]`/`arr[3]` kept running; a later kill of the running `arr[1]` still worked (/home/claude/tmp/bug-1908/fixed-run.log). The real worker's IPC delivery path is not exercised here (no systemd/dbus in this container).
## References
- Related: none