Rongsong Shen
d26ff8870d
The buglog was one 27K-line JSON file whose fields covered only symptom,
root cause and fix, as free text: 2% of entries named the code version a
bug was found on, 12% the build profile, 36% a way to reproduce, and 58% of
the file was OpenWolf hook auto-detections ("Missing guard clause",
"Function not marked async" on .md files).
- .claude/skills/buglog/bug-record.md: the record standard. Eight
sections (Summary as the title, Description, Environment, Reproduction;
Root cause, Fix, Verification, References once fixed), each with the
content it must contain -- e.g. Environment needs the OS, the libc the
failing binary loads (from its ELF interpreter: glibc 2.42 in the nix
shell vs 2.40 on the host), toolchain, build profile and commit.
- .wolf/bugs/<id>.md: 500 records migrated from buglog.json; every field
verified to round-trip. They carry legacy: true and an explicit "not
recorded" where the old log had no such field; component derived from
file paths, severity from keywords (else unknown), fix_commits from
hashes in status_note/fix that git resolves (91 of 463 fixed).
- .wolf/buglog-archive.json: the 710 hook auto-detections, verbatim.
- .wolf/buglog.json is now an index generated from the records, keeping
the fields OpenWolf's hooks and `openwolf bug search` read.
- scripts/buglog.py rewritten: new (template + Environment collected from
the machine and --binary), check, status (refuses a status the record
doesn't support; fixed needs Root cause, Fix, Verification and
--commit hashes git resolves), note, recur, set, show, list, triage,
env, index. 29 unittest cases in scripts/tests/test_buglog.py.
- scripts/git-hooks/pre-commit runs `check --staged`: incomplete staged
records, a legacy record hand-changed to fixed without the fix-time
sections, a new record claiming legacy: true, or a stale index refuse
the commit.
- scripts/gitea-sync.py reads the records: issue body = the record's
sections plus linked fix commits (origin remote, GITEA_COMMIT_REPO to
override), and a body hash so an issue's text is refreshed when the
record changes.
- bug-1866 rewritten to the full standard as the worked example.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
3.6 KiB
| id | status | status_updated | status_note | component | severity | timestamp | last_seen | occurrences | files | tags | related | legacy | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| bug-1820 | fixed | 2026-09-24 | build | unknown | 2026-09-17T02:45:00Z | 2026-09-19T09:00:00Z | 1 |
|
|
|
true |
bug-1820: User report: after deploying a real (root, sudo-launched) xcecd worker via /xce-deploy-ha and…
Description
User report: after deploying a real (root, sudo-launched) xcecd worker via /xce-deploy-ha and stopping the cluster with scripts/stop-all.sh, xcecd keeps running -- ps shows a leftover xcecd process (a different PID than the one stop-all.sh reported stopping) still bound to the deploy dir's logs after stop-all.sh reported "Stopping worker-1 (PID )" with no warning.
Environment
Not recorded (migrated from buglog.json).
Reproduction
Not recorded (migrated from buglog.json).
Root cause
start-worker.sh's real-worker branch writes the pidfile via echo $$ > "$PIDFILE" and then does exec sudo "$WORKER_BIN" ... -- exec replaces the pidfile-writing shell's own process image with sudo IN PLACE (same PID), so the PID recorded in the pidfile is sudo's own PID, not the actual worker daemon's. sudo then forks a SEPARATE, root-owned child process that execs and runs the real xcecd daemon; sudo itself stays alive as a thin parent/monitor. stop-all.sh's (and cleanup.sh's, identical logic) kill loop did kill -9 "$PID" against the recorded (sudo) PID -- SIGKILL terminates sudo instantly but is never forwarded to its child (SIGKILL cannot be caught/relayed by the dying parent), so the real xcecd process is silently orphaned (reparented to init) and keeps running. The scripts' own pre-existing "still alive after kill -9" warning branch also never fired for this exact case, because it re-checks the SAME (sudo) PID, which genuinely is dead by then -- so the orphaned child produced zero warning output, making the bug fully silent.
Fix
FIXED (2026-09-17). In both generated scripts/stop-all.sh and scripts/cleanup.sh (src/tools/deploy-ha.sh), before killing a pidfile'd PID, check whether that PID's own /proc/$PID/cmdline starts with "sudo " (the telltale sign left by exec sudo ...); if so, find its child process(es) via pgrep -P "$PID" and kill them first via sudo kill -9 (they run as root), before killing the sudo PID itself. The post-kill "still alive" check was also extended to additionally test for the child PID's continued existence (via /proc/$CHILD presence, which needs no kill permission, since kill -0 on a root-owned PID from a non-root caller returns EPERM regardless of whether the process is alive). Live-verified end-to-end: deployed a fresh 3-node cluster with XCE_REAL_WORKER=1 (forcing the sudo xcecd path), confirmed the exact sudo-parent/root-child PID pair via ps axwww, ran the fixed stop-all.sh and confirmed both processes died with the new "is a sudo wrapper -- also stopping its child" line printed and zero processes remaining afterward; repeated the same reproduction against cleanup.sh with the same clean result. No unit test exists for this bash tooling (deploy-ha.sh has no meson-registered test suite, consistent with other shell tooling in this project e.g. xce-build.sh); verification is this live end-to-end reproduction.
Verification
Not recorded (migrated from buglog.json).
References
- Related: bug-1720