From a437eff2c961e0de6f16cc40ed1058e31a2bc8ca Mon Sep 17 00:00:00 2001 From: "lei.chen" Date: Fri, 30 Oct 2026 02:02:04 +0000 Subject: [PATCH] wolf: log bug-1980 (jobs submit stdin block) as fixed Co-Authored-By: Claude Code --- .wolf/buglog.json | 26 ++++++++++++ .wolf/bugs/bug-1980.md | 91 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 117 insertions(+) create mode 100644 .wolf/bugs/bug-1980.md diff --git a/.wolf/buglog.json b/.wolf/buglog.json index 843699eb..ecde947b 100644 --- a/.wolf/buglog.json +++ b/.wolf/buglog.json @@ -245,6 +245,32 @@ "00f81c5a" ], "record": ".wolf/bugs/bug-1979.md" + }, + { + "id": "bug-1980", + "status": "fixed", + "status_updated": "2026-10-30", + "status_note": "Gated stdin-script read on len(args)==0; regression test + live t1 verification", + "component": "xcecli", + "severity": "hang", + "timestamp": "2026-10-30T01:52:00Z", + "last_seen": "2026-10-30T01:52:00Z", + "occurrences": 1, + "error_message": "jobs submit with explicit command blocks forever on a held-open stdin pipe\n\n**Observed:** a `xcecli jobs submit -q normal -R \"rusage[extlic=1]\" -- /tmp/e2e/licjob.sh\nsleep 120` process (pid 705795) sat in `unix_stream_data_wait` on fd 0 for the rest of\nthe session (found in the user's process list the next day). No RPC was ever sent; the\njob did not exist on the controller. A minimal repro confirms the hang is in the CLI,\nnot the caller:\n\n```\ntimeout 8 xcecli jobs submit -q normal -R \"rusage[extlic=1]\" -- /tmp/e2e/licjob.sh sleep 15 < <(sleep 30)\n# rc=124 (timeout kill), no output, no job created\n```\n\n**Expected:** `jobs submit` with an explicit command on the command line must submit\nregardless of what stdin is attached to (bsub semantics: stdin is the job script only\nwhen no command is given).\n\n**Impact:** any tooling or human invocation that leaves a pipe on submit's stdin\n(heredoc quoting accidents, script wrappers) hangs the CLI forever with no error and\nno job.\n\n**Found by:** the user spotted the hung process in their process list the day after\nthe esd-recover verification session (the session's own submit probe, pid 705795).", + "file": "src/frontend/xcecli/cmd/jobs/submit.go", + "root_cause": "`JobSubmitter.ParseCmdArgs` (src/frontend/xcecli/cmd/jobs/submit.go:105-114) gated on\n\"stdin is not a char device\" and then `io.ReadAll(os.Stdin)` **before** looking at the\ncommand args: any non-terminal stdin was treated as a potential job script\n(`parseConfigFile`, the `#xce job submit` script feature). A pipe held open without\ndata or EOF blocks `io.ReadAll` forever. The command args present in the same\ninvocation are never considered — a silent-open stdin wins over the explicit command,\nand had data arrived it would have been reinterpreted as a config script next to it.", + "fix": "Gate the stdin-script read on `len(args) == 0`: stdin is consumed as a job script only\nwhen no command is given on the command line; an explicit command wins and stdin is\nnever read (`submit.go`, ParseCmdArgs). This matches the bsub semantics the CLI's help\ntext claims (`command args override config file`), removes the unbounded block, and\nremoves the data-ambiguity (stdin bytes silently merged with an explicit command).\n\nVerification:\n- Test: `TestParseCmdArgs_ExplicitCommandIgnoresHeldOpenStdin`\n (src/frontend/xcecli/cmd/jobs/submit_test.go) — a held-open pipe as os.Stdin plus\n explicit command args; RED pre-fix (test panics on the 30s timeout guard inside\n `io.ReadAll(os.Stdin)`, confirmed), GREEN post-fix (0.00s).\n- Full suite: `go test ./...` in src/frontend/xcecli — 11/11 packages ok.\n- Live (t1, fixed binary): `jobs submit ... -- licjob.sh sleep 15 < <(sleep 30)` →\n rc=0, Job submitted instantly, RUN→DONE, pool marker released (pre-fix rc=124).\n- Stdin-script path intact: `printf '#xce job submit -q normal\\nsleep 5\\n' |\n xcecli jobs submit` → submitted, DONE (parseConfigFile's os.Exit(0) path cannot be\n unit-tested in-process; covered live).\n- Suites: `go test ./...` in src/frontend/xcecli — 11/11 packages ok (cmd/jobs,\n cmd/users, cmd/clusters, cmd/hosts, common/format, common/output, common/request,\n filters, models, and the remaining two); apiserver/mcp modules untouched by the fix.", + "tags": [ + "cli", + "stdin", + "jobs-submit", + "external-license-resources" + ], + "related_bugs": [], + "fix_commits": [ + "01f61507" + ], + "record": ".wolf/bugs/bug-1980.md" } ] } diff --git a/.wolf/bugs/bug-1980.md b/.wolf/bugs/bug-1980.md new file mode 100644 index 00000000..dd0784bb --- /dev/null +++ b/.wolf/bugs/bug-1980.md @@ -0,0 +1,91 @@ +--- +id: "bug-1980" +status: "fixed" +status_updated: "2026-10-30" +status_note: "Gated stdin-script read on len(args)==0; regression test + live t1 verification" +component: "xcecli" +severity: "hang" +timestamp: "2026-10-30T01:52:00Z" +last_seen: "2026-10-30T01:52:00Z" +occurrences: 1 +files: ["src/frontend/xcecli/cmd/jobs/submit.go"] +tags: ["cli", "stdin", "jobs-submit", "external-license-resources"] +related: [] +fix_commits: ["01f61507"] +summary: "jobs submit with explicit command blocks forever on a held-open stdin pipe" +--- + +# bug-1980: jobs submit with explicit command blocks forever on a held-open stdin pipe + +## Description +**Observed:** a `xcecli jobs submit -q normal -R "rusage[extlic=1]" -- /tmp/e2e/licjob.sh +sleep 120` process (pid 705795) sat in `unix_stream_data_wait` on fd 0 for the rest of +the session (found in the user's process list the next day). No RPC was ever sent; the +job did not exist on the controller. A minimal repro confirms the hang is in the CLI, +not the caller: + +``` +timeout 8 xcecli jobs submit -q normal -R "rusage[extlic=1]" -- /tmp/e2e/licjob.sh sleep 15 < <(sleep 30) +# rc=124 (timeout kill), no output, no job created +``` + +**Expected:** `jobs submit` with an explicit command on the command line must submit +regardless of what stdin is attached to (bsub semantics: stdin is the job script only +when no command is given). + +**Impact:** any tooling or human invocation that leaves a pipe on submit's stdin +(heredoc quoting accidents, script wrappers) hangs the CLI forever with no error and +no job. + +**Found by:** the user spotted the hung process in their process list the day after +the esd-recover verification session (the session's own submit probe, pid 705795). + +## Environment +- OS: Ubuntu 24.04.2 LTS; kernel Linux 6.8.0-147-generic x86_64 +- libc: glibc 2.39 (host ldd); xcecli is static Go (CGO_ENABLED=0, "not a dynamic executable") +- Toolchain: go 1.23.x +- Build: debug (src/build.dbg), deployed /opt/xce-denix/bin/xcecli +- Code: working @ 7520cca7 (E2 external-resources branch line) +- Deployment: t1 test cluster + +## Reproduction +1. `timeout 8 xcecli jobs submit -q normal -- sleep 15 < <(sleep 30)` (or any pipe that + stays open without writing data or EOF) +2. rc=124, process blocked in `unix_stream_data_wait` on the stdin socket +- Frequency: deterministic +- Evidence: /proc/705795/fd trace at discovery (2026-10-30 10:19 local); repro above + +## Root cause +`JobSubmitter.ParseCmdArgs` (src/frontend/xcecli/cmd/jobs/submit.go:105-114) gated on +"stdin is not a char device" and then `io.ReadAll(os.Stdin)` **before** looking at the +command args: any non-terminal stdin was treated as a potential job script +(`parseConfigFile`, the `#xce job submit` script feature). A pipe held open without +data or EOF blocks `io.ReadAll` forever. The command args present in the same +invocation are never considered — a silent-open stdin wins over the explicit command, +and had data arrived it would have been reinterpreted as a config script next to it. + +## Fix +Gate the stdin-script read on `len(args) == 0`: stdin is consumed as a job script only +when no command is given on the command line; an explicit command wins and stdin is +never read (`submit.go`, ParseCmdArgs). This matches the bsub semantics the CLI's help +text claims (`command args override config file`), removes the unbounded block, and +removes the data-ambiguity (stdin bytes silently merged with an explicit command). + +## Verification +- Test: `TestParseCmdArgs_ExplicitCommandIgnoresHeldOpenStdin` + (src/frontend/xcecli/cmd/jobs/submit_test.go) — a held-open pipe as os.Stdin plus + explicit command args; RED pre-fix (test panics on the 30s timeout guard inside + `io.ReadAll(os.Stdin)`, confirmed), GREEN post-fix (0.00s). +- Full suite: `go test ./...` in src/frontend/xcecli — 11/11 packages ok. +- Live (t1, fixed binary): `jobs submit ... -- licjob.sh sleep 15 < <(sleep 30)` → + rc=0, Job submitted instantly, RUN→DONE, pool marker released (pre-fix rc=124). +- Stdin-script path intact: `printf '#xce job submit -q normal\nsleep 5\n' | + xcecli jobs submit` → submitted, DONE (parseConfigFile's os.Exit(0) path cannot be + unit-tested in-process; covered live). +- Suites: `go test ./...` in src/frontend/xcecli — 11/11 packages ok (cmd/jobs, + cmd/users, cmd/clusters, cmd/hosts, common/format, common/output, common/request, + filters, models, and the remaining two); apiserver/mcp modules untouched by the fix. + +## References +- Found while verifying xceesd kill/restart recover for the external-resources + manual (§11.5 scenario 4); the hung process was that session's own submit probe.