myxce/.wolf/bugs/bug-1980.md

id	status	status_updated	status_note	component	severity	timestamp	last_seen	occurrences	files	tags	related	fix_commits	summary
bug-1980	fixed	2026-10-30	Gated stdin-script read on len(args)==0; regression test + live t1 verification	xcecli	hang	2026-10-30T01:52:00Z	2026-10-30T01:52:00Z	1	src/frontend/xcecli/cmd/jobs/submit.go	cli stdin jobs-submit external-license-resources		01f61507	jobs submit with explicit command blocks forever on a held-open stdin pipe

bug-1980: jobs submit with explicit command blocks forever on a held-open stdin pipe

Description

Observed: a xcecli jobs submit -q normal -R "rusage[extlic=1]" -- /tmp/e2e/licjob.sh sleep 120 process (pid 705795) sat in unix_stream_data_wait on fd 0 for the rest of the session (found in the user's process list the next day). No RPC was ever sent; the job did not exist on the controller. A minimal repro confirms the hang is in the CLI, not the caller:

timeout 8 xcecli jobs submit -q normal -R "rusage[extlic=1]" -- /tmp/e2e/licjob.sh sleep 15 < <(sleep 30)
# rc=124 (timeout kill), no output, no job created

Expected: jobs submit with an explicit command on the command line must submit regardless of what stdin is attached to (bsub semantics: stdin is the job script only when no command is given).

Impact: any tooling or human invocation that leaves a pipe on submit's stdin (heredoc quoting accidents, script wrappers) hangs the CLI forever with no error and no job.

Found by: the user spotted the hung process in their process list the day after the esd-recover verification session (the session's own submit probe, pid 705795).

Environment

• OS: Ubuntu 24.04.2 LTS; kernel Linux 6.8.0-147-generic x86_64
• libc: glibc 2.39 (host ldd); xcecli is static Go (CGO_ENABLED=0, "not a dynamic executable")
• Toolchain: go 1.23.x
• Build: debug (src/build.dbg), deployed /opt/xce-denix/bin/xcecli
• Code: working @ 7520cca7 (E2 external-resources branch line)
• Deployment: t1 test cluster

Reproduction

1. timeout 8 xcecli jobs submit -q normal -- sleep 15 < <(sleep 30) (or any pipe that stays open without writing data or EOF)
2. rc=124, process blocked in unix_stream_data_wait on the stdin socket

• Frequency: deterministic
• Evidence: /proc/705795/fd trace at discovery (2026-10-30 10:19 local); repro above

Root cause

JobSubmitter.ParseCmdArgs (src/frontend/xcecli/cmd/jobs/submit.go:105-114) gated on "stdin is not a char device" and then io.ReadAll(os.Stdin) before looking at the command args: any non-terminal stdin was treated as a potential job script (parseConfigFile, the #xce job submit script feature). A pipe held open without data or EOF blocks io.ReadAll forever. The command args present in the same invocation are never considered — a silent-open stdin wins over the explicit command, and had data arrived it would have been reinterpreted as a config script next to it.

Fix

Gate the stdin-script read on len(args) == 0: stdin is consumed as a job script only when no command is given on the command line; an explicit command wins and stdin is never read (submit.go, ParseCmdArgs). This matches the bsub semantics the CLI's help text claims (command args override config file), removes the unbounded block, and removes the data-ambiguity (stdin bytes silently merged with an explicit command).

Verification

• Test: TestParseCmdArgs_ExplicitCommandIgnoresHeldOpenStdin (src/frontend/xcecli/cmd/jobs/submit_test.go) — a held-open pipe as os.Stdin plus explicit command args; RED pre-fix (test panics on the 30s timeout guard inside io.ReadAll(os.Stdin), confirmed), GREEN post-fix (0.00s).
• Full suite: go test ./... in src/frontend/xcecli — 11/11 packages ok.
• Live (t1, fixed binary): jobs submit ... -- licjob.sh sleep 15 < <(sleep 30) → rc=0, Job submitted instantly, RUN→DONE, pool marker released (pre-fix rc=124).
• Stdin-script path intact: printf '#xce job submit -q normal\nsleep 5\n' | xcecli jobs submit → submitted, DONE (parseConfigFile's os.Exit(0) path cannot be unit-tested in-process; covered live).
• Suites: go test ./... in src/frontend/xcecli — 11/11 packages ok (cmd/jobs, cmd/users, cmd/clusters, cmd/hosts, common/format, common/output, common/request, filters, models, and the remaining two); apiserver/mcp modules untouched by the fix.

References

• Found while verifying xceesd kill/restart recover for the external-resources manual (§11.5 scenario 4); the hung process was that session's own submit probe.