myxce/.wolf/bugs/bug-1936.md

id	status	status_updated	status_note	component	severity	timestamp	last_seen	occurrences	files	tags	related	fix_commits	gitea_issue	gitea_synced_status	gitea_synced_hash
bug-1936	fixed	2026-10-10		apiserver	wrong-result	2026-10-10T08:14:36Z	2026-10-10T08:14:36Z	1	src/frontend/apiserver/backend/convert.go src/frontend/apiserver/controllers/jobs.go	apiserver submit exdata limits		d01ced9b 725b1d23 edad8c73	367	fixed	150c307e8371

bug-1936: apiserver drops cwd, env, run/resource limits, file transfers and I/O paths of REST job submissions

Description

Observed: (source reading, .kiro/reviews/project-gap-review-2026-10-10.md (P0); to be confirmed by a failing test before the fix) modelJobToSubmit() (src/frontend/apiserver/backend/convert.go:323-375) builds the controller's job_startinfo_t with only cmdline, execuser, launch_type and pre/post-exec; it never sets exdata, exechome or execcwd:

si := (*C.job_startinfo_t)(C.calloc(1, C.sizeof_job_startinfo_t))
si.cmdline = C.CString(job.Command)
... // no si.exdata / si.exechome / si.execcwd

So --cwd, --env, --file transfers, interactive mode, every *_limit flag (incl. -W run limit), -i/-o/-e, asked hosts and begin/end time are kept only in the apiserver's memory and never reach the controller or the worker.

Expected: Every submit option the apiserver accepts reaches the worker in the job's start info (exdata, exec home, cwd), or is rejected as unsupported. Impact: Silent wrong result: jobs run without the limits, environment, working directory and I/O paths the user asked for (e.g. a -W run limit is never enforced). Found by: review: .kiro/reviews/project-gap-review-2026-10-10.md (P0)

Environment

• OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
• libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2)
• Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
• Build: asan (xce.build)
• Code: scheduler-framework @ 94472ee3-dirty
• Deployment: any (found by source reading; the failing test that confirms it is named in Verification)

Reproduction

1. Submit through REST/xcecli with xcecli jobs submit -W 1 --cwd /tmp --env A=1 sleep 120 on any deployment.
2. Inspect the job's start info at the worker (worker log of the launched unit) or the controller's job.start_info: no exdata, no cwd; the job runs past one minute.

• Frequency: deterministic (by code reading; confirmed by the RED test of the fix)
• Evidence: the code sites quoted in Description; no logs kept.

Root cause

modelJobToSubmit() (backend/convert.go) filled only cmdline, execuser, launch_type and pre/post-exec of job_startinfo_t; the extended data processJobRequest() (controllers/jobs.go) builds (env, limits, transfers, interactive) and the cwd stayed in apiserver memory. Pre-existing, origin not bisected.

UPDATE (2026-10-10): the remaining gaps: xcecd's job-assignment message to the per-job worker process (job_assignment_t, src/worker/include/uv_ipc.h) had no cwd or home field, so the job was spawned with no working directory and exechome was never applied; parse_limits() (xcecd_exdata.c) ignored nofile_limit; and the apiserver accepted in/out/err files, out_dir, asked hosts, begin/term time and transfer files, which have no field in the controller's submit data or start info (the worker parses transfer files but never stages them).

Fix

backend/convert_pure.go: new pure submitStartInfo() sets exdata from ExtendData, execcwd (ExecCWD, else CWD) and exechome; rawJobToModel restores ExtendData/CWD; new ValidateExdata() rejects non-object or >16383-byte exdata (the worker would fail the job). backend/convert.go: copies/frees the three fields and reads exdata back for jobs and arrays. controllers/jobs.go: validates exdata and no longer writes "transfer_files": null. The controller passes start info through unchanged (WAL submit facts encode the whole JobStartInfo).

UPDATE (2026-10-10, 725b1d23, edad8c73), per remaining item:

• --cwd: honoured. job_assignment_t gains cwd/home, filled by new ipc_fill_job_assignment() (uv_ipc.c, always NUL-terminated); new uv_proc_enter_job_dirs() (uv_proc.c), called from worker_job_callback() (xcecd_worker.c), enters the cwd before pre-exec, so pre-exec and the job run there. An unusable cwd falls back to /tmp with a WARN (xcecli always sends the client's cwd, so failing would break submissions from directories absent on the worker host). The apiserver rejects a relative or >1023-byte cwd.
• exec home: honoured. New REST field exec_home (models/jobs.go, absolute, <=1023 bytes) → start info exechome → the worker sets HOME (an explicit --env HOME= wins).
• nofile_limit: honoured. parse_limits() reads it into res_limits_t.nofile_limit; xcecd_cgroups.c sets LimitNOFILE on the job unit (code-read only: needs D-Bus). xcecli has no flag for it.
• in_file, out_file, err_file, out_dir, asked_hosts, begin_time, term_time, transfer_files: rejected. New preCheckSubmitOptions() (controllers/jobs.go) in CreateJob returns 400 "job submission option(s) ... not supported" before upload handling or job-id allocation. -o/-e were not honoured because the worker's output fetch is locked to its own output path (bug-1508). Swagger regenerated. The same options are still copied and dropped on the job restart path: logged as a separate bug.

Verification

• Test: TestSubmitStartInfo_CarriesExdataCwdAndHome, TestSubmitStartInfo_PrefersExecCWD, TestRawJobArrayToModels_RestoresExdataAndCwd, TestValidateExdata (backend/convert_test.go), TestJobHandler_SubmitRejectsNonObjectExtendData (controllers/jobs_handler_test.go): RED before (e.g. convert_test.go:1272: exdata: got "", want "{\"limits\":{\"run_limit\":60},...}"), GREEN after.
• Suites: xce-build.sh all asan xce.build --incremental on the merged branch, 2026-10-10: all C suites Ok 155 / Fail 0, frontend Go unit tests ok, xceadmin tests ok; apiserver CGO_ENABLED=0 go test ./... (incl. middleware, router, cmd) ok
• Second fix: test_assignment_carries_cwd_and_home, ..._enters_cwd_and_sets_home, ..._unusable_cwd_falls_back, ..._spawned_after_enter_runs_in_cwd (new src/worker/testsuite/test_job_dirs.c; RED e.g. test_job_dirs.c:111:test_assignment_carries_cwd_and_home:FAIL: Expected '/data/run' Was '', test_job_dirs.c:207:...: the job did not write pwd.out in cwd), test_apiserver_nofile_limit_parsed (test_exdata_parse.c; RED :148: FAIL: Expected 2048 Was 0), TestJobHandler_SubmitRejectsUnsupportedOptions (RED jobs_handler_test.go:1631: in_file: status = 200, want 400), TestJobHandler_SubmitRejectsBadCwdAndExecHome (RED :1655: exec_home=home/u: status = 500, want 400), TestJobHandler_SubmitCarriesCwdAndExecHome, TestJobHandler_SubmitNofileLimitInExdata (controllers/jobs_handler_test.go); GREEN after. TestJobHandler_SubmitArrayFileNameSubstitution now checks %J/%I substitution on cwd (out/err files are refused).
• Suites: xce-build.sh all asan xce.build --incremental on the merged branch (scheduler-framework @ edad8c73), 2026-10-10: all C suites Ok 158 / Fail 0, frontend Go unit tests (apiserver + xcecli) ok; apiserver CGO_ENABLED=0 go test ./... (all packages) ok
• Live: n/a: unit/fixture-level verification only; no live deployment was run (follow-up)

References

• Fix commits: d01ced9b (first fix), 725b1d23, edad8c73 (worktree originals 3e571f25, 200127bb)
• Review: .kiro/reviews/project-gap-review-2026-10-10.md