docs(buglog): bug-1967, bug-1955, bug-1936 fixed; log bug-1968 (restart drops options)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
edad8c7385
commit
05465a20bf
5 changed files with 148 additions and 34 deletions
|
|
@ -1,8 +1,8 @@
|
|||
---
|
||||
id: "bug-1936"
|
||||
status: "partial"
|
||||
status: "fixed"
|
||||
status_updated: "2026-10-10"
|
||||
status_note: "exdata/cwd/home now reach the worker; still not honoured end to end: --cwd (job process spawned with cwd NULL, no cwd in job_assignment_t), exec home (no REST field), nofile_limit (not parsed), -i/-o/-e and out_dir, asked hosts and begin/end time (no submit fields), file stage-in/out (not implemented)"
|
||||
status_note: ""
|
||||
component: "apiserver"
|
||||
severity: "wrong-result"
|
||||
timestamp: "2026-10-10T08:14:36Z"
|
||||
|
|
@ -11,7 +11,7 @@ occurrences: 1
|
|||
files: ["src/frontend/apiserver/backend/convert.go", "src/frontend/apiserver/controllers/jobs.go"]
|
||||
tags: ["apiserver", "submit", "exdata", "limits"]
|
||||
related: []
|
||||
fix_commits: ["d01ced9b"]
|
||||
fix_commits: ["d01ced9b", "725b1d23", "edad8c73"]
|
||||
gitea_issue: 367
|
||||
gitea_synced_status: "partial"
|
||||
gitea_synced_hash: "658095c85aa2"
|
||||
|
|
@ -48,14 +48,25 @@ So `--cwd`, `--env`, `--file` transfers, interactive mode, every `*_limit` flag
|
|||
## Root cause
|
||||
`modelJobToSubmit()` (`backend/convert.go`) filled only cmdline, execuser, launch_type and pre/post-exec of `job_startinfo_t`; the extended data `processJobRequest()` (`controllers/jobs.go`) builds (env, limits, transfers, interactive) and the cwd stayed in apiserver memory. Pre-existing, origin not bisected.
|
||||
|
||||
UPDATE (2026-10-10): the remaining gaps: xcecd's job-assignment message to the per-job worker process (`job_assignment_t`, `src/worker/include/uv_ipc.h`) had no cwd or home field, so the job was spawned with no working directory and exechome was never applied; `parse_limits()` (`xcecd_exdata.c`) ignored `nofile_limit`; and the apiserver accepted in/out/err files, out_dir, asked hosts, begin/term time and transfer files, which have no field in the controller's submit data or start info (the worker parses transfer files but never stages them).
|
||||
|
||||
## Fix
|
||||
`backend/convert_pure.go`: new pure `submitStartInfo()` sets exdata from `ExtendData`, execcwd (`ExecCWD`, else `CWD`) and exechome; `rawJobToModel` restores `ExtendData`/`CWD`; new `ValidateExdata()` rejects non-object or >16383-byte exdata (the worker would fail the job). `backend/convert.go`: copies/frees the three fields and reads exdata back for jobs and arrays. `controllers/jobs.go`: validates exdata and no longer writes `"transfer_files": null`. The controller passes start info through unchanged (WAL submit facts encode the whole JobStartInfo).
|
||||
|
||||
UPDATE (2026-10-10, `725b1d23`, `edad8c73`), per remaining item:
|
||||
- `--cwd`: honoured. `job_assignment_t` gains `cwd`/`home`, filled by new `ipc_fill_job_assignment()` (`uv_ipc.c`, always NUL-terminated); new `uv_proc_enter_job_dirs()` (`uv_proc.c`), called from `worker_job_callback()` (`xcecd_worker.c`), enters the cwd before pre-exec, so pre-exec and the job run there. An unusable cwd falls back to `/tmp` with a WARN (xcecli always sends the client's cwd, so failing would break submissions from directories absent on the worker host). The apiserver rejects a relative or >1023-byte cwd.
|
||||
- exec home: honoured. New REST field `exec_home` (`models/jobs.go`, absolute, <=1023 bytes) → start info exechome → the worker sets `HOME` (an explicit `--env HOME=` wins).
|
||||
- `nofile_limit`: honoured. `parse_limits()` reads it into `res_limits_t.nofile_limit`; `xcecd_cgroups.c` sets `LimitNOFILE` on the job unit (code-read only: needs D-Bus). xcecli has no flag for it.
|
||||
- `in_file`, `out_file`, `err_file`, `out_dir`, `asked_hosts`, `begin_time`, `term_time`, `transfer_files`: rejected. New `preCheckSubmitOptions()` (`controllers/jobs.go`) in `CreateJob` returns 400 "job submission option(s) ... not supported" before upload handling or job-id allocation. `-o/-e` were not honoured because the worker's output fetch is locked to its own output path (bug-1508). Swagger regenerated.
|
||||
The same options are still copied and dropped on the job restart path: logged as a separate bug.
|
||||
|
||||
## Verification
|
||||
- Test: `TestSubmitStartInfo_CarriesExdataCwdAndHome`, `TestSubmitStartInfo_PrefersExecCWD`, `TestRawJobArrayToModels_RestoresExdataAndCwd`, `TestValidateExdata` (`backend/convert_test.go`), `TestJobHandler_SubmitRejectsNonObjectExtendData` (`controllers/jobs_handler_test.go`): RED before (e.g. `convert_test.go:1272: exdata: got "", want "{\"limits\":{\"run_limit\":60},...}"`), GREEN after.
|
||||
- Suites: `xce-build.sh all asan xce.build --incremental` on the merged branch, 2026-10-10: all C suites Ok 155 / Fail 0, frontend Go unit tests ok, xceadmin tests ok; apiserver `CGO_ENABLED=0 go test ./...` (incl. middleware, router, cmd) ok
|
||||
- Second fix: `test_assignment_carries_cwd_and_home`, `..._enters_cwd_and_sets_home`, `..._unusable_cwd_falls_back`, `..._spawned_after_enter_runs_in_cwd` (new `src/worker/testsuite/test_job_dirs.c`; RED e.g. `test_job_dirs.c:111:test_assignment_carries_cwd_and_home:FAIL: Expected '/data/run' Was ''`, `test_job_dirs.c:207:...: the job did not write pwd.out in cwd`), `test_apiserver_nofile_limit_parsed` (`test_exdata_parse.c`; RED `:148: FAIL: Expected 2048 Was 0`), `TestJobHandler_SubmitRejectsUnsupportedOptions` (RED `jobs_handler_test.go:1631: in_file: status = 200, want 400`), `TestJobHandler_SubmitRejectsBadCwdAndExecHome` (RED `:1655: exec_home=home/u: status = 500, want 400`), `TestJobHandler_SubmitCarriesCwdAndExecHome`, `TestJobHandler_SubmitNofileLimitInExdata` (`controllers/jobs_handler_test.go`); GREEN after. `TestJobHandler_SubmitArrayFileNameSubstitution` now checks %J/%I substitution on `cwd` (out/err files are refused).
|
||||
- Suites: `xce-build.sh all asan xce.build --incremental` on the merged branch (scheduler-framework @ edad8c73), 2026-10-10: all C suites Ok 158 / Fail 0, frontend Go unit tests (apiserver + xcecli) ok; apiserver `CGO_ENABLED=0 go test ./...` (all packages) ok
|
||||
- Live: n/a: unit/fixture-level verification only; no live deployment was run (follow-up)
|
||||
|
||||
## References
|
||||
- Fix commit: `d01ced9b` (on `scheduler-framework`; worker-branch original in the merge)
|
||||
- Fix commits: `d01ced9b` (first fix), `725b1d23`, `edad8c73` (worktree originals `3e571f25`, `200127bb`)
|
||||
- Review: `.kiro/reviews/project-gap-review-2026-10-10.md`
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue