lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs(buglog): bug-1968 fixed (restart carries only honoured options; Jsonb/resreq copies)

Browse source 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-10 14:23:45 +00:00
parent 6a2c69f8bd
commit e88e440b7f
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
2 changed files with 26 additions and 15 deletions
Download patch file Download diff file Expand all files Collapse all files

27
.wolf/bugs/bug-1968.md
View file

@ -1,8 +1,8 @@
---
id: "bug-1968"
status: "open"
status: "fixed"
status_updated: "2026-10-10"
status_note: "newly logged"
status_note: ""
component: "apiserver"
severity: "wrong-result"
timestamp: "2026-10-10T11:55:48Z"
@ -11,6 +11,7 @@ occurrences: 1
files: ["src/frontend/apiserver/controllers/jobs.go"]
tags: ["apiserver", "restart", "submit"]
related: ["bug-1936"]
fix_commits: ["6a2c69f8"]
gitea_issue: 399
gitea_synced_status: "open"
gitea_synced_hash: "a735a6546a0b"
@ -34,6 +35,8 @@ if v, ok := oldJob["out_dir"].(string); ok { ... } // in_file, err_file,
**Impact:** wrong result: a restarted job ignores the asked hosts, begin/term time and I/O paths the user gave, with no error.
**Found by:** review during the bug-1936 fix, 2026-10-10.
UPDATE (2026-10-10, during the fix): the same rebuild also lost options the job DOES get. Rows come from `store.StructToOrderMap`, which keeps each field's Go type, so `oldJob["chkpnt_info"].(string)` and `oldJob["extend_data"].(string)` never matched `models.Jsonb`: env/limits (exdata) were dropped on every restart and a checkpointed job never restarted from its checkpoint (`restart_command`, `XCE_JOB_CHECKPOINT_PID`). It also read `oldJob["res_req"]` (the column is `resreq`), dropping the resource requirement, and never copied `exec_home`.
## Environment
- OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
- libc: glibc 2.42-47 (n/a: Go apiserver, CGO_ENABLED=0 tests)
@ -44,21 +47,27 @@ if v, ok := oldJob["out_dir"].(string); ok { ... } // in_file, err_file,
## Reproduction
1. Submit a job, let it finish.
2. `xcecli jobs restart -m <host> <jobid>` (or restart a job whose original row has begin_time/out_file).
2. `POST /xce/v1/jobs/<jobid>/restart` with body `{"asked_hosts":"<host>"}` (xcecli `jobs restart` has no `-m` flag and sends none), or restart a job whose row has out_file/out_dir set.
3. The restart returns success; the new job is placed without regard to `<host>` (scheduler never receives asked hosts).
- Frequency: deterministic (code path)
- Evidence: the code quoted in Description; no logs kept.
## Root cause
TODO: required for fixed/partial
`RestartJobByID()` (`src/frontend/apiserver/controllers/jobs.go`) rebuilds the new job by copying fields out of the old job's stored row. Its copy list predates bug-1936: it accepted `asked_hosts` in the request and copied asked hosts, begin/term time, in/out/err files and `out_dir`, none of which any job gets. The list was also never checked against the row's real keys and types: JSONB columns are `models.Jsonb` (not `string`) and the resource requirement key is `resreq` (not `res_req`), so those copies silently did nothing. Pre-existing, origin not bisected.
## Fix
TODO: required for fixed/partial
`controllers/jobs.go`:
- `preCheckJobRestartRequest()` refuses a non-blank `asked_hosts` with 400 ("job restart option asked_hosts not supported"), like `preCheckSubmitOptions()` (bug-1936); the request field stays so the client gets that message, not "unknown field". `processJobRestartRequest()` (which only copied asked hosts) is removed.
- The restart no longer carries over asked hosts, begin/term time, in/out/err files or `out_dir`.
- New `rowJSON()` reads a JSONB row column as `models.Jsonb` or string; used for `chkpnt_info` and `extend_data`, so a checkpointed job restarts with its restart command and checkpoint pid, and every restart keeps the old job's extended data (env, limits).
- Copies `resreq` (was `res_req`) and `exec_home`.
## Verification
- Test: TODO: name + file; RED before the fix, GREEN after
- Suites: TODO: suite results and build profile
- Live: TODO: soak/deploy run, or n/a and why
- Test: `TestJobHandler_RestartRejectsAskedHosts`, `TestJobHandler_RestartCarriesOnlyHonouredOptions`, `TestJobHandler_RestartFromCheckpoint` (`src/frontend/apiserver/controllers/jobs_handler_test.go`): RED before (`jobs_handler_test.go:1537: status = 200, want 400`; `:1592: restarted job kept asked_hosts, which no job gets` (also in_file/out_file/err_file/out_dir); `:1597: ExtendData = "", want ...`; `:1606: ResReq = "", want rusage[mem=100]`; `:1637: Command = "sleep 1", want the checkpoint's restart command`), GREEN after; with the fix stashed the three fail again. `TestJobHandler_RestartNamedJob` (bug-1898) still passes.
- Suites: `xce-build.sh apiserver asan xce.build --incremental` (CGO build; apiserver + xcecli Go tests all ok), 2026-10-10; apiserver `CGO_ENABLED=0 go test ./...` all packages ok
- Live: n/a: handler-level fixture tests only; no live restart of a checkpointed job was run (needs CRIU on a real worker)
## References
- Related: none
- Fix commit: `6a2c69f8`
- Related: bug-1936 (submit-side rejection), bug-1898 (restart identity)