62 lines
5.2 KiB
Markdown
62 lines
5.2 KiB
Markdown
---
|
|
id: "bug-1896"
|
|
status: "fixed"
|
|
status_updated: "2026-10-01"
|
|
status_note: ""
|
|
component: "controller"
|
|
severity: "wrong-result"
|
|
timestamp: "2026-09-30T04:32:34Z"
|
|
last_seen: "2026-09-30T04:32:34Z"
|
|
occurrences: 1
|
|
files: ["src/frontend/apiserver/controllers/jobs.go (CreateJob, createArrayName)", "src/schema/controller.capnp (job submit arraySize)", "src/include/persist.h (WAL_JOB_ARRAY_SUBMIT)", "src/controller/job.c (array expansion)", "src/frontend/apiserver/backend/convert_pure.go (rawJobArrayToModels)"]
|
|
tags: ["job-array", "apiserver", "controller", "wal", "lsf-compat"]
|
|
related: ["bug-1894"]
|
|
fix_commits: ["c30e5f1d", "ef728435", "e6c484a0", "d2c87f18"]
|
|
---
|
|
|
|
# bug-1896: Job arrays ignore the indices given in -J: arr[1-3] and arr[1,5,9] both become elements [0]..[2]
|
|
|
|
## Description
|
|
**Observed:** submitting `xcecli jobs submit -J 'arr[1-3]' -q normal sleep 300` created elements numbered from 0:
|
|
```
|
|
Job <1-0-1790742148-0[0]> is submitted to default queue <normal>.
|
|
1-0-1790742148-0[0] admin RUN normal xce 1*worker* arr[0]
|
|
1-0-1790742148-0[1] admin RUN normal xce 1*worker* arr[1]
|
|
1-0-1790742148-0[2] admin RUN normal xce 1*worker* arr[2]
|
|
```
|
|
**Expected:** the elements carry the indices the user gave -- [1], [2], [3] here; [1], [5], [9] for `arr[1,5,9]`; steps such as `[1-10:2]` honored -- as in LSF, whose -J array syntax XCE follows (LSF indices start at 1).
|
|
**Impact:** wrong result for every array job whose indices are not exactly 0..n-1: element numbers, %I substitution in output/input file names, and any index-based job control (`jobs kill <id>[5]`) refer to indices the user never asked for; gaps and steps are silently dropped.
|
|
**Found by:** user question while verifying bug-1894's array-job check, 2026-09-29.
|
|
|
|
## Environment
|
|
- OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
|
|
- libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2)
|
|
- Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
|
|
- Build: asan (xce.build)
|
|
- Code: scheduler-framework @ 38afb98c-dirty
|
|
- Deployment: local 3-node HA controller cluster + xcecd_virtual worker + scheduler + apiserver on :8080 (deploy-ha.sh + start-all.sh)
|
|
|
|
## Reproduction
|
|
1. Deploy a cluster with an apiserver (deploy-ha.sh + scripts/start-all.sh); `xcecli login -u admin -p admin --url http://localhost:8080`
|
|
2. `xcecli jobs submit -J 'arr[1-3]' -q normal sleep 300 </dev/null; xcecli jobs list -u all`
|
|
- Frequency: deterministic -- 1/1, 2026-09-29 (/home/claude/tmp/xce-1894b)
|
|
- Evidence: code path -- createArrayName() parses the indices correctly ([1,2,3]), but CreateJob keeps only `arraySize = len(parsedIndices)`; the submit message carries only arraySize (controller.capnp), the WAL_JOB_ARRAY_SUBMIT entry only (base_jobid, array_size) (persist.h), and elements are expanded as base+0..base+size-1 by the controller and by the apiserver's rawJobArrayToModels. Deploy dir /home/claude/tmp/xce-1894b
|
|
|
|
## Root cause
|
|
Every layer numbered elements from 0 and none read the user's index expression. The apiserver's submit path counted the elements in `-J name[...]` and kept only the count, dropping the start, lists and steps; `GenerateJobID`/libxce `idgen` returned the array block with its first element at index 0 (`block|0`), and the scheduler's expansion (`first + i`) and the harness's `jobsub -a` then produced elements 0..n-1. Nothing rejected index 0 or the forms XCE cannot represent.
|
|
|
|
## Fix
|
|
Elements are numbered 1..N (spec `job-array-one-based-index`, option B: first element = `array-id | 1`, no WAL or wire change):
|
|
- apiserver: `ParseSubmitIndexExpr` accepts only `name[1-N]`/`name[1]` and rejects every other form with a specific message (start other than 1, lists, steps, `%N`, malformed, interactive); `GenerateArrayJobID` returns the array identity (c30e5f1d, 0318ac5f, b88e3526).
|
|
- libxce/controller: `idgen_array` returns `block|1`; the controller rejects a new array starting at 0 with `E_XCE_ARRAY_START0` (ef728435, e6c484a0).
|
|
- harness: `jobsub -a N` creates elements 1..N; `$jobid` is the array identity and `$jobid[k]` an element (d2c87f18, 1812a80d).
|
|
- selections `<array-id>[k]` / `[a-b]` and name selections validated and honored by queries and control (2a8490bc..40917a01, bc616e8c, 58ca7abb); worker sets `XCE_JOBINDEX` to the element number (7aa00f0d).
|
|
Scope decision (requirements.md, Out of scope): lists `[1,5,9]`, steps `[1-10:2]` and starts other than 1 are rejected with a message rather than implemented.
|
|
|
|
## Verification
|
|
- Test: `controllers/jobarray_test.go` (ParseSubmitIndexExpr, RED on the old count-only path), `jobs_handler_test.go` array submit, `lib/testsuite/test_idgen.c` (`idgen_array` returns `block|1`), `test_jobsubmit_validate.c` (index-0 array rejected), `harness/testsuite/test_autovars.c`; each RED before its task's change, GREEN after
|
|
- Suites: meson 126/126 OK; all apiserver and xcecli Go packages ok (asan, src/build.jai, task 10.1)
|
|
- Live: 2026-10-01 xce-deploy-ha + xcecli: `jobs submit -J 'arr[1-3]'` -> `Job <1-0-1790863351-0> ... (3 elements)`, listed as `[1]`,`[2]`,`[3]`; `bad[0-2]` rejected ("element numbers start at 1"); `kill <array-id>[2]` acted on element 2 only. HA `--array-soak` and multihost skills PASS with elements 1..N.
|
|
|
|
## References
|
|
- Related: none
|