lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs(buglog): bug-1896 fixed by job-array-one-based-index

Browse source 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-01 14:54:14 +00:00
parent 8a7c134d9b
commit faa228d592
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
2 changed files with 25 additions and 14 deletions
Download patch file Download diff file Expand all files Collapse all files

17
.wolf/buglog.json
View file

@ -16640,9 +16640,9 @@
},
{
"id": "bug-1896",
"status": "open",
"status_updated": "2026-09-30",
"status_note": "newly logged",
"status": "fixed",
"status_updated": "2026-10-01",
"status_note": "",
"component": "controller",
"severity": "wrong-result",
"timestamp": "2026-09-30T04:32:34Z",
@ -16650,8 +16650,8 @@
"occurrences": 1,
"error_message": "Job arrays ignore the indices given in -J: arr[1-3] and arr[1,5,9] both become elements [0]..[2]\n\n**Observed:** submitting `xcecli jobs submit -J 'arr[1-3]' -q normal sleep 300` created elements numbered from 0:\n```\nJob <1-0-1790742148-0[0]> is submitted to default queue <normal>.\n1-0-1790742148-0[0] admin RUN normal xce 1*worker* arr[0]\n1-0-1790742148-0[1] admin RUN normal xce 1*worker* arr[1]\n1-0-1790742148-0[2] admin RUN normal xce 1*worker* arr[2]\n```\n**Expected:** the elements carry the indices the user gave -- [1], [2], [3] here; [1], [5], [9] for `arr[1,5,9]`; steps such as `[1-10:2]` honored -- as in LSF, whose -J array syntax XCE follows (LSF indices start at 1).\n**Impact:** wrong result for every array job whose indices are not exactly 0..n-1: element numbers, %I substitution in output/input file names, and any index-based job control (`jobs kill <id>[5]`) refer to indices the user never asked for; gaps and steps are silently dropped.\n**Found by:** user question while verifying bug-1894's array-job check, 2026-09-29.",
"file": "src/frontend/apiserver/controllers/jobs.go (CreateJob, createArrayName), src/schema/controller.capnp (job submit arraySize), src/include/persist.h (WAL_JOB_ARRAY_SUBMIT), src/controller/job.c (array expansion), src/frontend/apiserver/backend/convert_pure.go (rawJobArrayToModels)",
"root_cause": "TODO: required for fixed/partial",
"fix": "TODO: required for fixed/partial\n\nVerification:\n- Test: TODO: name + file; RED before the fix, GREEN after\n- Suites: TODO: suite results and build profile\n- Live: TODO: soak/deploy run, or n/a and why",
"root_cause": "Every layer numbered elements from 0 and none read the user's index expression. The apiserver's submit path counted the elements in `-J name[...]` and kept only the count, dropping the start, lists and steps; `GenerateJobID`/libxce `idgen` returned the array block with its first element at index 0 (`block|0`), and the scheduler's expansion (`first + i`) and the harness's `jobsub -a` then produced elements 0..n-1. Nothing rejected index 0 or the forms XCE cannot represent.",
"fix": "Elements are numbered 1..N (spec `job-array-one-based-index`, option B: first element = `array-id | 1`, no WAL or wire change):\n- apiserver: `ParseSubmitIndexExpr` accepts only `name[1-N]`/`name[1]` and rejects every other form with a specific message (start other than 1, lists, steps, `%N`, malformed, interactive); `GenerateArrayJobID` returns the array identity (c30e5f1d, 0318ac5f, b88e3526).\n- libxce/controller: `idgen_array` returns `block|1`; the controller rejects a new array starting at 0 with `E_XCE_ARRAY_START0` (ef728435, e6c484a0).\n- harness: `jobsub -a N` creates elements 1..N; `$jobid` is the array identity and `$jobid[k]` an element (d2c87f18, 1812a80d).\n- selections `<array-id>[k]` / `[a-b]` and name selections validated and honored by queries and control (2a8490bc..40917a01, bc616e8c, 58ca7abb); worker sets `XCE_JOBINDEX` to the element number (7aa00f0d).\nScope decision (requirements.md, Out of scope): lists `[1,5,9]`, steps `[1-10:2]` and starts other than 1 are rejected with a message rather than implemented.\n\nVerification:\n- Test: `controllers/jobarray_test.go` (ParseSubmitIndexExpr, RED on the old count-only path), `jobs_handler_test.go` array submit, `lib/testsuite/test_idgen.c` (`idgen_array` returns `block|1`), `test_jobsubmit_validate.c` (index-0 array rejected), `harness/testsuite/test_autovars.c`; each RED before its task's change, GREEN after\n- Suites: meson 126/126 OK; all apiserver and xcecli Go packages ok (asan, src/build.jai, task 10.1)\n- Live: 2026-10-01 xce-deploy-ha + xcecli: `jobs submit -J 'arr[1-3]'` -> `Job <1-0-1790863351-0> ... (3 elements)`, listed as `[1]`,`[2]`,`[3]`; `bad[0-2]` rejected (\"element numbers start at 1\"); `kill <array-id>[2]` acted on element 2 only. HA `--array-soak` and multihost skills PASS with elements 1..N.",
"tags": [
"job-array",
"apiserver",
@ -16662,7 +16662,12 @@
"related_bugs": [
"bug-1894"
],
"fix_commits": [],
"fix_commits": [
"c30e5f1d",
"ef728435",
"e6c484a0",
"d2c87f18"
],
"record": ".wolf/bugs/bug-1896.md"
},
{

22
.wolf/bugs/bug-1896.md
View file

@ -1,8 +1,8 @@
---
id: "bug-1896"
status: "open"
status_updated: "2026-09-30"
status_note: "newly logged"
status: "fixed"
status_updated: "2026-10-01"
status_note: ""
component: "controller"
severity: "wrong-result"
timestamp: "2026-09-30T04:32:34Z"
@ -11,6 +11,7 @@ occurrences: 1
files: ["src/frontend/apiserver/controllers/jobs.go (CreateJob, createArrayName)", "src/schema/controller.capnp (job submit arraySize)", "src/include/persist.h (WAL_JOB_ARRAY_SUBMIT)", "src/controller/job.c (array expansion)", "src/frontend/apiserver/backend/convert_pure.go (rawJobArrayToModels)"]
tags: ["job-array", "apiserver", "controller", "wal", "lsf-compat"]
related: ["bug-1894"]
fix_commits: ["c30e5f1d", "ef728435", "e6c484a0", "d2c87f18"]
---
# bug-1896: Job arrays ignore the indices given in -J: arr[1-3] and arr[1,5,9] both become elements [0]..[2]
@ -42,15 +43,20 @@ Job <1-0-1790742148-0[0]> is submitted to default queue <normal>.
- Evidence: code path -- createArrayName() parses the indices correctly ([1,2,3]), but CreateJob keeps only `arraySize = len(parsedIndices)`; the submit message carries only arraySize (controller.capnp), the WAL_JOB_ARRAY_SUBMIT entry only (base_jobid, array_size) (persist.h), and elements are expanded as base+0..base+size-1 by the controller and by the apiserver's rawJobArrayToModels. Deploy dir /home/claude/tmp/xce-1894b
## Root cause
TODO: required for fixed/partial
Every layer numbered elements from 0 and none read the user's index expression. The apiserver's submit path counted the elements in `-J name[...]` and kept only the count, dropping the start, lists and steps; `GenerateJobID`/libxce `idgen` returned the array block with its first element at index 0 (`block|0`), and the scheduler's expansion (`first + i`) and the harness's `jobsub -a` then produced elements 0..n-1. Nothing rejected index 0 or the forms XCE cannot represent.
## Fix
TODO: required for fixed/partial
Elements are numbered 1..N (spec `job-array-one-based-index`, option B: first element = `array-id | 1`, no WAL or wire change):
- apiserver: `ParseSubmitIndexExpr` accepts only `name[1-N]`/`name[1]` and rejects every other form with a specific message (start other than 1, lists, steps, `%N`, malformed, interactive); `GenerateArrayJobID` returns the array identity (c30e5f1d, 0318ac5f, b88e3526).
- libxce/controller: `idgen_array` returns `block|1`; the controller rejects a new array starting at 0 with `E_XCE_ARRAY_START0` (ef728435, e6c484a0).
- harness: `jobsub -a N` creates elements 1..N; `$jobid` is the array identity and `$jobid[k]` an element (d2c87f18, 1812a80d).
- selections `<array-id>[k]` / `[a-b]` and name selections validated and honored by queries and control (2a8490bc..40917a01, bc616e8c, 58ca7abb); worker sets `XCE_JOBINDEX` to the element number (7aa00f0d).
Scope decision (requirements.md, Out of scope): lists `[1,5,9]`, steps `[1-10:2]` and starts other than 1 are rejected with a message rather than implemented.
## Verification
- Test: TODO: name + file; RED before the fix, GREEN after
- Suites: TODO: suite results and build profile
- Live: TODO: soak/deploy run, or n/a and why
- Test: `controllers/jobarray_test.go` (ParseSubmitIndexExpr, RED on the old count-only path), `jobs_handler_test.go` array submit, `lib/testsuite/test_idgen.c` (`idgen_array` returns `block|1`), `test_jobsubmit_validate.c` (index-0 array rejected), `harness/testsuite/test_autovars.c`; each RED before its task's change, GREEN after
- Suites: meson 126/126 OK; all apiserver and xcecli Go packages ok (asan, src/build.jai, task 10.1)
- Live: 2026-10-01 xce-deploy-ha + xcecli: `jobs submit -J 'arr[1-3]'` -> `Job <1-0-1790863351-0> ... (3 elements)`, listed as `[1]`,`[2]`,`[3]`; `bad[0-2]` rejected ("element numbers start at 1"); `kill <array-id>[2]` acted on element 2 only. HA `--array-soak` and multihost skills PASS with elements 1..N.
## References
- Related: none