diff --git a/.wolf/buglog.json b/.wolf/buglog.json index 3fe2f02d..665fa013 100644 --- a/.wolf/buglog.json +++ b/.wolf/buglog.json @@ -16640,9 +16640,9 @@ }, { "id": "bug-1896", - "status": "open", - "status_updated": "2026-09-30", - "status_note": "newly logged", + "status": "fixed", + "status_updated": "2026-10-01", + "status_note": "", "component": "controller", "severity": "wrong-result", "timestamp": "2026-09-30T04:32:34Z", @@ -16650,8 +16650,8 @@ "occurrences": 1, "error_message": "Job arrays ignore the indices given in -J: arr[1-3] and arr[1,5,9] both become elements [0]..[2]\n\n**Observed:** submitting `xcecli jobs submit -J 'arr[1-3]' -q normal sleep 300` created elements numbered from 0:\n```\nJob <1-0-1790742148-0[0]> is submitted to default queue .\n1-0-1790742148-0[0] admin RUN normal xce 1*worker* arr[0]\n1-0-1790742148-0[1] admin RUN normal xce 1*worker* arr[1]\n1-0-1790742148-0[2] admin RUN normal xce 1*worker* arr[2]\n```\n**Expected:** the elements carry the indices the user gave -- [1], [2], [3] here; [1], [5], [9] for `arr[1,5,9]`; steps such as `[1-10:2]` honored -- as in LSF, whose -J array syntax XCE follows (LSF indices start at 1).\n**Impact:** wrong result for every array job whose indices are not exactly 0..n-1: element numbers, %I substitution in output/input file names, and any index-based job control (`jobs kill [5]`) refer to indices the user never asked for; gaps and steps are silently dropped.\n**Found by:** user question while verifying bug-1894's array-job check, 2026-09-29.", "file": "src/frontend/apiserver/controllers/jobs.go (CreateJob, createArrayName), src/schema/controller.capnp (job submit arraySize), src/include/persist.h (WAL_JOB_ARRAY_SUBMIT), src/controller/job.c (array expansion), src/frontend/apiserver/backend/convert_pure.go (rawJobArrayToModels)", - "root_cause": "TODO: required for fixed/partial", - "fix": "TODO: required for fixed/partial\n\nVerification:\n- Test: TODO: name + file; RED before the fix, GREEN after\n- Suites: TODO: suite results and build profile\n- Live: TODO: soak/deploy run, or n/a and why", + "root_cause": "Every layer numbered elements from 0 and none read the user's index expression. The apiserver's submit path counted the elements in `-J name[...]` and kept only the count, dropping the start, lists and steps; `GenerateJobID`/libxce `idgen` returned the array block with its first element at index 0 (`block|0`), and the scheduler's expansion (`first + i`) and the harness's `jobsub -a` then produced elements 0..n-1. Nothing rejected index 0 or the forms XCE cannot represent.", + "fix": "Elements are numbered 1..N (spec `job-array-one-based-index`, option B: first element = `array-id | 1`, no WAL or wire change):\n- apiserver: `ParseSubmitIndexExpr` accepts only `name[1-N]`/`name[1]` and rejects every other form with a specific message (start other than 1, lists, steps, `%N`, malformed, interactive); `GenerateArrayJobID` returns the array identity (c30e5f1d, 0318ac5f, b88e3526).\n- libxce/controller: `idgen_array` returns `block|1`; the controller rejects a new array starting at 0 with `E_XCE_ARRAY_START0` (ef728435, e6c484a0).\n- harness: `jobsub -a N` creates elements 1..N; `$jobid` is the array identity and `$jobid[k]` an element (d2c87f18, 1812a80d).\n- selections `[k]` / `[a-b]` and name selections validated and honored by queries and control (2a8490bc..40917a01, bc616e8c, 58ca7abb); worker sets `XCE_JOBINDEX` to the element number (7aa00f0d).\nScope decision (requirements.md, Out of scope): lists `[1,5,9]`, steps `[1-10:2]` and starts other than 1 are rejected with a message rather than implemented.\n\nVerification:\n- Test: `controllers/jobarray_test.go` (ParseSubmitIndexExpr, RED on the old count-only path), `jobs_handler_test.go` array submit, `lib/testsuite/test_idgen.c` (`idgen_array` returns `block|1`), `test_jobsubmit_validate.c` (index-0 array rejected), `harness/testsuite/test_autovars.c`; each RED before its task's change, GREEN after\n- Suites: meson 126/126 OK; all apiserver and xcecli Go packages ok (asan, src/build.jai, task 10.1)\n- Live: 2026-10-01 xce-deploy-ha + xcecli: `jobs submit -J 'arr[1-3]'` -> `Job <1-0-1790863351-0> ... (3 elements)`, listed as `[1]`,`[2]`,`[3]`; `bad[0-2]` rejected (\"element numbers start at 1\"); `kill [2]` acted on element 2 only. HA `--array-soak` and multihost skills PASS with elements 1..N.", "tags": [ "job-array", "apiserver", @@ -16662,7 +16662,12 @@ "related_bugs": [ "bug-1894" ], - "fix_commits": [], + "fix_commits": [ + "c30e5f1d", + "ef728435", + "e6c484a0", + "d2c87f18" + ], "record": ".wolf/bugs/bug-1896.md" }, { diff --git a/.wolf/bugs/bug-1896.md b/.wolf/bugs/bug-1896.md index b0181a63..f9c46284 100644 --- a/.wolf/bugs/bug-1896.md +++ b/.wolf/bugs/bug-1896.md @@ -1,8 +1,8 @@ --- id: "bug-1896" -status: "open" -status_updated: "2026-09-30" -status_note: "newly logged" +status: "fixed" +status_updated: "2026-10-01" +status_note: "" component: "controller" severity: "wrong-result" timestamp: "2026-09-30T04:32:34Z" @@ -11,6 +11,7 @@ occurrences: 1 files: ["src/frontend/apiserver/controllers/jobs.go (CreateJob, createArrayName)", "src/schema/controller.capnp (job submit arraySize)", "src/include/persist.h (WAL_JOB_ARRAY_SUBMIT)", "src/controller/job.c (array expansion)", "src/frontend/apiserver/backend/convert_pure.go (rawJobArrayToModels)"] tags: ["job-array", "apiserver", "controller", "wal", "lsf-compat"] related: ["bug-1894"] +fix_commits: ["c30e5f1d", "ef728435", "e6c484a0", "d2c87f18"] --- # bug-1896: Job arrays ignore the indices given in -J: arr[1-3] and arr[1,5,9] both become elements [0]..[2] @@ -42,15 +43,20 @@ Job <1-0-1790742148-0[0]> is submitted to default queue . - Evidence: code path -- createArrayName() parses the indices correctly ([1,2,3]), but CreateJob keeps only `arraySize = len(parsedIndices)`; the submit message carries only arraySize (controller.capnp), the WAL_JOB_ARRAY_SUBMIT entry only (base_jobid, array_size) (persist.h), and elements are expanded as base+0..base+size-1 by the controller and by the apiserver's rawJobArrayToModels. Deploy dir /home/claude/tmp/xce-1894b ## Root cause -TODO: required for fixed/partial +Every layer numbered elements from 0 and none read the user's index expression. The apiserver's submit path counted the elements in `-J name[...]` and kept only the count, dropping the start, lists and steps; `GenerateJobID`/libxce `idgen` returned the array block with its first element at index 0 (`block|0`), and the scheduler's expansion (`first + i`) and the harness's `jobsub -a` then produced elements 0..n-1. Nothing rejected index 0 or the forms XCE cannot represent. ## Fix -TODO: required for fixed/partial +Elements are numbered 1..N (spec `job-array-one-based-index`, option B: first element = `array-id | 1`, no WAL or wire change): +- apiserver: `ParseSubmitIndexExpr` accepts only `name[1-N]`/`name[1]` and rejects every other form with a specific message (start other than 1, lists, steps, `%N`, malformed, interactive); `GenerateArrayJobID` returns the array identity (c30e5f1d, 0318ac5f, b88e3526). +- libxce/controller: `idgen_array` returns `block|1`; the controller rejects a new array starting at 0 with `E_XCE_ARRAY_START0` (ef728435, e6c484a0). +- harness: `jobsub -a N` creates elements 1..N; `$jobid` is the array identity and `$jobid[k]` an element (d2c87f18, 1812a80d). +- selections `[k]` / `[a-b]` and name selections validated and honored by queries and control (2a8490bc..40917a01, bc616e8c, 58ca7abb); worker sets `XCE_JOBINDEX` to the element number (7aa00f0d). +Scope decision (requirements.md, Out of scope): lists `[1,5,9]`, steps `[1-10:2]` and starts other than 1 are rejected with a message rather than implemented. ## Verification -- Test: TODO: name + file; RED before the fix, GREEN after -- Suites: TODO: suite results and build profile -- Live: TODO: soak/deploy run, or n/a and why +- Test: `controllers/jobarray_test.go` (ParseSubmitIndexExpr, RED on the old count-only path), `jobs_handler_test.go` array submit, `lib/testsuite/test_idgen.c` (`idgen_array` returns `block|1`), `test_jobsubmit_validate.c` (index-0 array rejected), `harness/testsuite/test_autovars.c`; each RED before its task's change, GREEN after +- Suites: meson 126/126 OK; all apiserver and xcecli Go packages ok (asan, src/build.jai, task 10.1) +- Live: 2026-10-01 xce-deploy-ha + xcecli: `jobs submit -J 'arr[1-3]'` -> `Job <1-0-1790863351-0> ... (3 elements)`, listed as `[1]`,`[2]`,`[3]`; `bad[0-2]` rejected ("element numbers start at 1"); `kill [2]` acted on element 2 only. HA `--array-soak` and multihost skills PASS with elements 1..N. ## References - Related: none