docs(job-array-one-based-index): document the supported array syntax (task 9.3)

src/docs/xce.md gains a 'Job arrays' section: name[1-N] and name[1],
numbering from 1, the API-server limit (default 1000, at most 32767),
<array-id>[k] and <array-id>[a-b] selection (with the peek/restart
caveat), XCE_JOBINDEX, the harness jobsub -a N / $jobid[k], and every
LSF array feature this release does not support. The superseded
statements in job-array-single-wal-record and xce-test-e2e-primitives
are annotated (history kept), the controller-job-cleanup and
job-dependency briefs describe 1-based numbering, and the roadmap gains
an entry.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

.kiro/specs/controller-job-cleanup/brief.md

@@ -84,7 +84,11 @@ Codebase exploration, 2026-09-29.
   (`opget.c:159-176`, `xceapi.c:787`). Control operations return `-ENOENT`.
 - Worker resync ignores unknown jobids (`resync.c:638-647`), which is safe.
 - Array jobs: there is no parent object. Each element is an independent job with
-  a jobid in `[base, base+array_size)`.
+  its own jobid. Elements are numbered 1..N per `job-array-one-based-index`
+  (2026-10-01): element k of array `<array-id>` is `<array-id>[k]`, k = 1..N
+  (arrays submitted before that change may still have elements 0..N-1). The
+  earlier note here, jobid in `[base, base+array_size)`, described numbering
+  from 0.
 - There is no accounting or history file, so a removed job leaves no record.
 
 **Job dependencies**

.kiro/specs/job-array-one-based-index/tasks.md

@@ -217,7 +217,7 @@
   - Done when both scripts pass a syntax check and every element check in them refers to indices 1..N
   - _Requirements: 4.1, 4.5_
 
-- [ ] 9.3 Document the supported array syntax
+- [x] 9.3 Document the supported array syntax
   - Add a "Job arrays" section to the user documentation listing the supported forms and the LSF features not supported (other starts, lists, steps, `%N`, per-element `%I`/`%J`, interactive arrays, `LSB_JOBINDEX`/`LSB_JOBID`)
   - Mark the superseded statements in the job-array-single-wal-record and xce-test-e2e-primitives specs, update the numbering notes in the controller-job-cleanup and job-dependency briefs, and add a roadmap entry
   - Done when the user documentation has the section with every item 9.4 lists and each referenced spec document carries its note

.kiro/specs/job-array-single-wal-record/requirements.md

@@ -64,6 +64,8 @@ This feature builds on the apply seam introduced by `raft-apply-decoupling` (now
 2. The Controller shall support querying, cancelling, and completing an individual array element by its per-element identity exactly as it does for job elements created via the current per-element path.
 3. This feature shall not change any client-visible identity, status, or addressing behavior for an individual array element.
 
+> **Superseded by job-array-one-based-index (2026-10-01):** the commitment above that element identities stay unchanged for clients, with elements numbered from 0 (indices `[0, size)`), no longer holds. Since `job-array-one-based-index`, a submitted array's elements are numbered 1 through N (`name[1-N]`), each element is shown and addressed as `<array-id>[k]`, and submission reports the array identity rather than an element identity (see that spec's Requirements 1, 3 and 5). The single-WAL-record storage, replication and recovery of arrays specified here are unchanged. Kept for history.
+
 ### Requirement 6: Single push to scheduler and API server, with local expansion
 
 **Objective:** As a maintainer of scheduler and API-server load, I want both the scheduler and the API server to receive one push per committed array, so that push volume to either consumer no longer scales linearly with array size.

.kiro/specs/job-dependency/brief.md

@@ -43,7 +43,11 @@ Codebase exploration, 2026-09-29.
 - **Controller job table.** `c->jobmgr` holds jobs. The scheduler's `core->jobs`
   also keeps every job forever today, which `controller-job-cleanup` changes.
 - **Array jobs.** There is no parent object. Each element is an independent job
-  with jobid `base+i`, so an element can be referenced by its own ID.
+  with its own jobid. Elements are numbered 1..N per `job-array-one-based-index`
+  (2026-10-01) and are referenced as `<array-id>[k]`, k = 1..N; a bare element
+  jobid is rejected by the CLI/REST listing and control commands (use
+  `<array-id>[k]`; peek and restart still take the element's own jobid). The earlier note here,
+  jobid `base+i` with i from 0, described numbering from 0.
 - **Submit paths:** REST (apiserver, fed every job by controller push,
   cgo into libxce), `xcecli jobs submit`, the harness `xce_test jobsub`
   (`harness/test.c:3575`), and libxce directly.

.kiro/specs/xce-test-e2e-primitives/design.md

@@ -75,6 +75,7 @@ graph TD
   - **Session/verdict state** (file-scope): `g_session_failed` (int), `g_continue_on_error` (bool), and auto-vars `g_last_status` (int), `g_last_jobid[XCE_MAXNAME_LEN]` (string), `g_last_jobcount` (int).
   - **`preprocess_and_run(char* line)`** (new): handle blank/`#`; leading `!` → `system(rest)`, set `g_last_status` from `WEXITSTATUS`; else substitute auto-vars then call `execute_line`. Update `g_session_failed`/`g_last_status` from the result; honor abort-vs-continue. Called from the main loop in place of the direct `execute_line`.
   - **`substitute_autovars(line) → expanded`** (new): replace whole-token `$?`, `$jobid`, `$jobcount` with current values; a reference with no current value yields a command failure (Req 3.6).
+    > **Superseded by job-array-one-based-index (2026-10-01):** whole-token-only substitution is relaxed: a token that is `$jobid` followed immediately by `[` (e.g. `$jobid[3]`) now has `$jobid` replaced and the rest of the token copied unchanged, and fails like an unset `$jobid` when it is unset (job-array-one-based-index Requirement 4.10, 4.11). Kept for history.
   - **New command handlers**: `cmd_wait_primary`, `cmd_expect_jobs`, `cmd_expect_job`, `cmd_echo`, registered in the command table(s).
   - **`cmd_set`**: add `continue-on-error` toggle.
   - **`cmd_jobsub`**: gate the "ok" report on a successful reply status + valid JobId; on success set `g_last_jobid`; otherwise print a typed error and return `<0` (bug-786).

.kiro/specs/xce-test-e2e-primitives/requirements.md

@@ -42,6 +42,8 @@ This feature adds a small set of scripting primitives to the `xce_test` client s
 1. When any command completes, the xce_test client shall set `$?` to that command's status (zero on success, non-zero on failure).
 2. When a `jobsub` command receives a successful acknowledgment, the xce_test client shall set `$jobid` to the acknowledged JobId.
 3. When a subsequent command references `$jobid` (e.g. `expect-job $jobid`), the xce_test client shall substitute the most recently acknowledged JobId.
+
+> **Superseded by job-array-one-based-index (2026-10-01):** 3.2/3.3 no longer hold for array submissions. After `jobsub -a N` (N >= 2) the acknowledged identity is the array identity (no element number), `$jobid` is set to it, and elements are named `$jobid[k]`, k = 1..N; `$jobid` followed immediately by `[` is now substituted too (see job-array-one-based-index Requirement 4.5, 4.10, 4.11). Ordinary `jobsub` is unchanged. Kept for history.
 4. Where the `$jobcount` auto-variable is provided, when a `joblist` command completes, the xce_test client shall set `$jobcount` to the number of jobs returned.
 5. The xce_test client shall not provide user-assignable variables, arithmetic, or expression evaluation beyond substituting this fixed set of auto-variables.
 6. If a command references an auto-variable that has no current value, then the xce_test client shall fail that command rather than substitute an undefined or garbage value.
@@ -66,6 +68,8 @@ This feature adds a small set of scripting primitives to the `xce_test` client s
 3. When `expect-job <jobid>` is issued and the given JobId is present in the job list, the xce_test client shall treat the assertion as passed.
 4. If `expect-job <jobid>` is issued and the given JobId is absent, then the xce_test client shall fail the assertion and report the missing JobId.
 
+> **Superseded by job-array-one-based-index (2026-10-01):** for arrays, `expect-job <jobid>` no longer matches text: it compares job numbers, an identity without `[k]` that no job has fails with a hint that elements must be named `<array-id>[k]`, `<array-id>[k]` checks element k, and a range `<array-id>[a-b]` is refused (job-array-one-based-index Requirement 4.7-4.9, 5.13, 5.15). Kept for history.
+
 ### Requirement 6: Script ergonomics
 
 **Objective:** As an E2E script author, I want comments, blank lines, and progress messages, so that scenario scripts are readable and self-documenting.
@@ -81,6 +85,8 @@ This feature adds a small set of scripting primitives to the `xce_test` client s
 
 #### Acceptance Criteria
 1. When `jobsub` receives a successful acknowledgment carrying a valid JobId, the xce_test client shall report success and set `$jobid` to that JobId.
+
+> **Superseded by job-array-one-based-index (2026-10-01):** for `jobsub -a N` (N >= 2) the reported identity and `$jobid` are the array identity, not an element JobId; elements are `$jobid[1]`..`$jobid[N]` (job-array-one-based-index Requirement 4.1, 4.5). Kept for history.
 2. If `jobsub` does not receive a successful acknowledgment with a valid JobId, then the xce_test client shall report a typed failure (e.g. redirect, no-master, not-ready, or timeout) and set a non-zero status.
 3. The xce_test client shall never report `jobsub` success with a malformed or uninitialized JobId.
 

.kiro/steering/roadmap.md

@@ -84,6 +84,7 @@ The protocol **drive** is NOT fully event-driven. Root cause: `peer_ctrl_msg_han
 - [ ] **component-rejoin-resync** — **PROMOTED to spec 2026-08-10** (`/kiro-discovery`; brief at `.kiro/specs/component-rejoin-resync/brief.md`) from the "lightweight rejoin re-attach" item below. **apiserver + scheduler** reconnect/rejoin resync efficiency — today each does a full re-join and receives full state again (whole job list, queue/resource/config state), not a delta; for a large cluster this is sizable but not yet a measured problem. Sibling of `worker-rejoin-resync` — disjoint data (list/config state vs. job-execution facts), NOT merged into that spec. Not started: first step is investigation (is reconnect cost actually a measured problem at realistic cluster sizes?) before committing to any delta/session-token mechanism — original YAGNI discipline still applies, this spec is the promoted vehicle for that investigation, not a rubber-stamped design. Original decision (libxce-failover design review) retained below for provenance. Dependencies: libxce-failover (rejoin hook — implemented), component-failover-adoption (daemon rejoin content — in progress, sequence alongside/after). Next: `/kiro-spec-requirements component-rejoin-resync` when ready to begin (not immediate).
 - [x] ~~lightweight rejoin re-attach (failover optimization)~~ — **recorded 2026-06-16; deferred (YAGNI)**; apiserver/scheduler half **PROMOTED to `component-rejoin-resync` 2026-08-10** (see above); worker half already promoted to `worker-rejoin-resync`. No longer tracked as a bare backlog item — provenance only. Original text: on controller failover a reconnecting daemon does a full re-join and apiserver/scheduler receive **full** state (e.g. the whole job list), not a delta; for a large cluster this is sizable but rare. Decision (libxce-failover design review): accept full-state rejoin — it is simple, correct, and still cheaper than a full-history replace with no snapshot (cf. IBM LSF). Future optimization: a session-token "re-attach" hello that lets the new primary skip re-pushing already-replicated state, or a delta-only state transfer keyed on the consumer's last-known committed LSN. Delta reconciliation is non-trivial (the primary must know what each consumer already holds). Promote only if failover-rejoin latency/bandwidth is **measured** as a problem at scale.
 - [x] job-array-single-wal-record — **IMPLEMENTED, validate-impl GO 2026-07-17** (reopened once 2026-07-16 for three user-directed residual gaps — bug-1443 bulk-insert partial-visibility, task 4.2's boot-replay fence gap, bug-1444 replay_wal array-blind tally — all closed, re-validated fresh GO). **PROMOTED to a spec 2026-06-24** (`/kiro-discovery`; brief at `.kiro/specs/job-array-single-wal-record/brief.md`; now tracked under the *Raft Apply-Ordering & Hook Decoupling* phase above, sequenced after raft-apply-decoupling). Original note retained for provenance: Enhance controller & scheduler so a job-array submission persists, replicates, and pushes to the scheduler as a **single WAL record** rather than one record per array element. Today a job array fans out to N per-job state changes, so a 10k-element array means 10k WAL appends + 10k replicated commits + 10k scheduler pushes — amplifying commit-engine load, replication bandwidth, and failover replay time linearly in array size. Scope sketch: (a) controller writes one WAL entry describing the array (base spec + index range/count + per-index overrides) committed atomically through the commit barrier; (b) replication carries the single record so standbys reconstruct the whole array from one committed LSN; (c) scheduler receives the array as one push and expands locally. Must preserve per-element identity/state addressing after expansion (status, cancel, completion still operate per task id). Dependencies: ha-commit-engine + ha-commit-barrier-rollout (single atomic commit), ha-consistency-invariants (replicated scheduling state). Spec advised before implementation; promote via `/kiro-spec-init "job-array-single-wal-record"` when ready.
+- [ ] **job-array-one-based-index** — **NEW 2026-09-30** (spec `.kiro/specs/job-array-one-based-index/`); **IMPLEMENTED per spec 2026-10-01 (tasks 1.1–9.3 done), validation pending in tasks 10.1 (affected suites) and 10.2 (live cluster)**. LSF-compatible array numbering: `xcecli jobs submit -J 'name[1-N]'` / `'name[1]'` create elements numbered 1..N (previously 0..N-1, and the indices in `-J` were ignored — bug-1896); submit reports the array identity and element count; elements are shown and addressed as `<array-id>[k]`, control commands also take `<array-id>[a-b]`; unsupported LSF forms (other starts, index lists, steps, `%N`, interactive arrays) are rejected with specific messages; API-server `max_array_size` default 1000, clamped to 32767; running elements see `XCE_JOBINDEX` (0 for ordinary jobs); harness `jobsub -a N` numbers 1..N and `$jobid[k]` names elements; the controller rejects multi-element arrays starting at 0 from un-upgraded clients (`E_XCE_ARRAY_START0`). Supersedes `job-array-single-wal-record` Requirement 5 (identities unchanged, numbered from 0) and the `xce-test-e2e-primitives` `$jobid`/`expect-job` definitions (superseded notes added in those specs). User documentation: `src/docs/xce.md` "Job arrays" section, including the LSF features not supported. Bugs: bug-1896 (indices ignored, numbered from 0) fixed by this spec (record still `open` pending `buglog.py status` after validation); bug-1898 `partial` (query + filter-control paths fixed; `RestartJobByID` still builds unevaluated `CompFilterCondition`s, restart out of scope); bug-1899, bug-1900, bug-1901 fixed.
 - [ ] bulk job-array query/cancel — **recorded 2026-07-13, deferred out of `job-array-single-wal-record` scope by design.** A single client operation to query or cancel an entire array (not one element at a time) is explicitly out of scope for `job-array-single-wal-record` — see that spec's Boundary Context. Reason: array elements can be in heterogeneous states at request time (done/exited elements need no action; running elements need a worker kill signal; pending elements need a direct state transition to cancelled without ever running), which is a materially different bulk-operation problem from atomic array *submission* and needs its own requirements (partial-failure semantics, atomicity vs best-effort per element, etc.) rather than being folded into the submission spec. Dependencies: `job-array-single-wal-record` (per-element identity/addressing this would operate over). Promote via `/kiro-discovery` once job-array-single-wal-record ships.
 - [x] ~~WAL group-commit batching~~ — **recorded 2026-06-17 as deferred; ABSORBED INTO ha-commit-durability 2026-06-18 (design-review reversal).** The `/kiro-validate-design` review surfaced that the highest-volume WAL traffic is `WAL_MODE_RETAIN` facts (job start/finish/fail/rusage) — not array-collapsible and not covered by `job-array-single-wal-record` — so a pure per-entry synchronous design would penalize the dominant write path. Decision (with user): **do group-commit now** — repurpose `fsync_worker` from a timer into a demand-driven flusher (batch one `fdatasync` across pending appends, advance `durable_lsn`, wake commit-eval) as part of ha-commit-durability rather than deferring. No longer a separate backlog item.
 - [ ] primary step-down-on-quorum-loss — optional CP follow-on noted in ha-consistency-invariants brief (writes already blocked on quorum loss; step-down is an availability/clarity optimization).

src/docs/xce.md

@@ -54,6 +54,83 @@ cluster. Common operations:
 Built as a standalone Go binary; communicates with the API server over
 its REST interface.
 
+## Job arrays
+
+A job array is a set of identical jobs (elements) submitted at once. XCE
+supports a subset of the LSF array syntax.
+
+### Supported
+
+- **Submitting.** `xcecli jobs submit -J 'name[1-N]' <command>` submits an
+  array of N elements, numbered 1 through N. `-J 'name[1]'` submits a
+  one-element array. Element numbers always start at 1.
+- **Array size limit.** N may be at most the API server's
+  `max_array_size` setting (`config/app.yaml`; default 1000 when unset or 0,
+  never above 32767 — larger values are clamped to 32767 with a startup
+  warning). A larger N is rejected with a message stating the effective limit.
+- **Array name.** The part before `[` is 1 to 4094 characters, each an ASCII
+  letter, digit, `.`, `_` or `-`. No whitespace is allowed anywhere in the
+  expression.
+- **Submit result.** Submission reports the array identity, without an
+  element number, and the number of elements created, e.g.
+  `Job <array-id> is submitted to queue <normal> (N elements).` The REST
+  response carries the same identity and an `array_size` field.
+- **Referring to elements.** Each element is shown and addressed as
+  `<array-id>[k]`, in listings, job detail output, control command results and
+  in log entries that print a job identity in its written form (some log
+  entries still print the raw job number). Element numbers in a selection are
+  at most 32767.
+  - Listing (`xcecli jobs list`) and control commands (`kill`, `stop`,
+    `resume`, `checkpoint`) accept one element, `<array-id>[k]`.
+  - Control commands (`kill`, `stop`, `resume`, ...) also accept a range,
+    `<array-id>[a-b]` (`a` equal to `b` allowed). Queries accept a range only
+    by job name, e.g. `-J 'name[a-b]'`.
+  - `-J 'name[k]'` selects element k of every array named `name`.
+  - An identity without `[...]` selects every element of that array (and any
+    ordinary job with that identity), as before.
+  - In listing and control commands, a bare element job number (for example
+    the value of `XCE_JOBID`) is rejected: name the element as
+    `<array-id>[k]`.
+  - `xcecli jobs peek` and `xcecli jobs restart` do not accept
+    `<array-id>[k]` yet: peek an element by its own job number
+    (`XCE_JOBID`); restart takes job IDs as before.
+- **Element number in the job's environment.** A running element finds its
+  own number in `XCE_JOBINDEX` (1..N). An ordinary job gets
+  `XCE_JOBINDEX=0`. A user-supplied `XCE_JOBINDEX` is overridden, with a
+  warning in the worker log. `XCE_JOBID` is the element's own job number, as
+  before.
+- **Test harness.** `xce_test jobsub -a N` submits elements 1 through N
+  (2 <= N <= 32767, regardless of `max_array_size`; `-a 1` submits an
+  ordinary job). `$jobid` is then the array identity and `$jobid[k]` names
+  element k (for example `expect-job $jobid[3]`).
+
+### LSF array features not supported
+
+Each of these is rejected at submission with a message naming the problem; no
+job or element is created.
+
+- **Starting numbers other than 1**, e.g. `name[0-9]` ("element numbers start
+  at 1") or `name[1001-2000]` / `name[5]` ("only name[1-N] and name[1] are
+  currently supported").
+- **Index lists**, e.g. `name[1-5,10]` or `name[1,3,5]` ("only a single range
+  name[1-N] is supported").
+- **Index steps**, e.g. `name[1-10:2]` (same message).
+- **`%N` running limits**, e.g. `name[1-100]%5` ("running limits are not
+  supported").
+- **Interactive arrays**: an interactive job (`-I`) whose name contains `[`
+  or `]` is rejected ("interactive jobs cannot be arrays").
+
+These LSF features are not available either:
+
+- **Per-element `%I`/`%J` file-name substitution.** `%I` and `%J` in file
+  names (`-i`, `-o`, `-e`, output directory, working directory) are replaced
+  once, at submission, with the first element's values: `%I` becomes `1` and
+  `%J` becomes `<array-id>[1]` for every element. All elements therefore write
+  to the same files unless the job script chooses its own file names from
+  `XCE_JOBINDEX`.
+- **`LSB_JOBINDEX` / `LSB_JOBID` variables** are not set. Scripts read
+  `XCE_JOBINDEX` (element number) and `XCE_JOBID` (job number) instead.
+
 ## shell
 
 An interactive bash shell used for troubleshooting and inspection. Not