docs(job-array-one-based-index): document the supported array syntax (task 9.3)
src/docs/xce.md gains a 'Job arrays' section: name[1-N] and name[1], numbering from 1, the API-server limit (default 1000, at most 32767), <array-id>[k] and <array-id>[a-b] selection (with the peek/restart caveat), XCE_JOBINDEX, the harness jobsub -a N / $jobid[k], and every LSF array feature this release does not support. The superseded statements in job-array-single-wal-record and xce-test-e2e-primitives are annotated (history kept), the controller-job-cleanup and job-dependency briefs describe 1-based numbering, and the roadmap gains an entry. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
24dafa712f
commit
6439e5852f
8 changed files with 98 additions and 3 deletions
|
|
@ -54,6 +54,83 @@ cluster. Common operations:
|
|||
Built as a standalone Go binary; communicates with the API server over
|
||||
its REST interface.
|
||||
|
||||
## Job arrays
|
||||
|
||||
A job array is a set of identical jobs (elements) submitted at once. XCE
|
||||
supports a subset of the LSF array syntax.
|
||||
|
||||
### Supported
|
||||
|
||||
- **Submitting.** `xcecli jobs submit -J 'name[1-N]' <command>` submits an
|
||||
array of N elements, numbered 1 through N. `-J 'name[1]'` submits a
|
||||
one-element array. Element numbers always start at 1.
|
||||
- **Array size limit.** N may be at most the API server's
|
||||
`max_array_size` setting (`config/app.yaml`; default 1000 when unset or 0,
|
||||
never above 32767 — larger values are clamped to 32767 with a startup
|
||||
warning). A larger N is rejected with a message stating the effective limit.
|
||||
- **Array name.** The part before `[` is 1 to 4094 characters, each an ASCII
|
||||
letter, digit, `.`, `_` or `-`. No whitespace is allowed anywhere in the
|
||||
expression.
|
||||
- **Submit result.** Submission reports the array identity, without an
|
||||
element number, and the number of elements created, e.g.
|
||||
`Job <array-id> is submitted to queue <normal> (N elements).` The REST
|
||||
response carries the same identity and an `array_size` field.
|
||||
- **Referring to elements.** Each element is shown and addressed as
|
||||
`<array-id>[k]`, in listings, job detail output, control command results and
|
||||
in log entries that print a job identity in its written form (some log
|
||||
entries still print the raw job number). Element numbers in a selection are
|
||||
at most 32767.
|
||||
- Listing (`xcecli jobs list`) and control commands (`kill`, `stop`,
|
||||
`resume`, `checkpoint`) accept one element, `<array-id>[k]`.
|
||||
- Control commands (`kill`, `stop`, `resume`, ...) also accept a range,
|
||||
`<array-id>[a-b]` (`a` equal to `b` allowed). Queries accept a range only
|
||||
by job name, e.g. `-J 'name[a-b]'`.
|
||||
- `-J 'name[k]'` selects element k of every array named `name`.
|
||||
- An identity without `[...]` selects every element of that array (and any
|
||||
ordinary job with that identity), as before.
|
||||
- In listing and control commands, a bare element job number (for example
|
||||
the value of `XCE_JOBID`) is rejected: name the element as
|
||||
`<array-id>[k]`.
|
||||
- `xcecli jobs peek` and `xcecli jobs restart` do not accept
|
||||
`<array-id>[k]` yet: peek an element by its own job number
|
||||
(`XCE_JOBID`); restart takes job IDs as before.
|
||||
- **Element number in the job's environment.** A running element finds its
|
||||
own number in `XCE_JOBINDEX` (1..N). An ordinary job gets
|
||||
`XCE_JOBINDEX=0`. A user-supplied `XCE_JOBINDEX` is overridden, with a
|
||||
warning in the worker log. `XCE_JOBID` is the element's own job number, as
|
||||
before.
|
||||
- **Test harness.** `xce_test jobsub -a N` submits elements 1 through N
|
||||
(2 <= N <= 32767, regardless of `max_array_size`; `-a 1` submits an
|
||||
ordinary job). `$jobid` is then the array identity and `$jobid[k]` names
|
||||
element k (for example `expect-job $jobid[3]`).
|
||||
|
||||
### LSF array features not supported
|
||||
|
||||
Each of these is rejected at submission with a message naming the problem; no
|
||||
job or element is created.
|
||||
|
||||
- **Starting numbers other than 1**, e.g. `name[0-9]` ("element numbers start
|
||||
at 1") or `name[1001-2000]` / `name[5]` ("only name[1-N] and name[1] are
|
||||
currently supported").
|
||||
- **Index lists**, e.g. `name[1-5,10]` or `name[1,3,5]` ("only a single range
|
||||
name[1-N] is supported").
|
||||
- **Index steps**, e.g. `name[1-10:2]` (same message).
|
||||
- **`%N` running limits**, e.g. `name[1-100]%5` ("running limits are not
|
||||
supported").
|
||||
- **Interactive arrays**: an interactive job (`-I`) whose name contains `[`
|
||||
or `]` is rejected ("interactive jobs cannot be arrays").
|
||||
|
||||
These LSF features are not available either:
|
||||
|
||||
- **Per-element `%I`/`%J` file-name substitution.** `%I` and `%J` in file
|
||||
names (`-i`, `-o`, `-e`, output directory, working directory) are replaced
|
||||
once, at submission, with the first element's values: `%I` becomes `1` and
|
||||
`%J` becomes `<array-id>[1]` for every element. All elements therefore write
|
||||
to the same files unless the job script chooses its own file names from
|
||||
`XCE_JOBINDEX`.
|
||||
- **`LSB_JOBINDEX` / `LSB_JOBID` variables** are not set. Scripts read
|
||||
`XCE_JOBINDEX` (element number) and `XCE_JOBID` (job number) instead.
|
||||
|
||||
## shell
|
||||
|
||||
An interactive bash shell used for troubleshooting and inspection. Not
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue