docs(spec): application-profile: job starter wraps the full job, task starter wraps every task startup
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
a50c0e236b
commit
f2406caaa7
2 changed files with 30 additions and 14 deletions
|
|
@ -183,10 +183,20 @@ fits XCE's LSF-style job model.
|
||||||
entries), its task range (min / max), and per-job execution limits as
|
entries), its task range (min / max), and per-job execution limits as
|
||||||
maximums (memory, runtime, CPU, tasks), mapped onto the existing worker
|
maximums (memory, runtime, CPU, tasks), mapped onto the existing worker
|
||||||
cgroup limits.
|
cgroup limits.
|
||||||
5. **Job starter** (decided 2026-10-12): a wrapper the worker puts in
|
5. **Job starter and task starter** (decided 2026-10-12, user): two
|
||||||
front of the job command (e.g. `module load openfoam &&`,
|
wrappers, both frozen on the dispatch record.
|
||||||
`apptainer exec img`), frozen on the dispatch record. Design decides
|
- `job_starter` wraps the **full job**: the job command, started once
|
||||||
whether it wraps the MPI launcher or each task.
|
on the first host (for an MPI job, the launcher such as `mpirun`),
|
||||||
|
e.g. `module load openfoam &&`.
|
||||||
|
- `task_starter` wraps **every task startup** that XCE's workers
|
||||||
|
perform, on every host including remote tasks (`xce_taskhost_t`),
|
||||||
|
e.g. `apptainer exec img` or a binding/affinity wrapper.
|
||||||
|
- A serial (single-task) job's command is both the job and its task:
|
||||||
|
it is wrapped by both, `job_starter` outermost (design confirms).
|
||||||
|
- Tasks an MPI runtime starts by itself (its own daemons, not through
|
||||||
|
XCE's workers) are out of XCE's reach; for those the job starter or
|
||||||
|
the launch type's arguments apply. Which launch types start tasks
|
||||||
|
through XCE is settled in design.
|
||||||
6. **Application parameters (`params`)** (decided 2026-10-12): settings
|
6. **Application parameters (`params`)** (decided 2026-10-12): settings
|
||||||
XCE carries but never interprets, handled by the application, e.g.
|
XCE carries but never interprets, handled by the application, e.g.
|
||||||
|
|
||||||
|
|
@ -333,7 +343,8 @@ fits XCE's LSF-style job model.
|
||||||
Cap'n Proto; WAL submit/dispatch facts and snapshot carry them with field
|
Cap'n Proto; WAL submit/dispatch facts and snapshot carry them with field
|
||||||
completeness):
|
completeness):
|
||||||
- on the submit data: the profile name;
|
- on the submit data: the profile name;
|
||||||
- on `start_info` (what the worker gets): the job starter and the
|
- on `start_info` (what the worker gets): the job starter, the task
|
||||||
|
starter (also on each `xce_taskhost_t` entry for remote tasks) and the
|
||||||
profile's env, kept **separate** from the job's env (see "Environment of
|
profile's env, kept **separate** from the job's env (see "Environment of
|
||||||
the outer layer"); the queue-named `qPreExec` / `qPostExec` carry the
|
the outer layer"); the queue-named `qPreExec` / `qPostExec` carry the
|
||||||
profile's outer pre/post-exec (renamed in design);
|
profile's outer pre/post-exec (renamed in design);
|
||||||
|
|
@ -403,7 +414,8 @@ fits XCE's LSF-style job model.
|
||||||
tasks: { min: 32, max: 512 }
|
tasks: { min: 32, max: 512 }
|
||||||
limits: { runtime: 24h, memory: 128G } # maximums
|
limits: { runtime: 24h, memory: 128G } # maximums
|
||||||
env: { FOAM_INST_DIR: /opt/openfoam }
|
env: { FOAM_INST_DIR: /opt/openfoam }
|
||||||
job_starter: "module load openfoam &&"
|
job_starter: "module load openfoam &&"
|
||||||
|
task_starter: "/opt/xce/app/openfoam/bind.sh"
|
||||||
pre_exec: /opt/xce/app/openfoam/pre.sh
|
pre_exec: /opt/xce/app/openfoam/pre.sh
|
||||||
post_exec: /opt/xce/app/openfoam/post.sh
|
post_exec: /opt/xce/app/openfoam/post.sh
|
||||||
job_pre_post_exec: allow
|
job_pre_post_exec: allow
|
||||||
|
|
@ -432,7 +444,7 @@ How each profile attribute combines with the job's own value (owned by
|
||||||
| resreq `span` / `same` / `order` | only if essential (e.g. a code that runs on one host only) | set them if the profile does not; otherwise rejected |
|
| resreq `span` / `same` / `order` | only if essential (e.g. a code that runs on one host only) | set them if the profile does not; otherwise rejected |
|
||||||
| task range | `min` / `max` | choose within it; no `-n` means `min` |
|
| task range | `min` / `max` | choose within it; no `-n` means `min` |
|
||||||
| env | variables the application needs | add others; not redefine these |
|
| env | variables the application needs | add others; not redefine these |
|
||||||
| launch type, mode, sandbox options, job starter | fixed values | nothing |
|
| launch type, mode, sandbox options, job starter, task starter | fixed values | nothing |
|
||||||
| `job_pre_post_exec` | `allow` / `forbid` | use its own pre/post-exec only if `allow` |
|
| `job_pre_post_exec` | `allow` / `forbid` | use its own pre/post-exec only if `allow` |
|
||||||
| limits | maximums | ask for less (checked like a ceiling) |
|
| limits | maximums | ask for less (checked like a ceiling) |
|
||||||
| pre/post-exec | outer layer, always runs | add its own inner layer if the profile allows |
|
| pre/post-exec | outer layer, always runs | add its own inner layer if the profile allows |
|
||||||
|
|
@ -581,7 +593,8 @@ restarted, and the scheduler no longer re-merges it.
|
||||||
`order`) and task count, so a running job's reservations (e.g.
|
`order`) and task count, so a running job's reservations (e.g.
|
||||||
`rusage[license=1]`) do not change when the profile changes, and are
|
`rusage[license=1]`) do not change when the profile changes, and are
|
||||||
rebuilt identically after a scheduler failover or restart;
|
rebuilt identically after a scheduler failover or restart;
|
||||||
- execution: mode, mode options, launch type, job starter, the
|
- execution: mode, mode options, launch type, job starter, task
|
||||||
|
starter, the
|
||||||
profile's env and the job's env (separately), outer pre/post-exec, the
|
profile's env and the job's env (separately), outer pre/post-exec, the
|
||||||
`job_pre_post_exec` setting, effective limits;
|
`job_pre_post_exec` setting, effective limits;
|
||||||
- the resolved profile name (after aliases) and the content hash the
|
- the resolved profile name (after aliases) and the content hash the
|
||||||
|
|
@ -737,9 +750,11 @@ Decided 2026-10-12 (user), so a rename never strands jobs:
|
||||||
- The profile name and the job's own values persisted on the job.
|
- The profile name and the job's own values persisted on the job.
|
||||||
- The `mode:` schema field (only `direct`), the structured resreq schema
|
- The `mode:` schema field (only `direct`), the structured resreq schema
|
||||||
(`select`, `rusage` entries with `per:` / `default`, `span`, `same`,
|
(`select`, `rusage` entries with `per:` / `default`, `span`, `same`,
|
||||||
`order`), the job starter, `params` (`--app-param`) and annotations,
|
`order`), the job and task starters, `params` (`--app-param`) and
|
||||||
|
annotations,
|
||||||
exported as `XCE_APP_PARAM_*` / `XCE_APP_ANN_*`.
|
exported as `XCE_APP_PARAM_*` / `XCE_APP_ANN_*`.
|
||||||
- New `start_info` fields (job starter, profile env separate from the job
|
- New `start_info` fields (job starter, task starter, profile env
|
||||||
|
separate from the job
|
||||||
env) and exdata handling with `job-config-merge`.
|
env) and exdata handling with `job-config-merge`.
|
||||||
- Profile content hash (shared `src/lib` function) and its field on the
|
- Profile content hash (shared `src/lib` function) and its field on the
|
||||||
scheduler's dispatch decision.
|
scheduler's dispatch decision.
|
||||||
|
|
@ -758,8 +773,8 @@ Decided 2026-10-12 (user), so a rename never strands jobs:
|
||||||
- **Open for requirements / design**:
|
- **Open for requirements / design**:
|
||||||
- Multi-host jobs: does the profile's outer pre/post-exec run once (first
|
- Multi-host jobs: does the profile's outer pre/post-exec run once (first
|
||||||
host) or on every host (today's queue pre-exec runs in `xcecd_worker`
|
host) or on every host (today's queue pre-exec runs in `xcecd_worker`
|
||||||
on each host it starts)? Does the job starter wrap the MPI launcher or
|
on each host it starts)? Remote tasks get the profile env and the job
|
||||||
each task? Remote tasks get the profile env and the job env separately.
|
env separately. (Job vs. task starter: decided, item 5 above.)
|
||||||
- Confirm a dispatch declined for a missing or changed profile does not
|
- Confirm a dispatch declined for a missing or changed profile does not
|
||||||
count as a run or against requeue limits.
|
count as a run or against requeue limits.
|
||||||
- ~~How resource defaults combine with a job's own resreq~~ — settled
|
- ~~How resource defaults combine with a job's own resreq~~ — settled
|
||||||
|
|
@ -772,7 +787,8 @@ Decided 2026-10-12 (user), so a rename never strands jobs:
|
||||||
limits a job may lower (profile limits are maximums), personal presets
|
limits a job may lower (profile limits are maximums), personal presets
|
||||||
(client side), `app:override` and inheritance (deferred); execution modes
|
(client side), `app:override` and inheritance (deferred); execution modes
|
||||||
split into `job-sandbox-execution`; v1 attributes are the four plus the
|
split into `job-sandbox-execution`; v1 attributes are the four plus the
|
||||||
job starter, others ship with their job features; profile history and
|
job starter (plus a task starter for every task startup, decided
|
||||||
|
2026-10-12), others ship with their job features; profile history and
|
||||||
change audit deferred to the roadmap (`config-history-audit`): the
|
change audit deferred to the roadmap (`config-history-audit`): the
|
||||||
content hash and the dispatch record already show what each run used;
|
content hash and the dispatch record already show what each run used;
|
||||||
no job labels needed here (presets record themselves in `exdata.meta`;
|
no job labels needed here (presets record themselves in `exdata.meta`;
|
||||||
|
|
|
||||||
|
|
@ -41,7 +41,7 @@ One line per part; details below.
|
||||||
| env | the job adds; cannot redefine the profile's |
|
| env | the job adds; cannot redefine the profile's |
|
||||||
| `params` | declared by the profile; the job's value wins (`--app-param`); undeclared names rejected; exported as `XCE_APP_PARAM_*` |
|
| `params` | declared by the profile; the job's value wins (`--app-param`); undeclared names rejected; exported as `XCE_APP_PARAM_*` |
|
||||||
| annotations | the profile's; exported as `XCE_APP_ANN_*`; the job cannot change them |
|
| annotations | the profile's; exported as `XCE_APP_ANN_*`; the job cannot change them |
|
||||||
| mode, launch type, job starter, outer pre/post-exec | the profile's |
|
| mode, launch type, job starter (wraps the full job), task starter (wraps every task startup), outer pre/post-exec | the profile's |
|
||||||
| compound / alternative job resreq | only the profile's `select` applies; a profile `rusage` default or `span`/`same`/`order` → rejected (until PR-15) |
|
| compound / alternative job resreq | only the profile's `select` applies; a profile `rusage` default or `span`/`same`/`order` → rejected (until PR-15) |
|
||||||
| queue ceilings, project quotas | check the merged result; never change it |
|
| queue ceilings, project quotas | check the merged result; never change it |
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue