docs(spec): application-profile: job starter wraps the full job, task starter wraps every task startup

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-13 03:15:43 +00:00
parent a50c0e236b
commit f2406caaa7
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
2 changed files with 30 additions and 14 deletions

View file

@ -183,10 +183,20 @@ fits XCE's LSF-style job model.
entries), its task range (min / max), and per-job execution limits as
maximums (memory, runtime, CPU, tasks), mapped onto the existing worker
cgroup limits.
5. **Job starter** (decided 2026-10-12): a wrapper the worker puts in
front of the job command (e.g. `module load openfoam &&`,
`apptainer exec img`), frozen on the dispatch record. Design decides
whether it wraps the MPI launcher or each task.
5. **Job starter and task starter** (decided 2026-10-12, user): two
wrappers, both frozen on the dispatch record.
- `job_starter` wraps the **full job**: the job command, started once
on the first host (for an MPI job, the launcher such as `mpirun`),
e.g. `module load openfoam &&`.
- `task_starter` wraps **every task startup** that XCE's workers
perform, on every host including remote tasks (`xce_taskhost_t`),
e.g. `apptainer exec img` or a binding/affinity wrapper.
- A serial (single-task) job's command is both the job and its task:
it is wrapped by both, `job_starter` outermost (design confirms).
- Tasks an MPI runtime starts by itself (its own daemons, not through
XCE's workers) are out of XCE's reach; for those the job starter or
the launch type's arguments apply. Which launch types start tasks
through XCE is settled in design.
6. **Application parameters (`params`)** (decided 2026-10-12): settings
XCE carries but never interprets, handled by the application, e.g.
@ -333,7 +343,8 @@ fits XCE's LSF-style job model.
Cap'n Proto; WAL submit/dispatch facts and snapshot carry them with field
completeness):
- on the submit data: the profile name;
- on `start_info` (what the worker gets): the job starter and the
- on `start_info` (what the worker gets): the job starter, the task
starter (also on each `xce_taskhost_t` entry for remote tasks) and the
profile's env, kept **separate** from the job's env (see "Environment of
the outer layer"); the queue-named `qPreExec` / `qPostExec` carry the
profile's outer pre/post-exec (renamed in design);
@ -403,7 +414,8 @@ fits XCE's LSF-style job model.
tasks: { min: 32, max: 512 }
limits: { runtime: 24h, memory: 128G } # maximums
env: { FOAM_INST_DIR: /opt/openfoam }
job_starter: "module load openfoam &&"
job_starter: "module load openfoam &&"
task_starter: "/opt/xce/app/openfoam/bind.sh"
pre_exec: /opt/xce/app/openfoam/pre.sh
post_exec: /opt/xce/app/openfoam/post.sh
job_pre_post_exec: allow
@ -432,7 +444,7 @@ How each profile attribute combines with the job's own value (owned by
| resreq `span` / `same` / `order` | only if essential (e.g. a code that runs on one host only) | set them if the profile does not; otherwise rejected |
| task range | `min` / `max` | choose within it; no `-n` means `min` |
| env | variables the application needs | add others; not redefine these |
| launch type, mode, sandbox options, job starter | fixed values | nothing |
| launch type, mode, sandbox options, job starter, task starter | fixed values | nothing |
| `job_pre_post_exec` | `allow` / `forbid` | use its own pre/post-exec only if `allow` |
| limits | maximums | ask for less (checked like a ceiling) |
| pre/post-exec | outer layer, always runs | add its own inner layer if the profile allows |
@ -581,7 +593,8 @@ restarted, and the scheduler no longer re-merges it.
`order`) and task count, so a running job's reservations (e.g.
`rusage[license=1]`) do not change when the profile changes, and are
rebuilt identically after a scheduler failover or restart;
- execution: mode, mode options, launch type, job starter, the
- execution: mode, mode options, launch type, job starter, task
starter, the
profile's env and the job's env (separately), outer pre/post-exec, the
`job_pre_post_exec` setting, effective limits;
- the resolved profile name (after aliases) and the content hash the
@ -737,9 +750,11 @@ Decided 2026-10-12 (user), so a rename never strands jobs:
- The profile name and the job's own values persisted on the job.
- The `mode:` schema field (only `direct`), the structured resreq schema
(`select`, `rusage` entries with `per:` / `default`, `span`, `same`,
`order`), the job starter, `params` (`--app-param`) and annotations,
`order`), the job and task starters, `params` (`--app-param`) and
annotations,
exported as `XCE_APP_PARAM_*` / `XCE_APP_ANN_*`.
- New `start_info` fields (job starter, profile env separate from the job
- New `start_info` fields (job starter, task starter, profile env
separate from the job
env) and exdata handling with `job-config-merge`.
- Profile content hash (shared `src/lib` function) and its field on the
scheduler's dispatch decision.
@ -758,8 +773,8 @@ Decided 2026-10-12 (user), so a rename never strands jobs:
- **Open for requirements / design**:
- Multi-host jobs: does the profile's outer pre/post-exec run once (first
host) or on every host (today's queue pre-exec runs in `xcecd_worker`
on each host it starts)? Does the job starter wrap the MPI launcher or
each task? Remote tasks get the profile env and the job env separately.
on each host it starts)? Remote tasks get the profile env and the job
env separately. (Job vs. task starter: decided, item 5 above.)
- Confirm a dispatch declined for a missing or changed profile does not
count as a run or against requeue limits.
- ~~How resource defaults combine with a job's own resreq~~ — settled
@ -772,7 +787,8 @@ Decided 2026-10-12 (user), so a rename never strands jobs:
limits a job may lower (profile limits are maximums), personal presets
(client side), `app:override` and inheritance (deferred); execution modes
split into `job-sandbox-execution`; v1 attributes are the four plus the
job starter, others ship with their job features; profile history and
job starter (plus a task starter for every task startup, decided
2026-10-12), others ship with their job features; profile history and
change audit deferred to the roadmap (`config-history-audit`): the
content hash and the dispatch record already show what each run used;
no job labels needed here (presets record themselves in `exdata.meta`;

View file

@ -41,7 +41,7 @@ One line per part; details below.
| env | the job adds; cannot redefine the profile's |
| `params` | declared by the profile; the job's value wins (`--app-param`); undeclared names rejected; exported as `XCE_APP_PARAM_*` |
| annotations | the profile's; exported as `XCE_APP_ANN_*`; the job cannot change them |
| mode, launch type, job starter, outer pre/post-exec | the profile's |
| mode, launch type, job starter (wraps the full job), task starter (wraps every task startup), outer pre/post-exec | the profile's |
| compound / alternative job resreq | only the profile's `select` applies; a profile `rusage` default or `span`/`same`/`order` → rejected (until PR-15) |
| queue ceilings, project quotas | check the merged result; never change it |