docs(spec): application-profile hooks and starters; kill-path requirement; proposals 2-5

- worker-run hooks: pre/post_job_exec (once, first host) and
  pre/post_host_exec (once per host); no per-task hooks
- task starter reaches MPI ranks through the mpirun line XCE builds
  (correction); XCE_JOB_HOST_DIR, XCE_HOST_NTASKS, XCE_HOST_TASK_INDEX
- hook failure: job EXIT with a reason naming the hook; post hooks still run
- post hooks must survive every way a job ends: run limit is systemd
  RuntimeMaxUSec stopping the unit (worker included) and post-exec is only
  bounded by the unset TimeoutStopUSec; requirement for an explicit stop
  timeout and OOM survival (not logged as a bug: post-exec does run today)
- default profile always usable with queue:submit; service profile
  rejected for batch jobs; warning when no host matches the profile select

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-13 03:28:10 +00:00
parent f2406caaa7
commit 863ba98ac8
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
2 changed files with 99 additions and 17 deletions

View file

@ -41,7 +41,7 @@ One line per part; details below.
| env | the job adds; cannot redefine the profile's |
| `params` | declared by the profile; the job's value wins (`--app-param`); undeclared names rejected; exported as `XCE_APP_PARAM_*` |
| annotations | the profile's; exported as `XCE_APP_ANN_*`; the job cannot change them |
| mode, launch type, job starter (wraps the full job), task starter (wraps every task startup), outer pre/post-exec | the profile's |
| mode, launch type, job starter (wraps the full job), task starter (wraps every task), hooks `pre/post_job_exec` (first host) and `pre/post_host_exec` (every host), run by the worker | the profile's |
| compound / alternative job resreq | only the profile's `select` applies; a profile `rusage` default or `span`/`same`/`order` → rejected (until PR-15) |
| queue ceilings, project quotas | check the merged result; never change it |
@ -148,12 +148,17 @@ preset.
- Moves from the queue to the application profile. Queue `pre_exec` /
`post_exec` are deprecated (`xceadmin config build` warns, then rejects).
- Two layers remain, each with one owner:
1. **Profile pre/post-exec**: admin-defined, always runs (fixed); the job
cannot replace or skip it.
1. **Profile hooks**: admin-defined, always run (fixed), run by the worker
in a clean environment; the job cannot replace or skip them. Since
2026-10-12 they are `pre/post_job_exec` (once, first host) and
`pre/post_host_exec` (once per host); details and kill-path
requirements in the `application-profile` brief, "Hooks and starters".
2. **Job pre/post-exec**: the user's own (`start_info` today); allowed or
forbidden by the profile (e.g. a sandbox profile may forbid it).
- Order: profile pre → job pre → job → job post → profile post. A failed
profile pre-exec skips the rest; profile post-exec still runs for cleanup.
- Order: profile `pre_job_exec` → job pre → job → job post → profile
`post_job_exec` on the first host; `pre_host_exec` / `post_host_exec`
around each host's tasks. A failed profile pre hook skips the rest; the
matching post hook still runs for cleanup.
- The worker already runs exactly two layers (`queue_pre_exec()` /
`queue_post_exec()`, `worker/src/xcecd_worker.c`; stages
`JOB_EXEC_STAGE_Q_PRE_EXEC` / `Q_POST_EXEC`), so the worker keeps its