docs(spec): application-profile params and annotations; 0-valued rusage dropped; app-submit-hook backlog

- params: declared application settings XCE carries but never interprets;
  exported as XCE_APP_PARAM_*; job overrides with --app-param; undeclared
  names rejected; stored in exdata app_params
- annotations: metadata for tools, also exported as XCE_APP_ANN_* (name
  conversion, collision check, size limits, no secrets)
- merge drops 0-valued rusage terms so the scheduler sees no term (today a
  0 term still rejects hosts where the resource is over-committed)
- app-submit-hook (esub-style) recorded as a roadmap backlog item only

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-13 03:04:32 +00:00
parent f658b0a4b0
commit a50c0e236b
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
6 changed files with 80 additions and 14 deletions

View file

@ -98,6 +98,12 @@ consumed by `service-supervision` (worker probes and restarts),
essential); nothing tunes itself. Backlog: `profile-usage-insight`.
- Job details show the effective resreq with each term's origin (profile
or job).
- Deferred: a submit-time hook run for the application (LSF `esub`
style) that inspects or adjusts each submission; roadmap backlog
`app-submit-hook` (it cannot run in the controller, would add latency
and a failure mode to every apiserver submit, and bypasses the merge
rules; `params`, annotations, the job starter and pre-exec cover most
needs).
- Deferred: a privileged `app:override` for one-off exceptions (until
`job-modify` exists, an admin kills and resubmits, or uses a variant
profile; recurring variants are presets), and profile inheritance
@ -181,8 +187,49 @@ fits XCE's LSF-style job model.
front of the job command (e.g. `module load openfoam &&`,
`apptainer exec img`), frozen on the dispatch record. Design decides
whether it wraps the MPI launcher or each task.
6. **Application parameters (`params`)** (decided 2026-10-12): settings
XCE carries but never interprets, handled by the application, e.g.
```yaml
params:
- { name: solver, default: pimpleFoam, allowed: [pimpleFoam, simpleFoam] }
- { name: decompose, default: scotch }
- { name: case_dir, pattern: "^/shared/cases/.*" } # no default
```
- Each is delivered to the job as `XCE_APP_PARAM_<NAME>`, with the
profile env (separate from the job env, frozen on the dispatch
record), so the job, the job starter and the outer pre/post-exec can
read it.
- The job overrides with `--app-param name=value` (also in presets);
the job's value wins, like `rusage` defaults. Only declared names are
accepted (a typo is rejected at submit). XCE checks only the shape
(`allowed` list or `pattern`), never the meaning. A parameter with no
default and no job value is not set.
- The job's values are stored in its exdata (`app_params`), already
persisted; no new job field.
- The portal can build an application's submit form from the declared
parameters.
7. **Annotations** (decided 2026-10-12): free-form key/value facts about
the application (owner, version, docs URL, licence server name), like
Kubernetes annotations. Shown to tools (profile listing, portal, job
details) **and** exported to the job as `XCE_APP_ANN_<KEY>`, so an
admin writes a value once for both. The job cannot change them.
- Name conversion: keys may be Kubernetes-style
(`cfd.org/solver-family`); the env name is uppercased with every
non-alphanumeric character replaced by `_`
(`XCE_APP_ANN_CFD_ORG_SOLVER_FAMILY`). `xceadmin` rejects two
annotation keys (or two params) that convert to the same name.
- Limits on count and size, so a profile cannot bloat every job's env.
- Visible to every `app:view` holder: no secrets (the secret-name
warning applies). All are exported in v1; an opt-out only if
needed.
- `env` vs annotations vs params: `env` is a variable the application
expects under its own name; annotations are facts for tools and the
job; params are the user's choices. `XCE_APP_*` names are set by XCE
only; the reserved `XCE_` prefix keeps jobs and profiles from setting
them directly, so a job cannot fake one.
All of these are essentials under the additive rule, except the `rusage`
entries, which are default reservations the job may override.
entries and `params`, which are defaults the job may override.
- **A profile attribute ships with its job feature** (user, 2026-10-12): a
profile attribute is added by the spec that builds the job-level feature
it configures, never ahead of it, so no profile setting is dead config.
@ -419,9 +466,20 @@ resreq:
caller's per-type (`lib/resreq.c:842`) and the controller passes
`PER_TASK` (`controller/job.c:3909`); `resources.yaml` has no per-type
field. A per-resource default there may come later.
- `name=0` in the job means "reserve none" (the only way to drop a profile
default); requirements must confirm the scheduler treats a 0 reservation
that way.
- **A 0-valued term is removed by the merge** (decided 2026-10-12, user):
after combining profile and job, every `rusage` term whose value is 0 is
dropped, so the scheduler sees exactly "no term". `name=0` is how a job
drops a profile default. Needed because the scheduler does not treat 0 as
absent today: a 0 term still becomes a wanted entry
(`calc_wanted_host_resvals()`, `lib/resreq.c`), and `unsatisfy_()`
(`scheduler/src/resource.c:164`, `total < inuse + uncomit + wanted`)
rejects a host where that resource is already over-committed
(`inuse > total`, possible because memory is charged
`max(reserved, actual)`), and the term shows in decision-trace rows and
pend reasons; with no term the resource is not checked at all (fast path
at `resource.c:351`). Applies to every 0 term (profile, job, preset, or a
job without a profile). Job details still show "license: 0 (overrides the
profile default)".
- `xceadmin` validates each `rusage` entry against `resources.yaml` (name
exists) instead of parsing a string; later per-resource fields
(`duration`, a `min:` floor for counted resources such as licences, if
@ -679,7 +737,8 @@ Decided 2026-10-12 (user), so a rename never strands jobs:
- The profile name and the job's own values persisted on the job.
- The `mode:` schema field (only `direct`), the structured resreq schema
(`select`, `rusage` entries with `per:` / `default`, `span`, `same`,
`order`) and the job starter.
`order`), the job starter, `params` (`--app-param`) and annotations,
exported as `XCE_APP_PARAM_*` / `XCE_APP_ANN_*`.
- New `start_info` fields (job starter, profile env separate from the job
env) and exdata handling with `job-config-merge`.
- Profile content hash (shared `src/lib` function) and its field on the
@ -701,7 +760,6 @@ Decided 2026-10-12 (user), so a rename never strands jobs:
host) or on every host (today's queue pre-exec runs in `xcecd_worker`
on each host it starts)? Does the job starter wrap the MPI launcher or
each task? Remote tasks get the profile env and the job env separately.
- Confirm a `rusage` value of `0` means "reserve none" in the scheduler.
- Confirm a dispatch declined for a missing or changed profile does not
count as a run or against requeue limits.
- ~~How resource defaults combine with a job's own resreq~~ — settled
@ -721,7 +779,9 @@ Decided 2026-10-12 (user), so a rename never strands jobs:
general labels are the `job-labels` backlog item); compound and
alternative resreqs (only the profile `select` applies); structured
resreq with `per:` default `task`; separate profile env; empty profile
name = no profile; a file profile named `default` replaces the built-in.
name = no profile; a file profile named `default` replaces the built-in;
0-valued `rusage` terms removed by the merge; `params` and annotations
(exported to env) in v1; submit hook deferred (`app-submit-hook`).
- **Out**:
- The `sandbox` mode and its options, I/O directories, capability
advertisement, placement by mode and the worker sandbox mechanics