docs(spec): application-profile params and annotations; 0-valued rusage dropped; app-submit-hook backlog
- params: declared application settings XCE carries but never interprets; exported as XCE_APP_PARAM_*; job overrides with --app-param; undeclared names rejected; stored in exdata app_params - annotations: metadata for tools, also exported as XCE_APP_ANN_* (name conversion, collision check, size limits, no secrets) - merge drops 0-valued rusage terms so the scheduler sees no term (today a 0 term still rejects hosts where the resource is over-committed) - app-submit-hook (esub-style) recorded as a roadmap backlog item only Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
f658b0a4b0
commit
a50c0e236b
GPG key ID:
80533536B0EB651C
6 changed files with 80 additions and 14 deletions
|
|
@ -98,6 +98,12 @@ consumed by `service-supervision` (worker probes and restarts),
|
|||
essential); nothing tunes itself. Backlog: `profile-usage-insight`.
|
||||
- Job details show the effective resreq with each term's origin (profile
|
||||
or job).
|
||||
- Deferred: a submit-time hook run for the application (LSF `esub`
|
||||
style) that inspects or adjusts each submission; roadmap backlog
|
||||
`app-submit-hook` (it cannot run in the controller, would add latency
|
||||
and a failure mode to every apiserver submit, and bypasses the merge
|
||||
rules; `params`, annotations, the job starter and pre-exec cover most
|
||||
needs).
|
||||
- Deferred: a privileged `app:override` for one-off exceptions (until
|
||||
`job-modify` exists, an admin kills and resubmits, or uses a variant
|
||||
profile; recurring variants are presets), and profile inheritance
|
||||
|
|
@ -181,8 +187,49 @@ fits XCE's LSF-style job model.
|
|||
front of the job command (e.g. `module load openfoam &&`,
|
||||
`apptainer exec img`), frozen on the dispatch record. Design decides
|
||||
whether it wraps the MPI launcher or each task.
|
||||
6. **Application parameters (`params`)** (decided 2026-10-12): settings
|
||||
XCE carries but never interprets, handled by the application, e.g.
|
||||
|
||||
```yaml
|
||||
params:
|
||||
- { name: solver, default: pimpleFoam, allowed: [pimpleFoam, simpleFoam] }
|
||||
- { name: decompose, default: scotch }
|
||||
- { name: case_dir, pattern: "^/shared/cases/.*" } # no default
|
||||
```
|
||||
- Each is delivered to the job as `XCE_APP_PARAM_<NAME>`, with the
|
||||
profile env (separate from the job env, frozen on the dispatch
|
||||
record), so the job, the job starter and the outer pre/post-exec can
|
||||
read it.
|
||||
- The job overrides with `--app-param name=value` (also in presets);
|
||||
the job's value wins, like `rusage` defaults. Only declared names are
|
||||
accepted (a typo is rejected at submit). XCE checks only the shape
|
||||
(`allowed` list or `pattern`), never the meaning. A parameter with no
|
||||
default and no job value is not set.
|
||||
- The job's values are stored in its exdata (`app_params`), already
|
||||
persisted; no new job field.
|
||||
- The portal can build an application's submit form from the declared
|
||||
parameters.
|
||||
7. **Annotations** (decided 2026-10-12): free-form key/value facts about
|
||||
the application (owner, version, docs URL, licence server name), like
|
||||
Kubernetes annotations. Shown to tools (profile listing, portal, job
|
||||
details) **and** exported to the job as `XCE_APP_ANN_<KEY>`, so an
|
||||
admin writes a value once for both. The job cannot change them.
|
||||
- Name conversion: keys may be Kubernetes-style
|
||||
(`cfd.org/solver-family`); the env name is uppercased with every
|
||||
non-alphanumeric character replaced by `_`
|
||||
(`XCE_APP_ANN_CFD_ORG_SOLVER_FAMILY`). `xceadmin` rejects two
|
||||
annotation keys (or two params) that convert to the same name.
|
||||
- Limits on count and size, so a profile cannot bloat every job's env.
|
||||
- Visible to every `app:view` holder: no secrets (the secret-name
|
||||
warning applies). All are exported in v1; an opt-out only if
|
||||
needed.
|
||||
- `env` vs annotations vs params: `env` is a variable the application
|
||||
expects under its own name; annotations are facts for tools and the
|
||||
job; params are the user's choices. `XCE_APP_*` names are set by XCE
|
||||
only; the reserved `XCE_` prefix keeps jobs and profiles from setting
|
||||
them directly, so a job cannot fake one.
|
||||
All of these are essentials under the additive rule, except the `rusage`
|
||||
entries, which are default reservations the job may override.
|
||||
entries and `params`, which are defaults the job may override.
|
||||
- **A profile attribute ships with its job feature** (user, 2026-10-12): a
|
||||
profile attribute is added by the spec that builds the job-level feature
|
||||
it configures, never ahead of it, so no profile setting is dead config.
|
||||
|
|
@ -419,9 +466,20 @@ resreq:
|
|||
caller's per-type (`lib/resreq.c:842`) and the controller passes
|
||||
`PER_TASK` (`controller/job.c:3909`); `resources.yaml` has no per-type
|
||||
field. A per-resource default there may come later.
|
||||
- `name=0` in the job means "reserve none" (the only way to drop a profile
|
||||
default); requirements must confirm the scheduler treats a 0 reservation
|
||||
that way.
|
||||
- **A 0-valued term is removed by the merge** (decided 2026-10-12, user):
|
||||
after combining profile and job, every `rusage` term whose value is 0 is
|
||||
dropped, so the scheduler sees exactly "no term". `name=0` is how a job
|
||||
drops a profile default. Needed because the scheduler does not treat 0 as
|
||||
absent today: a 0 term still becomes a wanted entry
|
||||
(`calc_wanted_host_resvals()`, `lib/resreq.c`), and `unsatisfy_()`
|
||||
(`scheduler/src/resource.c:164`, `total < inuse + uncomit + wanted`)
|
||||
rejects a host where that resource is already over-committed
|
||||
(`inuse > total`, possible because memory is charged
|
||||
`max(reserved, actual)`), and the term shows in decision-trace rows and
|
||||
pend reasons; with no term the resource is not checked at all (fast path
|
||||
at `resource.c:351`). Applies to every 0 term (profile, job, preset, or a
|
||||
job without a profile). Job details still show "license: 0 (overrides the
|
||||
profile default)".
|
||||
- `xceadmin` validates each `rusage` entry against `resources.yaml` (name
|
||||
exists) instead of parsing a string; later per-resource fields
|
||||
(`duration`, a `min:` floor for counted resources such as licences, if
|
||||
|
|
@ -679,7 +737,8 @@ Decided 2026-10-12 (user), so a rename never strands jobs:
|
|||
- The profile name and the job's own values persisted on the job.
|
||||
- The `mode:` schema field (only `direct`), the structured resreq schema
|
||||
(`select`, `rusage` entries with `per:` / `default`, `span`, `same`,
|
||||
`order`) and the job starter.
|
||||
`order`), the job starter, `params` (`--app-param`) and annotations,
|
||||
exported as `XCE_APP_PARAM_*` / `XCE_APP_ANN_*`.
|
||||
- New `start_info` fields (job starter, profile env separate from the job
|
||||
env) and exdata handling with `job-config-merge`.
|
||||
- Profile content hash (shared `src/lib` function) and its field on the
|
||||
|
|
@ -701,7 +760,6 @@ Decided 2026-10-12 (user), so a rename never strands jobs:
|
|||
host) or on every host (today's queue pre-exec runs in `xcecd_worker`
|
||||
on each host it starts)? Does the job starter wrap the MPI launcher or
|
||||
each task? Remote tasks get the profile env and the job env separately.
|
||||
- Confirm a `rusage` value of `0` means "reserve none" in the scheduler.
|
||||
- Confirm a dispatch declined for a missing or changed profile does not
|
||||
count as a run or against requeue limits.
|
||||
- ~~How resource defaults combine with a job's own resreq~~ — settled
|
||||
|
|
@ -721,7 +779,9 @@ Decided 2026-10-12 (user), so a rename never strands jobs:
|
|||
general labels are the `job-labels` backlog item); compound and
|
||||
alternative resreqs (only the profile `select` applies); structured
|
||||
resreq with `per:` default `task`; separate profile env; empty profile
|
||||
name = no profile; a file profile named `default` replaces the built-in.
|
||||
name = no profile; a file profile named `default` replaces the built-in;
|
||||
0-valued `rusage` terms removed by the merge; `params` and annotations
|
||||
(exported to env) in v1; submit hook deferred (`app-submit-hook`).
|
||||
- **Out**:
|
||||
- The `sandbox` mode and its options, I/O directories, capability
|
||||
advertisement, placement by mode and the worker sandbox mechanics
|
||||
|
|
|
|||
|
|
@ -55,7 +55,9 @@ apiserver and display would disagree about what a job actually asked for.
|
|||
profile value binds, except `rusage`: union of profile and job
|
||||
resources, the job's term wins whole for a resource in both, the profile
|
||||
entry's `per:` (optional, default `task`) fixes the unit (an explicitly
|
||||
different per-type is rejected), `0` means none; profile resreq is
|
||||
different per-type is rejected), 0-valued terms are dropped after the
|
||||
merge so the scheduler sees no term; `params` declared by the profile,
|
||||
the job's value wins, undeclared names rejected; profile resreq is
|
||||
structured (`select`, `rusage` entries, `span`, `same`, `order`), see the
|
||||
at-a-glance table in `job-config-layering.md`; otherwise the job may add (more `select` terms, `span`/`same`/`order` the profile does not
|
||||
set, env variables, lower limits, an inner pre/post-exec if allowed)
|
||||
|
|
|
|||
|
|
@ -14,7 +14,7 @@ Discovery brief: `.kiro/specs/job-config-merge/brief.md`. Governing rules: `.kir
|
|||
**What should change:**
|
||||
- One merge function in `src/lib`, shared by controller, scheduler and (through cgo) apiserver. Its inputs are the job's own values, the application profile, the queue's policy and the project; its output is the job's effective values: resreq (select / rusage / span / same / order), task range, execution limits, pre/post-exec, env.
|
||||
- **One owner per attribute.** The queue owns scheduling policy, the application profile owns what the application needs and how it runs, and the project owns resource ownership. Only two interactions exist:
|
||||
- **Addition** (2026-10-12, replacing *default*/*fixed* markers): a profile holds only essentials and every profile value binds, except `rusage`: the union of profile and job resources, the job's term winning whole for a resource in both, with the unit from the profile entry's optional `per:` (default `task`; an explicitly different per-type is rejected) and `0` meaning none. The profile resreq is structured (`select`, `rusage` entries with `per:`/`default`, `span`, `same`, `order`); rules at a glance: `job-config-layering.md`. Otherwise the job may add to them (more `select` terms, `span`/`same`/`order` the profile does not set, env variables, lower limits) but never change or remove them; a conflict is rejected at submit. Convenience defaults are client-side presets and never reach the merge.
|
||||
- **Addition** (2026-10-12, replacing *default*/*fixed* markers): a profile holds only essentials and every profile value binds, except `rusage`: the union of profile and job resources, the job's term winning whole for a resource in both, with the unit from the profile entry's optional `per:` (default `task`; an explicitly different per-type is rejected) and 0-valued terms dropped after the merge (the scheduler then sees no term; today a 0 term still keeps a job off hosts where the resource is over-committed). Profile `params`: declared names only, the job's value wins. The profile resreq is structured (`select`, `rusage` entries with `per:`/`default`, `span`, `same`, `order`); rules at a glance: `job-config-layering.md`. Otherwise the job may add to them (more `select` terms, `span`/`same`/`order` the profile does not set, env variables, lower limits) but never change or remove them; a conflict is rejected at submit. Convenience defaults are client-side presets and never reach the merge.
|
||||
- **Ceiling:** queue ceilings and project quotas never change a value. A value that can never be satisfied is rejected at submit; a quota that is only full right now makes the job pend with a reason naming the queue or project.
|
||||
- **Live merge.** The WAL keeps only the job's own values plus its queue, profile and project names.
|
||||
- The controller (authoritative) and the apiserver (early) run the merge at submit to validate.
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ configuration level.
|
|||
|
||||
## Desired Outcome
|
||||
- **A preset is a set of job options** (`--app`, `--queue`, `-R`, `-n`,
|
||||
env, limits, ...), named and stored by the client. `xcecli jobs submit
|
||||
env, limits, `--app-param`, ...), named and stored by the client. `xcecli jobs submit
|
||||
--preset <name>` expands it; any command-line flag replaces the preset's
|
||||
value for that option as a whole (no resreq parsing). The submit carries
|
||||
only the expanded values.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue