lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs(spec): structured profile resreq; per: optional (default task); merge rules at a glance

Browse source 
- profile resreq split into select / rusage entries / span / same / order;
  rusage entries carry optional per: (default task, today's PER_DEFAULT
  behaviour) and optional default
- job's rusage term wins whole for a resource in both; unit must match;
  no modifier inheritance
- compound/alternative: only the profile select applies; any profile
  rusage default or span/same/order rejects until PR-15
- job-config-layering: merge rules at a glance table; profile = essentials
  plus default reservations

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-13 02:49:53 +00:00
parent 8f3a811a43
commit 1f9aaae050
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
4 changed files with 145 additions and 81 deletions
Download patch file Download diff file Expand all files Collapse all files

117
.kiro/specs/application-profile/brief.md
View file

@ -78,8 +78,12 @@ consumed by `service-supervision` (worker probes and restarts),
- **Additive rule:** the job may add to the profile's values, never change
or remove them. A conflict is rejected at submit with an error naming the
profile value, never resolved silently.
- **Convenience defaults** (tuning such as `rusage[mem=...]`,
`span[ptile=...]`, task count, extra env) go into **presets**, handled
- **Exception: default reservations.** The profile's structured
`rusage` entries hold per-resource defaults the job may override (see
"Essentials and the additive rule"), so REST/portal users without
presets still reserve sensibly.
- **Other convenience defaults** (tuning such as `span[ptile=...]`, task
count, extra env, different `rusage` values) go into **presets**, handled
fully by the client (`xcecli-presets`). The server only shares
admin-deployed presets read-only; presets never reach the controller,
WAL or scheduler and carry no privilege.
@ -316,7 +320,11 @@ fits XCE's LSF-style job model.
description: "OpenFOAM, MPI"
mode: direct
launch_type: openmpi
resreq: "select[ib] rusage[license_foam=1]" # essentials only
resreq:
select: "ib" # binding
rusage:
- { name: license_foam, per: job, default: 1 } # default reservation
- { name: mem, default: 4000 } # per task by default
tasks: { min: 32, max: 512 }
limits: { runtime: 24h, memory: 128G } # maximums
env: { FOAM_INST_DIR: /opt/openfoam }
@ -330,8 +338,9 @@ fits XCE's LSF-style job model.
job_pre_post_exec: forbid
```
Tuning such as `rusage[mem=4000] span[ptile=32]` and a task count of 64
is not in the profile; the admin publishes it as a site preset named
Default reservations live in the profile's `rusage` entries (the job may
override them); other tuning such as `span[ptile=32]` and a task count of
64 is not in the profile: the admin publishes it as a site preset named
`openfoam`.
### Essentials and the additive rule
@ -341,7 +350,7 @@ How each profile attribute combines with the job's own value (owned by
| Profile attribute | Holds | The job may |
|---|---|---|
| resreq `select` | required terms (`ib`, GPU model, OS) | add terms (ANDed) |
| resreq `rusage` | per-resource defaults (`license=1`, `mem=4000/task`) | add resources; override a profile resource's value (job wins, see below) |
| resreq `rusage` | entries with optional `per:` and `default` | add resources; override a profile default (job's term wins; unit must match, see below) |
| resreq `span` / `same` / `order` | only if essential (e.g. a code that runs on one host only) | set them if the profile does not; otherwise rejected |
| task range | `min` / `max` | choose within it; no `-n` means `min` |
| env | variables the application needs | add others; not redefine these |
@ -349,36 +358,57 @@ How each profile attribute combines with the job's own value (owned by
| limits | maximums | ask for less (checked like a ceiling) |
| pre/post-exec | outer layer, always runs | add its own inner layer if the profile allows |
**`rusage`: union, the job's value wins** (user decision 2026-10-12). The
effective `rusage[]` is the union of the profile's and the job's resources;
for a resource in both, the job's value wins, in either direction.
- The **per-type** (`/task`, `/job`, `/host`) is the profile's: a job term
without one inherits it (`mem=4000/task` + `mem=16000` gives
`mem=16000/task`); a job term with a *different* per-type is rejected at
submit ("resource mem is per task in profile openfoam"), never silently
reinterpreted (16000/job read as per task would multiply by the task
count).
- Other modifiers: a `duration` / `decay` the job writes replaces the
profile's; one it omits is inherited.
- `name=0` means "reserve none" (the only way to drop a profile resource);
requirements must confirm the scheduler treats a 0 reservation that way.
- Consequence: `rusage` is the one profile section that is a per-resource
**default**, not a binding essential; a profile cannot enforce a minimum
reservation. Hard protection of shared resources comes from `limits.yaml`
and project quotas; later, a "shared/counted" marker on the resource in
`resources.yaml` could forbid lowering below the profile value (licences),
if under-reservation proves a problem.
**Structured profile resreq; `rusage` defaults, the job's term wins** (user
decisions 2026-10-12). A profile's resreq is not one string but separate
parts, so each part's rule is visible from its shape:
```yaml
resreq:
select: "ib && os == rhel9" # binding: ANDed with the job's select
rusage: # defaults: the job may override per resource
- { name: license_foam, per: job, default: 1 }
- { name: mem, default: 4000 } # per: optional, defaults to task
- { name: gpu } # unit only: nothing reserved by default
span: "ptile=32" # binding if set
same: [switch]
order: "r15s"
```
| Part | Merge with the job's `-R` (an ordinary resreq string, parsed into terms) |
|---|---|
| `select` | ANDed with the job's |
| `rusage` entry with `default` | reserved at that value unless the job names the resource; then the job's term wins **whole** (value and `duration`/`decay`), in either direction |
| `rusage` entry without `default` | nothing reserved unless the job asks; it only fixes the unit |
| `per:` (optional, default `task`) | the unit for that resource: a job term without a per-type gets it; a job term with a **different** per-type is rejected at submit ("resource mem is per task in profile openfoam"), never silently reinterpreted (16000/job read as per task multiplies by the task count) |
| a resource only the job names | the job's term as written |
| `span` / `same` / `order` | the profile's if set (binding), otherwise the job's |
- `per:` defaults to `task`, today's behaviour: `PER_DEFAULT` takes the
caller's per-type (`lib/resreq.c:842`) and the controller passes
`PER_TASK` (`controller/job.c:3909`); `resources.yaml` has no per-type
field. A per-resource default there may come later.
- `name=0` in the job means "reserve none" (the only way to drop a profile
default); requirements must confirm the scheduler treats a 0 reservation
that way.
- `xceadmin` validates each `rusage` entry against `resources.yaml` (name
exists) instead of parsing a string; later per-resource fields
(`duration`, a `min:` floor for counted resources such as licences, if
under-reservation proves a problem) slot into the entry without new
grammar.
- Consequence: `rusage` is the one profile part that holds **defaults**; a
profile cannot enforce a minimum reservation. Hard protection of shared
resources comes from `limits.yaml` and project quotas.
- Side effects to handle in requirements: measured resources (memory) are
charged `max(reserved, actual)` once usage is reported
(`resource-accounting.md`), so lowering them mostly self-corrects, but
counted resources (licences) do not; xcecli and `--dry-run` warn when a
job lowers a profile value, and when a preset restates a profile value
(it would pin the job against later profile changes); job details label
each term's origin (profile or job); queue ceilings and project quotas
check the merged value, and the error names the job's override; a
profile `rusage` change changes the content hash, so in-flight dispatches
re-pend and pending jobs re-merge even if they override that resource
(accepted).
job lowers a profile default, and when a preset restates one (it would
pin the job against later profile changes); job details label each
`rusage` term as profile default or job override; queue ceilings and
project quotas check the merged value and the error names the job's
override; a profile `rusage` change changes the content hash, so
in-flight dispatches re-pend and pending jobs re-merge even if they
override that resource (accepted).
- A job with no `-R` gets the profile's essentials only; the scheduler's
existing empty-resreq default (`type == any`) still applies to what
neither sets.
@ -395,20 +425,15 @@ for a resource in both, the job's value wins, in either direction.
profile sets them they bind; otherwise the service sets them.
- Other later sections (I/O claims, relationship templates, `container:`)
state in their own specs how they follow the additive rule.
- **Compound and alternative resreqs** (decided 2026-10-12, user). Parts
are unnamed today, so a profile term can apply only where its meaning per
part is clear:
- the profile's `select` is ANDed into every part (and every `||`
alternative): a host property the application needs applies to every
host it runs on;
- a profile `rusage` resource that is **per task** is merged into every
part with the `rusage` rule above;
- a profile `rusage` resource that is per job, per host or `PER_DEFAULT`,
and a profile `span` / `same` / `order`, make a compound or alternative
job resreq **rejected** with an error naming the section, until PR-15
named parts let a profile target a part by role;
- admin guidance (not enforceable): no part-specific property (e.g. a GPU
model only one role needs) in a profile's `select`.
- **Compound and alternative resreqs** (decided 2026-10-12, user,
simplified). Parts are unnamed today: only the profile's `select` applies,
ANDed into every part and every `||` alternative. If the profile has any
`rusage` entry with a `default`, or sets `span` / `same` / `order`, a
compound or alternative job resreq is **rejected** with an error naming
the part, until PR-15 named parts let a profile target a part by role.
`rusage` entries without a default only fix a unit and do not block.
Admin guidance (not enforceable): no part-specific property (e.g. a GPU
model only one role needs) in a profile's `select`.
### Environment of the outer layer and reserved names
Decided 2026-10-12 (user), replacing the idea of an env denylist: