lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs(spec): structured profile resreq; per: optional (default task); merge rules at a glance

Browse source 
- profile resreq split into select / rusage entries / span / same / order;
  rusage entries carry optional per: (default task, today's PER_DEFAULT
  behaviour) and optional default
- job's rusage term wins whole for a resource in both; unit must match;
  no modifier inheritance
- compound/alternative: only the profile select applies; any profile
  rusage default or span/same/order rejects until PR-15
- job-config-layering: merge rules at a glance table; profile = essentials
  plus default reservations

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-13 02:49:53 +00:00
parent 8f3a811a43
commit 1f9aaae050
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
4 changed files with 145 additions and 81 deletions
Download patch file Download diff file Expand all files Collapse all files

99
.kiro/steering/job-config-layering.md
View file

@ -20,7 +20,7 @@ XCE instead gives **every attribute exactly one owning level**.
| Level | Owns | Never owns |
|---|---|---|
| **Queue**: scheduling policy | priority and ordering, host partition, dispatch/run windows, preemption policy, policy ceilings (max run time, max tasks, max hosts, ...) | resource requirement, anything about how a job executes (pre/post-exec, env, launch, limits enforced on the job) |
| **Application profile**: what the application needs and how it runs (essentials only, 2026-10-12) | resreq (select / rusage / span / same / order), task range (min / default / max), execution mode and options, launch type, env, **pre/post-exec**, execution limits (mem, runtime, CPU, tasks) | priority, queue choice, quotas |
| **Application profile**: what the application needs and how it runs (essentials plus default reservations, 2026-10-12) | resreq (select / rusage / span / same / order), task range (min / default / max), execution mode and options, launch type, env, **pre/post-exec**, execution limits (mem, runtime, CPU, tasks) | priority, queue choice, quotas |
| **Project**: resource ownership | membership, quotas / limits, later guarantees, accounting | how a job runs or is ordered |
| **Job** | its own values, only for attributes its profile allows | n/a |
@ -28,6 +28,21 @@ Access to each is separate: queue and profile by RBAC permission, project by
membership (`project-config`). Any allowed queue combines with any allowed
profile and project.
## Merge rules at a glance
One line per part; details below.
| Part | Rule |
|---|---|
| `select` | profile and job ANDed |
| `rusage` | union; the job's term wins for a resource in both; the unit (`per:`, default `task`) is the profile's and must match |
| `span` / `same` / `order` | the profile's if set, otherwise the job's |
| task range | the job chooses within the profile's `min` / `max` |
| limits | the profile's are maximums; the job may lower |
| env | the job adds; cannot redefine the profile's |
| mode, launch type, job starter, outer pre/post-exec | the profile's |
| compound / alternative job resreq | only the profile's `select` applies; a profile `rusage` default or `span`/`same`/`order` → rejected (until PR-15) |
| queue ceilings, project quotas | check the merged result; never change it |
## The only two interactions
1. **Addition: job vs. application profile** (user decision 2026-10-12,
@ -40,7 +55,7 @@ profile and project.
| Profile attribute | The job may |
|---|---|
| resreq `select` (required terms) | add terms (ANDed) |
| resreq `rusage` (per-resource defaults) | add resources; override a profile value (job wins; per-type is the profile's) |
| resreq `rusage` (entries: optional `per:`, `default`) | add resources; override a profile default (job's term wins; unit must match) |
| resreq `span` / `same` / `order` | set them if the profile does not |
| task range `min` / `max` | choose within it; no `-n` means `min` |
| env | add variables; not redefine the profile's |
@ -50,36 +65,57 @@ profile and project.
| `service:` probes, restart policy, ports / replica range / spread, rollout | nothing / choose within / set if the profile does not |
The one exception to "every profile value binds":
**`rusage`: union, the job's value wins** (user decision 2026-10-12). The
effective `rusage[]` is the union of the profile's and the job's resources;
for a resource in both, the job's value wins, in either direction.
- The **per-type** (`/task`, `/job`, `/host`) is the profile's: a job term
without one inherits it (`mem=4000/task` + `mem=16000` gives
`mem=16000/task`); a job term with a *different* per-type is rejected at
submit ("resource mem is per task in profile openfoam"), never silently
reinterpreted (16000/job read as per task would multiply by the task
count).
- Other modifiers: a `duration` / `decay` the job writes replaces the
profile's; one it omits is inherited.
- `name=0` means "reserve none" (the only way to drop a profile resource);
requirements must confirm the scheduler treats a 0 reservation that way.
- Consequence: `rusage` is the one profile section that is a per-resource
**default**, not a binding essential; a profile cannot enforce a minimum
reservation. Hard protection of shared resources comes from `limits.yaml`
and project quotas; later, a "shared/counted" marker on the resource in
`resources.yaml` could forbid lowering below the profile value (licences),
if under-reservation proves a problem.
**Structured profile resreq; `rusage` defaults, the job's term wins** (user
decisions 2026-10-12). A profile's resreq is not one string but separate
parts, so each part's rule is visible from its shape:
```yaml
resreq:
select: "ib && os == rhel9" # binding: ANDed with the job's select
rusage: # defaults: the job may override per resource
- { name: license_foam, per: job, default: 1 }
- { name: mem, default: 4000 } # per: optional, defaults to task
- { name: gpu } # unit only: nothing reserved by default
span: "ptile=32" # binding if set
same: [switch]
order: "r15s"
```
| Part | Merge with the job's `-R` (an ordinary resreq string, parsed into terms) |
|---|---|
| `select` | ANDed with the job's |
| `rusage` entry with `default` | reserved at that value unless the job names the resource; then the job's term wins **whole** (value and `duration`/`decay`), in either direction |
| `rusage` entry without `default` | nothing reserved unless the job asks; it only fixes the unit |
| `per:` (optional, default `task`) | the unit for that resource: a job term without a per-type gets it; a job term with a **different** per-type is rejected at submit ("resource mem is per task in profile openfoam"), never silently reinterpreted (16000/job read as per task multiplies by the task count) |
| a resource only the job names | the job's term as written |
| `span` / `same` / `order` | the profile's if set (binding), otherwise the job's |
- `per:` defaults to `task`, today's behaviour: `PER_DEFAULT` takes the
caller's per-type (`lib/resreq.c:842`) and the controller passes
`PER_TASK` (`controller/job.c:3909`); `resources.yaml` has no per-type
field. A per-resource default there may come later.
- `name=0` in the job means "reserve none" (the only way to drop a profile
default); requirements must confirm the scheduler treats a 0 reservation
that way.
- `xceadmin` validates each `rusage` entry against `resources.yaml` (name
exists) instead of parsing a string; later per-resource fields
(`duration`, a `min:` floor for counted resources such as licences, if
under-reservation proves a problem) slot into the entry without new
grammar.
- Consequence: `rusage` is the one profile part that holds **defaults**; a
profile cannot enforce a minimum reservation. Hard protection of shared
resources comes from `limits.yaml` and project quotas.
- Side effects to handle in requirements: measured resources (memory) are
charged `max(reserved, actual)` once usage is reported
(`resource-accounting.md`), so lowering them mostly self-corrects, but
counted resources (licences) do not; xcecli and `--dry-run` warn when a
job lowers a profile value, and when a preset restates a profile value
(it would pin the job against later profile changes); job details label
each term's origin (profile or job); queue ceilings and project quotas
check the merged value, and the error names the job's override; a
profile `rusage` change changes the content hash, so in-flight dispatches
re-pend and pending jobs re-merge even if they override that resource
(accepted).
job lowers a profile default, and when a preset restates one (it would
pin the job against later profile changes); job details label each
`rusage` term as profile default or job override; queue ceilings and
project quotas check the merged value and the error names the job's
override; a profile `rusage` change changes the content hash, so
in-flight dispatches re-pend and pending jobs re-merge even if they
override that resource (accepted).
2. **Ceiling: queue or project vs. the job's effective values.** A queue
ceiling or project quota never changes the job's values, it only checks
@ -94,9 +130,10 @@ ordering of queue against profile settings, no compound/alternative merge
stages.
## Convenience defaults are presets, not a level
Tuning that most jobs of an application want but none must have
(`rusage[mem=...]`, `span[ptile=...]`, a task count, extra env) is not in
the profile. It goes into **presets**, handled fully by the client
Apart from default reservations (the profile's `rusage` entries), tuning
that most jobs of an application want but none must have
(`span[ptile=...]`, a task count, extra env, different `rusage` values) is
not in the profile. It goes into **presets**, handled fully by the client
(`xcecli-presets`): a preset expands into job options before submit, so the
server only ever sees the job's own values. Presets are therefore not a
configuration level, own no attribute and carry no privilege. The apiserver