lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs(spec): rusage merge: union, job value wins, profile per-type; compound rule

Browse source 
- rusage: union of profile and job resources; for a resource in both the
  job's value wins (either direction); the profile's per-type applies and an
  explicitly different per-type is rejected; omitted modifiers inherited;
  name=0 means none
- rusage is the one profile section that is a per-resource default;
  shared-resource protection via limits.yaml/quotas, later a counted marker
- side effects recorded for requirements (lowering warnings, redundant
  preset values, origin labels, ceiling errors, hash churn)
- compound/alternative: profile select ANDed into every part, per-task
  rusage merged per part, other profile resreq sections reject until PR-15

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-12 14:49:10 +00:00
parent a68d2fc965
commit 8f3a811a43
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
4 changed files with 84 additions and 8 deletions
Download patch file Download diff file Expand all files Collapse all files

49
.kiro/specs/application-profile/brief.md
View file

@ -341,7 +341,7 @@ How each profile attribute combines with the job's own value (owned by
| Profile attribute | Holds | The job may |
|---|---|---|
| resreq `select` | required terms (`ib`, GPU model, OS) | add terms (ANDed) |
| resreq `rusage` | required consumptions (`license=1`) | add other resources; not change these |
| resreq `rusage` | per-resource defaults (`license=1`, `mem=4000/task`) | add resources; override a profile resource's value (job wins, see below) |
| resreq `span` / `same` / `order` | only if essential (e.g. a code that runs on one host only) | set them if the profile does not; otherwise rejected |
| task range | `min` / `max` | choose within it; no `-n` means `min` |
| env | variables the application needs | add others; not redefine these |
@ -349,6 +349,36 @@ How each profile attribute combines with the job's own value (owned by
| limits | maximums | ask for less (checked like a ceiling) |
| pre/post-exec | outer layer, always runs | add its own inner layer if the profile allows |
**`rusage`: union, the job's value wins** (user decision 2026-10-12). The
effective `rusage[]` is the union of the profile's and the job's resources;
for a resource in both, the job's value wins, in either direction.
- The **per-type** (`/task`, `/job`, `/host`) is the profile's: a job term
without one inherits it (`mem=4000/task` + `mem=16000` gives
`mem=16000/task`); a job term with a *different* per-type is rejected at
submit ("resource mem is per task in profile openfoam"), never silently
reinterpreted (16000/job read as per task would multiply by the task
count).
- Other modifiers: a `duration` / `decay` the job writes replaces the
profile's; one it omits is inherited.
- `name=0` means "reserve none" (the only way to drop a profile resource);
requirements must confirm the scheduler treats a 0 reservation that way.
- Consequence: `rusage` is the one profile section that is a per-resource
**default**, not a binding essential; a profile cannot enforce a minimum
reservation. Hard protection of shared resources comes from `limits.yaml`
and project quotas; later, a "shared/counted" marker on the resource in
`resources.yaml` could forbid lowering below the profile value (licences),
if under-reservation proves a problem.
- Side effects to handle in requirements: measured resources (memory) are
charged `max(reserved, actual)` once usage is reported
(`resource-accounting.md`), so lowering them mostly self-corrects, but
counted resources (licences) do not; xcecli and `--dry-run` warn when a
job lowers a profile value, and when a preset restates a profile value
(it would pin the job against later profile changes); job details label
each term's origin (profile or job); queue ceilings and project quotas
check the merged value, and the error names the job's override; a
profile `rusage` change changes the content hash, so in-flight dispatches
re-pend and pending jobs re-merge even if they override that resource
(accepted).
- A job with no `-R` gets the profile's essentials only; the scheduler's
existing empty-resreq default (`type == any`) still applies to what
neither sets.
@ -365,9 +395,20 @@ How each profile attribute combines with the job's own value (owned by
profile sets them they bind; otherwise the service sets them.
- Other later sections (I/O claims, relationship templates, `container:`)
state in their own specs how they follow the additive rule.
- Compound or alternative resreqs: a compound job resreq under a profile
whose resreq sets terms is rejected until PR-15 named parts define
per-part merging (leaning; requirements decide).
- **Compound and alternative resreqs** (decided 2026-10-12, user). Parts
are unnamed today, so a profile term can apply only where its meaning per
part is clear:
- the profile's `select` is ANDed into every part (and every `||`
alternative): a host property the application needs applies to every
host it runs on;
- a profile `rusage` resource that is **per task** is merged into every
part with the `rusage` rule above;
- a profile `rusage` resource that is per job, per host or `PER_DEFAULT`,
and a profile `span` / `same` / `order`, make a compound or alternative
job resreq **rejected** with an error naming the section, until PR-15
named parts let a profile target a part by role;
- admin guidance (not enforceable): no part-specific property (e.g. a GPU
model only one role needs) in a profile's `select`.
### Environment of the outer layer and reserved names
Decided 2026-10-12 (user), replacing the idea of an env denylist:

6
.kiro/specs/job-config-merge/brief.md
View file

@ -52,8 +52,10 @@ apiserver and display would disagree about what a job actually asked for.
ownership), so only two interactions exist:
- **Addition (job vs. profile)** (user decision 2026-10-12, replacing the
*default*/*fixed* markers): a profile holds only essentials and every
profile value binds; the job may add to them (more `select` terms,
other `rusage` resources, `span`/`same`/`order` the profile does not
profile value binds, except `rusage`: union of profile and job
resources, the job's value wins for a resource in both, the profile's
per-type applies (an explicitly different per-type is rejected), `0`
means none; otherwise the job may add (more `select` terms, `span`/`same`/`order` the profile does not
set, env variables, lower limits, an inner pre/post-exec if allowed)
but never change or remove them; conflicts are rejected at submit.
Table: `job-config-layering.md`. This replaces LSF's per-section merge

2
.kiro/specs/job-config-merge/requirements.md
View file

@ -14,7 +14,7 @@ Discovery brief: `.kiro/specs/job-config-merge/brief.md`. Governing rules: `.kir
**What should change:**
- One merge function in `src/lib`, shared by controller, scheduler and (through cgo) apiserver. Its inputs are the job's own values, the application profile, the queue's policy and the project; its output is the job's effective values: resreq (select / rusage / span / same / order), task range, execution limits, pre/post-exec, env.
- **One owner per attribute.** The queue owns scheduling policy, the application profile owns what the application needs and how it runs, and the project owns resource ownership. Only two interactions exist:
- **Addition** (2026-10-12, replacing *default*/*fixed* markers): a profile holds only essentials and every profile value binds. The job may add to them (more `select` terms, other `rusage` resources, `span`/`same`/`order` the profile does not set, env variables, lower limits) but never change or remove them; a conflict is rejected at submit. Convenience defaults are client-side presets and never reach the merge.
- **Addition** (2026-10-12, replacing *default*/*fixed* markers): a profile holds only essentials and every profile value binds, except `rusage`: the union of profile and job resources, the job's value winning for a resource in both, with the profile's per-type (`/task`, `/job`, `/host`; an explicitly different one is rejected) and `0` meaning none. Otherwise the job may add to them (more `select` terms, `span`/`same`/`order` the profile does not set, env variables, lower limits) but never change or remove them; a conflict is rejected at submit. Convenience defaults are client-side presets and never reach the merge.
- **Ceiling:** queue ceilings and project quotas never change a value. A value that can never be satisfied is rejected at submit; a quota that is only full right now makes the job pend with a reason naming the queue or project.
- **Live merge.** The WAL keeps only the job's own values plus its queue, profile and project names.
- The controller (authoritative) and the apiserver (early) run the merge at submit to validate.

35
.kiro/steering/job-config-layering.md
View file

@ -40,7 +40,7 @@ profile and project.
| Profile attribute | The job may |
|---|---|
| resreq `select` (required terms) | add terms (ANDed) |
| resreq `rusage` (required consumptions) | add other resources; not change these |
| resreq `rusage` (per-resource defaults) | add resources; override a profile value (job wins; per-type is the profile's) |
| resreq `span` / `same` / `order` | set them if the profile does not |
| task range `min` / `max` | choose within it; no `-n` means `min` |
| env | add variables; not redefine the profile's |
@ -48,6 +48,39 @@ profile and project.
| limits (maximums) | ask for less |
| pre/post-exec (outer layer) | add an inner layer if the profile allows |
| `service:` probes, restart policy, ports / replica range / spread, rollout | nothing / choose within / set if the profile does not |
The one exception to "every profile value binds":
**`rusage`: union, the job's value wins** (user decision 2026-10-12). The
effective `rusage[]` is the union of the profile's and the job's resources;
for a resource in both, the job's value wins, in either direction.
- The **per-type** (`/task`, `/job`, `/host`) is the profile's: a job term
without one inherits it (`mem=4000/task` + `mem=16000` gives
`mem=16000/task`); a job term with a *different* per-type is rejected at
submit ("resource mem is per task in profile openfoam"), never silently
reinterpreted (16000/job read as per task would multiply by the task
count).
- Other modifiers: a `duration` / `decay` the job writes replaces the
profile's; one it omits is inherited.
- `name=0` means "reserve none" (the only way to drop a profile resource);
requirements must confirm the scheduler treats a 0 reservation that way.
- Consequence: `rusage` is the one profile section that is a per-resource
**default**, not a binding essential; a profile cannot enforce a minimum
reservation. Hard protection of shared resources comes from `limits.yaml`
and project quotas; later, a "shared/counted" marker on the resource in
`resources.yaml` could forbid lowering below the profile value (licences),
if under-reservation proves a problem.
- Side effects to handle in requirements: measured resources (memory) are
charged `max(reserved, actual)` once usage is reported
(`resource-accounting.md`), so lowering them mostly self-corrects, but
counted resources (licences) do not; xcecli and `--dry-run` warn when a
job lowers a profile value, and when a preset restates a profile value
(it would pin the job against later profile changes); job details label
each term's origin (profile or job); queue ceilings and project quotas
check the merged value, and the error names the job's override; a
profile `rusage` change changes the content hash, so in-flight dispatches
re-pend and pending jobs re-merge even if they override that resource
(accepted).
2. **Ceiling: queue or project vs. the job's effective values.** A queue
ceiling or project quota never changes the job's values, it only checks
them: