diff --git a/.kiro/specs/application-profile/brief.md b/.kiro/specs/application-profile/brief.md index f13a96dd..8c6df0cf 100644 --- a/.kiro/specs/application-profile/brief.md +++ b/.kiro/specs/application-profile/brief.md @@ -341,7 +341,7 @@ How each profile attribute combines with the job's own value (owned by | Profile attribute | Holds | The job may | |---|---|---| | resreq `select` | required terms (`ib`, GPU model, OS) | add terms (ANDed) | -| resreq `rusage` | required consumptions (`license=1`) | add other resources; not change these | +| resreq `rusage` | per-resource defaults (`license=1`, `mem=4000/task`) | add resources; override a profile resource's value (job wins, see below) | | resreq `span` / `same` / `order` | only if essential (e.g. a code that runs on one host only) | set them if the profile does not; otherwise rejected | | task range | `min` / `max` | choose within it; no `-n` means `min` | | env | variables the application needs | add others; not redefine these | @@ -349,6 +349,36 @@ How each profile attribute combines with the job's own value (owned by | limits | maximums | ask for less (checked like a ceiling) | | pre/post-exec | outer layer, always runs | add its own inner layer if the profile allows | +**`rusage`: union, the job's value wins** (user decision 2026-10-12). The +effective `rusage[]` is the union of the profile's and the job's resources; +for a resource in both, the job's value wins, in either direction. +- The **per-type** (`/task`, `/job`, `/host`) is the profile's: a job term + without one inherits it (`mem=4000/task` + `mem=16000` gives + `mem=16000/task`); a job term with a *different* per-type is rejected at + submit ("resource mem is per task in profile openfoam"), never silently + reinterpreted (16000/job read as per task would multiply by the task + count). +- Other modifiers: a `duration` / `decay` the job writes replaces the + profile's; one it omits is inherited. +- `name=0` means "reserve none" (the only way to drop a profile resource); + requirements must confirm the scheduler treats a 0 reservation that way. +- Consequence: `rusage` is the one profile section that is a per-resource + **default**, not a binding essential; a profile cannot enforce a minimum + reservation. Hard protection of shared resources comes from `limits.yaml` + and project quotas; later, a "shared/counted" marker on the resource in + `resources.yaml` could forbid lowering below the profile value (licences), + if under-reservation proves a problem. +- Side effects to handle in requirements: measured resources (memory) are + charged `max(reserved, actual)` once usage is reported + (`resource-accounting.md`), so lowering them mostly self-corrects, but + counted resources (licences) do not; xcecli and `--dry-run` warn when a + job lowers a profile value, and when a preset restates a profile value + (it would pin the job against later profile changes); job details label + each term's origin (profile or job); queue ceilings and project quotas + check the merged value, and the error names the job's override; a + profile `rusage` change changes the content hash, so in-flight dispatches + re-pend and pending jobs re-merge even if they override that resource + (accepted). - A job with no `-R` gets the profile's essentials only; the scheduler's existing empty-resreq default (`type == any`) still applies to what neither sets. @@ -365,9 +395,20 @@ How each profile attribute combines with the job's own value (owned by profile sets them they bind; otherwise the service sets them. - Other later sections (I/O claims, relationship templates, `container:`) state in their own specs how they follow the additive rule. -- Compound or alternative resreqs: a compound job resreq under a profile - whose resreq sets terms is rejected until PR-15 named parts define - per-part merging (leaning; requirements decide). +- **Compound and alternative resreqs** (decided 2026-10-12, user). Parts + are unnamed today, so a profile term can apply only where its meaning per + part is clear: + - the profile's `select` is ANDed into every part (and every `||` + alternative): a host property the application needs applies to every + host it runs on; + - a profile `rusage` resource that is **per task** is merged into every + part with the `rusage` rule above; + - a profile `rusage` resource that is per job, per host or `PER_DEFAULT`, + and a profile `span` / `same` / `order`, make a compound or alternative + job resreq **rejected** with an error naming the section, until PR-15 + named parts let a profile target a part by role; + - admin guidance (not enforceable): no part-specific property (e.g. a GPU + model only one role needs) in a profile's `select`. ### Environment of the outer layer and reserved names Decided 2026-10-12 (user), replacing the idea of an env denylist: diff --git a/.kiro/specs/job-config-merge/brief.md b/.kiro/specs/job-config-merge/brief.md index 846e86a9..37b4b441 100644 --- a/.kiro/specs/job-config-merge/brief.md +++ b/.kiro/specs/job-config-merge/brief.md @@ -52,8 +52,10 @@ apiserver and display would disagree about what a job actually asked for. ownership), so only two interactions exist: - **Addition (job vs. profile)** (user decision 2026-10-12, replacing the *default*/*fixed* markers): a profile holds only essentials and every - profile value binds; the job may add to them (more `select` terms, - other `rusage` resources, `span`/`same`/`order` the profile does not + profile value binds, except `rusage`: union of profile and job + resources, the job's value wins for a resource in both, the profile's + per-type applies (an explicitly different per-type is rejected), `0` + means none; otherwise the job may add (more `select` terms, `span`/`same`/`order` the profile does not set, env variables, lower limits, an inner pre/post-exec if allowed) but never change or remove them; conflicts are rejected at submit. Table: `job-config-layering.md`. This replaces LSF's per-section merge diff --git a/.kiro/specs/job-config-merge/requirements.md b/.kiro/specs/job-config-merge/requirements.md index ef58cbdb..5db371bf 100644 --- a/.kiro/specs/job-config-merge/requirements.md +++ b/.kiro/specs/job-config-merge/requirements.md @@ -14,7 +14,7 @@ Discovery brief: `.kiro/specs/job-config-merge/brief.md`. Governing rules: `.kir **What should change:** - One merge function in `src/lib`, shared by controller, scheduler and (through cgo) apiserver. Its inputs are the job's own values, the application profile, the queue's policy and the project; its output is the job's effective values: resreq (select / rusage / span / same / order), task range, execution limits, pre/post-exec, env. - **One owner per attribute.** The queue owns scheduling policy, the application profile owns what the application needs and how it runs, and the project owns resource ownership. Only two interactions exist: - - **Addition** (2026-10-12, replacing *default*/*fixed* markers): a profile holds only essentials and every profile value binds. The job may add to them (more `select` terms, other `rusage` resources, `span`/`same`/`order` the profile does not set, env variables, lower limits) but never change or remove them; a conflict is rejected at submit. Convenience defaults are client-side presets and never reach the merge. + - **Addition** (2026-10-12, replacing *default*/*fixed* markers): a profile holds only essentials and every profile value binds, except `rusage`: the union of profile and job resources, the job's value winning for a resource in both, with the profile's per-type (`/task`, `/job`, `/host`; an explicitly different one is rejected) and `0` meaning none. Otherwise the job may add to them (more `select` terms, `span`/`same`/`order` the profile does not set, env variables, lower limits) but never change or remove them; a conflict is rejected at submit. Convenience defaults are client-side presets and never reach the merge. - **Ceiling:** queue ceilings and project quotas never change a value. A value that can never be satisfied is rejected at submit; a quota that is only full right now makes the job pend with a reason naming the queue or project. - **Live merge.** The WAL keeps only the job's own values plus its queue, profile and project names. - The controller (authoritative) and the apiserver (early) run the merge at submit to validate. diff --git a/.kiro/steering/job-config-layering.md b/.kiro/steering/job-config-layering.md index b2a4d30e..036ba14c 100644 --- a/.kiro/steering/job-config-layering.md +++ b/.kiro/steering/job-config-layering.md @@ -40,7 +40,7 @@ profile and project. | Profile attribute | The job may | |---|---| | resreq `select` (required terms) | add terms (ANDed) | - | resreq `rusage` (required consumptions) | add other resources; not change these | + | resreq `rusage` (per-resource defaults) | add resources; override a profile value (job wins; per-type is the profile's) | | resreq `span` / `same` / `order` | set them if the profile does not | | task range `min` / `max` | choose within it; no `-n` means `min` | | env | add variables; not redefine the profile's | @@ -48,6 +48,39 @@ profile and project. | limits (maximums) | ask for less | | pre/post-exec (outer layer) | add an inner layer if the profile allows | | `service:` probes, restart policy, ports / replica range / spread, rollout | nothing / choose within / set if the profile does not | + + The one exception to "every profile value binds": + **`rusage`: union, the job's value wins** (user decision 2026-10-12). The + effective `rusage[]` is the union of the profile's and the job's resources; + for a resource in both, the job's value wins, in either direction. + - The **per-type** (`/task`, `/job`, `/host`) is the profile's: a job term + without one inherits it (`mem=4000/task` + `mem=16000` gives + `mem=16000/task`); a job term with a *different* per-type is rejected at + submit ("resource mem is per task in profile openfoam"), never silently + reinterpreted (16000/job read as per task would multiply by the task + count). + - Other modifiers: a `duration` / `decay` the job writes replaces the + profile's; one it omits is inherited. + - `name=0` means "reserve none" (the only way to drop a profile resource); + requirements must confirm the scheduler treats a 0 reservation that way. + - Consequence: `rusage` is the one profile section that is a per-resource + **default**, not a binding essential; a profile cannot enforce a minimum + reservation. Hard protection of shared resources comes from `limits.yaml` + and project quotas; later, a "shared/counted" marker on the resource in + `resources.yaml` could forbid lowering below the profile value (licences), + if under-reservation proves a problem. + - Side effects to handle in requirements: measured resources (memory) are + charged `max(reserved, actual)` once usage is reported + (`resource-accounting.md`), so lowering them mostly self-corrects, but + counted resources (licences) do not; xcecli and `--dry-run` warn when a + job lowers a profile value, and when a preset restates a profile value + (it would pin the job against later profile changes); job details label + each term's origin (profile or job); queue ceilings and project quotas + check the merged value, and the error names the job's override; a + profile `rusage` change changes the content hash, so in-flight dispatches + re-pend and pending jobs re-merge even if they override that resource + (accepted). + 2. **Ceiling: queue or project vs. the job's effective values.** A queue ceiling or project quota never changes the job's values, it only checks them: