docs(spec): projects own shares of their parent's resources; project-guarantee becomes project-borrowing
A project's `share` (absolute or a percentage of its parent's) is carved from the parent; siblings never add up to more than the parent. v1 is strict: a project holds at most its share. A project's own jobs and its children without a share draw from its unassigned part. Borrowing idle shares above a project's own (`max`) and reclaim are future work, in the renamed project-borrowing brief. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
136e4a0132
commit
f3d4dba970
7 changed files with 235 additions and 148 deletions
76
.kiro/specs/project-borrowing/brief.md
Normal file
76
.kiro/specs/project-borrowing/brief.md
Normal file
|
|
@ -0,0 +1,76 @@
|
|||
# Brief: project-borrowing
|
||||
|
||||
(`/kiro-discovery`, 2026-10-09, Path E "configuration layering", as
|
||||
`project-guarantee`. Renamed and narrowed 2026-10-12 (user decisions):
|
||||
ownership moved into `project-config` as owned shares, so what remains here
|
||||
is borrowing and reclaim. Backlog: not started until `project-config` and
|
||||
`scheduler-preemption` land.)
|
||||
|
||||
## Problem
|
||||
With `project-config`, a project owns a share of its parent's resources and,
|
||||
in v1, holds at most that share (`job-config-layering.md` "Shares"). That
|
||||
keeps every owner's share available to it, but capacity a project owns and
|
||||
does not use sits idle while other projects' jobs wait. Sites want idle
|
||||
shares lent out and taken back when the owner needs them.
|
||||
|
||||
## Current State
|
||||
- Owned shares (absolute or percentage of the parent's), the unassigned
|
||||
part and `member_quota` are specified in `project-config` (Req 6), compiled
|
||||
into the internal limit plugin. No project can exceed its share.
|
||||
- No fairshare: it is in the queue YAML schema only and dropped by
|
||||
`xceadmin` (`mapping.py:80-97`).
|
||||
- Preemption is specified but not implemented (`scheduler-preemption`,
|
||||
requirements generated).
|
||||
|
||||
## Desired Outcome
|
||||
- **`max`** per project and resource (absolute or percentage of the
|
||||
parent's share, at least the project's share): the project may hold up to
|
||||
`max`, using capacity owned by other projects that is idle. Without
|
||||
`max`, v1's strict rule (hold at most the share) stays.
|
||||
- **Borrowing from idle capacity only**: a job may borrow capacity whose
|
||||
owner has no pending job ready to use it, so borrowers do not take
|
||||
capacity an owner is waiting for (precedents: YARN capacity scheduler
|
||||
`capacity` / `maximum-capacity`, Kueue `nominalQuota` /
|
||||
`borrowingLimit`).
|
||||
- **Reclaim**: when an owner's pending job needs its share back, borrowers
|
||||
are preempted, requeued or allowed to finish, by policy, through
|
||||
`scheduler-preemption`. Without preemption, borrowed jobs run to
|
||||
completion and the owner waits at most one job's run time (bounded by
|
||||
the queue's run-time ceiling).
|
||||
- Accounting and display of owned vs borrowed usage per project (REST,
|
||||
`xcecli`, REPL); pend reasons that say whether a job waits for its own
|
||||
share or for capacity to borrow.
|
||||
|
||||
## Approach
|
||||
To be chosen in discovery when the spec starts. Builds on the same project
|
||||
tree and usage counters as `project-config`; the borrowing check is a new
|
||||
scheduler step next to the generated share limits (a share becomes a
|
||||
borrowing threshold instead of a cap; `max` becomes the cap).
|
||||
|
||||
## Scope
|
||||
- **In**: `max`, borrowing rule, protection of owners, reclaim policy,
|
||||
owned/borrowed accounting, visibility.
|
||||
- **Out**: owned shares and caps (`project-config`); fairshare ordering
|
||||
(queues turn it on; weights from the project tree, later);
|
||||
accounting/billing history.
|
||||
|
||||
## Boundary Candidates
|
||||
- `max` config and validation.
|
||||
- Scheduler accounting of owned vs borrowed usage.
|
||||
- Reclaim through `scheduler-preemption`.
|
||||
|
||||
## Out of Boundary
|
||||
- Project definitions, membership and shares (`project-config`).
|
||||
|
||||
## Upstream / Downstream
|
||||
- **Upstream**: `project-config`, `scheduler-preemption`, `job-config-merge`.
|
||||
- **Downstream**: `lsf-config-migration` (guarantee pools, LSF
|
||||
`GuaranteedResourcePool`).
|
||||
|
||||
## Existing Spec Touchpoints
|
||||
- **Extends**: `project-config` (shares).
|
||||
- **Adjacent**: `scheduler-preemption`, `resource-distribution-mapping`.
|
||||
|
||||
## Constraints
|
||||
- Scheduler driver equivalence (live and simulator identical).
|
||||
- Touches scheduling policy and HA state: full 3-phase workflow.
|
||||
Loading…
Add table
Add a link
Reference in a new issue