docs(spec): projects own shares of their parent's resources; project-guarantee becomes project-borrowing

A project's `share` (absolute or a percentage of its parent's) is carved
from the parent; siblings never add up to more than the parent. v1 is
strict: a project holds at most its share. A project's own jobs and its
children without a share draw from its unassigned part. Borrowing idle
shares above a project's own (`max`) and reclaim are future work, in the
renamed project-borrowing brief.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

2026-10-13 06:38:39 +00:00

3.4 KiB

Raw Blame History

Brief: project-borrowing

(/kiro-discovery, 2026-10-09, Path E "configuration layering", as project-guarantee. Renamed and narrowed 2026-10-12 (user decisions): ownership moved into project-config as owned shares, so what remains here is borrowing and reclaim. Backlog: not started until project-config and scheduler-preemption land.)

Problem

With project-config, a project owns a share of its parent's resources and, in v1, holds at most that share (job-config-layering.md "Shares"). That keeps every owner's share available to it, but capacity a project owns and does not use sits idle while other projects' jobs wait. Sites want idle shares lent out and taken back when the owner needs them.

Current State

Owned shares (absolute or percentage of the parent's), the unassigned part and member_quota are specified in project-config (Req 6), compiled into the internal limit plugin. No project can exceed its share.
No fairshare: it is in the queue YAML schema only and dropped by xceadmin (mapping.py:80-97).
Preemption is specified but not implemented (scheduler-preemption, requirements generated).

Desired Outcome

max per project and resource (absolute or percentage of the parent's share, at least the project's share): the project may hold up to max, using capacity owned by other projects that is idle. Without max, v1's strict rule (hold at most the share) stays.
Borrowing from idle capacity only: a job may borrow capacity whose owner has no pending job ready to use it, so borrowers do not take capacity an owner is waiting for (precedents: YARN capacity scheduler capacity / maximum-capacity, Kueue nominalQuota / borrowingLimit).
Reclaim: when an owner's pending job needs its share back, borrowers are preempted, requeued or allowed to finish, by policy, through scheduler-preemption. Without preemption, borrowed jobs run to completion and the owner waits at most one job's run time (bounded by the queue's run-time ceiling).
Accounting and display of owned vs borrowed usage per project (REST, xcecli, REPL); pend reasons that say whether a job waits for its own share or for capacity to borrow.

Approach

To be chosen in discovery when the spec starts. Builds on the same project tree and usage counters as project-config; the borrowing check is a new scheduler step next to the generated share limits (a share becomes a borrowing threshold instead of a cap; max becomes the cap).

Scope

In: max, borrowing rule, protection of owners, reclaim policy, owned/borrowed accounting, visibility.
Out: owned shares and caps (project-config); fairshare ordering (queues turn it on; weights from the project tree, later); accounting/billing history.

Boundary Candidates

max config and validation.
Scheduler accounting of owned vs borrowed usage.
Reclaim through scheduler-preemption.

Out of Boundary

Project definitions, membership and shares (project-config).

Upstream / Downstream

Upstream: project-config, scheduler-preemption, job-config-merge.
Downstream: lsf-config-migration (guarantee pools, LSF GuaranteedResourcePool).

Existing Spec Touchpoints

Extends: project-config (shares).
Adjacent: scheduler-preemption, resource-distribution-mapping.

Constraints

Scheduler driver equivalence (live and simulator identical).
Touches scheduling policy and HA state: full 3-phase workflow.

3.4 KiB Raw Blame History