myxce/.kiro/specs/project-config/requirements.md

Requirements Document

Project Description (Input)

Discovery brief: .kiro/specs/project-config/brief.md. Governing rules: .kiro/steering/job-config-layering.md (2026-10-09): the project owns resource ownership (membership, owned shares, later borrowing, accounting) and never owns how a job runs or is ordered.

Who has the problem: cluster admins who need to control which users may charge work to a project and how much of the cluster each project may use; users who need to know which projects they can use and why a job is held by a project share.

Current situation:

• A project is a free-text label on the job (-P). Anyone can name any project. The apiserver only checks its length, and the scheduler maps an empty name to "default".
• No project is defined anywhere in cluster config.
• Project-scoped limits exist in limits.yaml, but the scheduler's project-group table is never loaded. The host-group and project-group managers were passed to build_limit swapped (bug-1933, fixed in 6ecd418d).
• xcecli jobs list -P is rejected by the apiserver ("unsupported filter").
• Nothing shows a project's usage.

What should change (v1 = owned shares): nested project definitions in cluster config, each owning a share of its parent's resources; default-project resolution; a project:submit check at submit (implied by membership); project shares enforced end to end and rolling up the tree; project visibility. Borrowing is project-borrowing.

Decisions (2026-10-10):

• Access is by project membership (2026-10-12: membership implies the RBAC permission project:submit; roles may also grant it).
• Open mode applies until any project is defined.
• Default project: group default first, then the cluster default.
• A pending job held by a project change pends with a reason.
• Projects nest, named by path (e.g. eng/hpc/cfd).
• A job may be charged to any project in the tree.
• Membership is not inherited between parent and child, and a user may be a member of any number of projects.
• Usage rolls up: a job counts against its project and every ancestor.
• RBAC interaction:
  • a new project scope for job permissions, so a project manager can view and control the jobs of a project and its whole subtree;
  • a project:view permission.
  • Runtime (delegated) membership edits are left for later; membership changes only through cluster config.

Decisions (2026-10-12, resource control under the project tree; job-config-layering.md "Resource control"):

• The project tree is the only place concurrent use is capped. Quotas are written on the project (quota, member_quota), not in limits.yaml; the limit plugin stays as the internal mechanism, and the scheduler compiles them into limits. (quota became share the same day, below.)
• limits.yaml is removed; this feature rejects its project-scoped entries, limits-to-owners retires the rest.
• member_quota caps each user's jobs in the project's whole subtree.
• (Superseded the same day by "owned shares", below: quota became share.)
• The full path is the project's identity and the only form a submit accepts; there is no last-segment shorthand.
• Members are users and user groups (with all and excludes); a sub-project is never a member.

Decisions (2026-10-12, owned shares; job-config-layering.md "Shares"):

• A project owns a share of its parent's resources, per resource, as an absolute amount or a percentage of the parent's effective share; the root's share is the cluster's capacity. Siblings' shares never add up to more than their parent's.
• v1 is strict: a project holds at most its share. Borrowing idle shares (a max above the share) and reclaiming them is future work (project-borrowing).
• A project's own jobs and its children without a share draw from its unassigned part (its share minus its children's shares), first come, first served.
• A project may declare a share for a resource only if its parent has one for that resource, or it is top-level.
• A resource no project declares a share for is not controlled (today's behaviour).

Decisions (2026-10-12, project RBAC):

• Submitting under a project is an RBAC permission, project:submit (scope type project), checked like queue:submit.
• Membership implies the grant: being a member of P gives an implicit project:submit on P only (membership is not inherited). Roles may also grant project:submit explicitly (e.g. to service accounts); a project scope covers the named projects and their subtrees, as for every project scope.
• Project admins: a project's admins (users and user groups) get an implicit grant of every job permission and project:view over the project's whole subtree. Being an admin does not imply project:submit.
• In production every job submission goes through the apiserver (REST: xcecli, UIs); the harness and direct client-library submits are test tools. The apiserver is therefore the one place that resolves the project and checks project:submit; the controller records what the apiserver decided and does not re-check RBAC.
• In this stage project definitions, members and admins change only through cluster config, so only the cluster administrator edits them; project admins manage jobs, not the project.

Introduction

This feature makes projects first-class cluster configuration entities that control resource ownership.

• Definitions: admins define a tree of projects named by path, each with its own members and owned shares, and say which user groups default to which project.
• Submit: the system resolves each job's project, admits only users holding project:submit on it (every member does), and records the project on the job.
• Shares: each project owns part of its parent's resources and holds at most that part with its own and its descendants' jobs; a held job's reason names the project.
• Visibility: users and admins can see projects and their usage against their shares.
• RBAC: submitting under a project is the project:submit permission, implied by membership; a project's admins get job permissions over its subtree; roles can grant the same permissions explicitly.
• Compatibility: a cluster that defines no projects keeps today's behaviour.

Boundary Context

• In scope:
  • nested project definitions (path name, description, members, admins, default-for groups, share, member_quota) and a cluster default project;
  • default-project resolution, the project:submit check at submit, and an "open" mode when no projects are defined;
  • pending-job behaviour when a project or membership changes;
  • owned shares (absolute or percentage), the unassigned part, and member_quota, compiled into the scheduler's limits, applying end to end, with project-naming pend reasons;
  • rejecting project-scoped entries in limits.yaml;
  • overlapping membership (a user in several projects);
  • project listing (as a tree) and detail with usage against shares;
  • RBAC: a project:submit permission implied by membership; project admins with implied job permissions over the subtree; a project scope for job permissions (covering the project's subtree) and a project:view permission;
  • job filtering by project.
• Out of scope:
  • borrowing idle shares above a project's own (max), protecting owners from borrowers, and reclaim by preemption (project-borrowing);
  • fairshare;
  • any project influence on job ordering or priority (queues own ordering);
  • accounting history and billing of finished jobs;
  • runtime edits of a project's members, admins or shares by project admins (project-delegated-membership; in this stage only the cluster administrator changes them, through cluster config);
  • per-user default projects in users.yaml;
  • membership inherited between parent and child projects;
  • moving or renaming a project while keeping its jobs' recorded name;
  • wildcard or pattern matching of member names;
  • retiring the rest of limits.yaml (user, queue and host scopes) and moving per-host slots to host config (limits-to-owners);
  • aggregate caps on queues (dropped by decision 2026-10-12, not deferred).
• Adjacent expectations:
  • Project shares are written on the project (decision 2026-10-12, reversing "limits.yaml stays the place where project quotas are written"). The scheduler's limit plugin enforces them as generated limits; admins never write those limits.
  • peer-authentication-hardening makes the controller accept user requests in production mode only from an authenticated apiserver connection; Requirement 3.12 relies on it, and until it lands the controller's trust in the apiserver's authorization is a trusted-network assumption.
  • Shares count what the scheduler allocated or reserved, not measured use; a job without a memory reservation can exceed its project's memory share at run time (accepted; profile default reservations close the gap).
  • User groups come from the existing user-group configuration.
  • Grants made through the role API are subject to the existing rbac-persistence gap (not durable across apiserver restart) until that spec lands.
  • job-config-merge treats project shares as ceilings: they hold or admit a job but never change its values.
  • The defect that resolved limit host and project scopes against each other's group tables (it also broke host-group limits) was fixed separately as bug-1933 (6ecd418d); Requirement 6.8 keeps it as a requirement.

Requirements

Requirement 1: Project definitions

Objective: As a cluster admin, I want to define projects and their members in the cluster configuration, so that project ownership is explicit and versioned with the rest of the cluster config.

Acceptance Criteria

1. The configuration tool shall accept project definitions consisting of a path name, an optional description, members (user names, user-group names, or all), optional excluded members, optional admins (user names or user-group names), an optional list of user groups for which the project is the default, an optional share (a map from resource name to an absolute amount or a percentage of the parent's share) and an optional member_quota (a map from resource name to an absolute amount).
2. The configuration tool shall treat a path name of one segment (e.g. eng) as a top-level project and a path of several /-separated segments (e.g. eng/hpc/cfd) as a child of the project named by the path without its last segment.
3. The configuration tool shall accept an optional cluster default project naming one defined project.
4. If a project definition repeats a path name, uses a reserved name, has an empty or malformed path segment, names a parent project that is not defined, names an undefined user group (in members, excluded members or admins), lists a default-for user group that is not covered by the project's members, has a share or member_quota entry naming an undeclared resource or giving a negative or non-numeric amount, has a percentage in member_quota or a percentage above 100% in share, declares a share for a resource its parent (when it has one) declares no share for, gives siblings shares of one resource in different forms (absolute and percentage), or gives siblings shares that add up to more than 100% or, when all are absolute and the parent's is absolute, to more than the parent's share, then the configuration tool shall reject the configuration with an error identifying the project and the offending field.
5. If the cluster default project names an undefined project, or a project whose members do not include every user, then the configuration tool shall reject the configuration with an error naming the cluster default project.
6. When a configuration with changed project definitions is applied by live configuration upgrade, the Cluster shall apply the new definitions to all subsequent submissions and scheduling decisions without a restart.
7. The Cluster shall keep project definitions, including their shares, across controller failover and restart identically to other cluster configuration.
8. If the limit configuration contains a limit scoped to projects (listed or per project), then the configuration tool shall reject the configuration with an error naming the limit and stating that project shares are written on the project.

Requirement 2: Project resolution at submit

Objective: As a user, I want my job to get the right project when I do not name one, so that I do not have to pass -P for my usual project.

Acceptance Criteria

1. When a job is submitted with a project path name, the Cluster shall use that project for the job, whether it is a top-level, inner or leaf project.
2. When a job is submitted without a project name and at least one defined project lists a user group the submitter belongs to as default-for, the Cluster shall use the first such project in configuration order.
3. When a job is submitted without a project name and no default-for rule matches the submitter, the Cluster shall use the cluster default project if one is configured, and otherwise the project default.
4. The Cluster shall record the resolved project name on the job at submit, show it in the job's details, and keep it unchanged across later configuration changes, failover and restart.
5. The Cluster shall apply the same project resolution to every submission through the REST API (xcecli and any other client of the API) and to every element of a job array.
6. The Cluster shall identify a project at submit only by its full path name; a name that is only the last segment of a defined project's path (e.g. cfd for eng/hpc/cfd) is treated as the top-level project of that name.

Requirement 3: Submit authorization

Objective: As a cluster admin, I want only users allowed on a project to submit work under it, so that projects control who may use their resources.

Acceptance Criteria

1. While at least one project is defined, when a job is submitted under a project, the Cluster shall accept the job's project if the submitter holds project:submit on it: implicitly as a member of that project, or through a role grant whose scope covers it.
2. If, while at least one project is defined, a job is submitted under a project on which the submitter does not hold project:submit, then the Cluster shall reject the submission with an error naming the project and stating that the user may not submit under it.
3. If, while at least one project is defined, a job is submitted under a project name that is not defined, then the Cluster shall reject the submission with an error naming the unknown project.
4. Where the submitter has the cluster administrator role, the Cluster shall accept any defined project without a further check.
5. The Cluster shall treat a user as a member of a project if the user is listed directly in that project's own members, belongs to a user group listed there, or those members include all, and the user is not excluded there.
6. The Cluster shall not infer membership of a project from membership of its parent, its children or any other project; the implicit project:submit of a member covers that project only.
7. The Cluster shall treat a role grant of project:submit with scope type all as covering every project, and with scope type project as covering the named projects and their descendants.
8. Where a user may submit under several projects, the Cluster shall accept the user's jobs under any of those projects.
9. The Cluster shall reject an unauthorized or unknown-project submission before the job is assigned a job ID or recorded.
10. The Cluster shall check project:submit in addition to, not instead of, the existing queue permission.
11. The Cluster shall record on the job whether its project was authorized by membership, by a role grant or by the cluster administrator role.
12. The apiserver shall be the only component that resolves the project and checks project:submit; the controller shall record the project and authorization basis it receives with the submission without checking them again.
13. While the controller runs in test mode, when a job is submitted directly to the controller, bypassing the apiserver (harness and client-library test paths), the Cluster shall record the project name as given (an empty name as default) with the authorization basis direct, without project resolution or authorization; Requirement 5 still applies to the pending job, except the membership hold of 5.2. (In production mode such a submit is rejected by peer-authentication-hardening.)

Requirement 4: Open mode when no projects are defined

Objective: As an operator of an existing cluster, I want clusters without project definitions to keep working as today, so that upgrading does not break existing submissions.

Acceptance Criteria

1. While no project is defined in the cluster configuration, the Cluster shall accept any project name at submit without a project:submit check.
2. While no project is defined in the cluster configuration, when a job is submitted without a project name, the Cluster shall record the project default.
3. When the first project definitions are applied to a cluster that had none, the Cluster shall apply Requirement 5 to pending jobs whose recorded project is undefined or does not include their owner.
4. While no project is defined, the Cluster shall not check project:submit.

Requirement 5: Pending jobs after project changes

Objective: As a cluster admin, I want project changes to take effect on work that has not started, without disturbing running work, so that removing a project or a member is effective and reversible.

Acceptance Criteria

1. If, while projects are defined, a pending job's recorded project is no longer defined, then the Scheduler shall keep the job pending with a reason naming the project and stating that it is not defined.
2. If a pending job's project was authorized by membership and its owner is no longer a member of the job's recorded project, then the Scheduler shall keep the job pending with a reason naming the project and stating that the owner is not a member. A job authorized by a role grant or the cluster administrator role is not held when roles change.
3. When a later configuration change makes a held job's project defined again, or restores its owner's membership, the Scheduler shall consider the job for scheduling again without user action.
4. While a job is running, the Cluster shall not stop, requeue or alter the job because of a change to its project's definition or membership.
5. The Cluster shall allow an admin to kill or modify a job held by Requirement 5 using the existing job controls.

Requirement 6: Owned shares

Objective: As a cluster admin, I want each project to own a share of its parent's resources and hold no more than it, so that a project can always get what it owns and no project, or member of one, takes more than its part.

Acceptance Criteria

1. The Scheduler shall compute each project's effective share of a resource as its absolute amount, or as its percentage of its parent's effective share; a top-level project's percentage is of the cluster's capacity of that resource (the static total over the cluster's hosts), so percentage shares change when hosts are added or removed.
2. The Scheduler shall count against a project's share of a resource the amount of that resource allocated to every running job whose recorded project is that project or one of its descendants.
3. The Scheduler shall treat a project's unassigned part of a resource as its effective share minus the effective shares of its children that declare a share of that resource (the cluster's capacity minus the top-level shares at the root), and shall count against it the jobs of the project itself and of its descendants that are not under a child with a share of that resource.
4. The Scheduler shall count against a project's member_quota for a resource, separately for each user, the amount of that resource allocated to that user's running jobs whose recorded project is that project or one of its descendants, whether or not the user is a direct member of that project.
5. The Scheduler shall dispatch a job only if, for every resource it is allocated, the job fits the share of every project on its path that declares one, the unassigned part it draws from, and the member_quota of its recorded project and of every ancestor.
6. The Scheduler shall not let a project hold more than its effective share by using capacity owned by another project (no borrowing in this feature).
7. While no project declares a share of a resource, the Scheduler shall not restrict that resource by project.
8. The Scheduler shall count a job only against its recorded project and that project's ancestors, never against other projects its owner belongs to.
9. If dispatching a job would exceed a share, an unassigned part or a member quota, then the Scheduler shall keep the job pending with a reason naming the project, the resource, which of the three is full, and its current usage and amount; the reason shall not name an internal limit.
10. When usage falls below a share, unassigned part or member quota, the Scheduler shall consider jobs held by it for scheduling again.
11. When a configuration change or a change in the cluster's hosts changes an effective share, the Scheduler shall apply the new amounts to all subsequent scheduling decisions without a restart, keep counting running jobs against their recorded project and its ancestors, and never stop or alter a running job because a share is now below its usage.
12. Where absolute shares of top-level projects add up to more than the cluster's current capacity of a resource, the Cluster shall report it in project diagnostics and in a configuration-tool warning; the Scheduler shall still apply each share as written.
13. The Scheduler shall resolve each generated limit's project and user scopes against project and user names only, so that a project name never matches a host group or user group (the scope separation fixed by bug-1933).
14. The live Scheduler and the scheduler simulator shall make identical share decisions for the same input.
15. The Scheduler shall show a project path containing / unchanged in pend reasons, decision traces and diagnostics.

Requirement 7: Project visibility

Objective: As a user or admin, I want to see projects, their members and their usage against shares, so that I know which projects I can use and why a job is held.

Acceptance Criteria

1. When a user lists projects through the REST API or xcecli, the Cluster shall return every project the user may submit under (as a member or through a role grant) or administers, with its full path name and description, shown in tree order, and mark the project that would be the user's default, so that a submit client can offer them for choosing the project to charge.
2. Where the requester has the cluster administrator role, or a project:view permission covering a project, the Cluster shall include that project in the list with its members, excluded members and default-for user groups.
3. When a user requests a project's details, the Cluster shall show, for the project and for each of its ancestors, the effective share (and the form it was written in), the unassigned part, and every member_quota entry, each with the project it is set on, the resource, the amount and the current usage counted against it (including descendants' jobs; for member_quota, the requesting user's own usage, and every user's usage to an admin or a holder of project:view covering the project).
4. If a user requests the details of a project they are neither a member nor an admin of, and they have neither the cluster administrator role nor a project:view permission covering the project, then the Cluster shall reject the request without revealing the project's members or usage.
5. When a user lists jobs filtered by project through the REST API or xcecli, the Cluster shall return only jobs with that recorded project, or, when the user asks to include sub-projects, with that project or any descendant, subject to the user's existing job-visibility permissions.
6. While no project is defined, the Cluster shall report that projects are not configured when projects are listed.
7. The scheduler's diagnostic interface shall show, for a named project, the shares, unassigned parts and member quotas that apply to it (its own and its ancestors'), their effective amounts and their current usage.

Requirement 8: Project-scoped access control

Objective: As a cluster admin, I want project admins and roles to hold rights over the jobs and details of a project and its sub-projects, so that project leads can manage their project's work without being cluster administrators.

Acceptance Criteria

1. The Cluster shall accept roles that grant a job permission (view, kill, modify or any other existing job permission) with the scope type project and a list of project path names as scope values.
2. The Cluster shall treat a project-scoped grant on a project as covering that project and every descendant project.
3. When a user performs a job operation, the Cluster shall allow it if any of the user's grants for that operation covers the job (scope all, self for the user's own jobs, or project covering the job's recorded project), and shall reject it with a permission error otherwise.
4. When a user lists jobs, the Cluster shall return exactly the jobs covered by the user's view grants.
5. The Cluster shall accept roles that grant project:view with the scope type all or project; a project scope covers the named projects and their descendants.
6. If, while projects are defined, a role is created or updated with a project scope value that names an undefined project, then the Cluster shall reject the change with an error naming the unknown project.
7. While a project scope value names a project that is no longer defined, the Cluster shall treat that value as covering no project.
8. The Cluster shall leave the behaviour of existing roles and grants unchanged; the default RBAC configuration shall grant no project-scoped permission and no explicit project:submit to any role other than the cluster administrator.
9. The Cluster shall accept roles that grant project:submit with the scope type all or project.
10. The Cluster shall treat each admin of a project (a listed user, or a member of a listed user group) as holding every job permission and project:view with scope project on that project, which covers its subtree, without any role assignment.
11. The Cluster shall not give a project admin project:submit on the project, nor any right to change a project's definition, members, admins or shares, by being its admin.
12. When a configuration change adds or removes a project admin, the Cluster shall apply the change to all subsequent requests without a restart.
13. When a user lists their projects, the Cluster shall mark the projects the user administers.

Requirement 9: Compatibility and durability

Objective: As an operator, I want projects added without data migration or downtime, so that existing jobs and records keep working.

Acceptance Criteria

1. When a job recorded before this feature is recovered, the Cluster shall keep its recorded project name, treating an empty name as default.
2. The Cluster shall accept a cluster configuration without any project definitions and behave as Requirement 4 describes.
3. When a configuration tool of an earlier version pushes a configuration without project definitions, the Cluster shall treat it as defining no projects.
4. While no project declares a share or member quota, the Scheduler shall dispatch jobs exactly as it would without this feature's share checks.