Rongsong Shen
b761878b58
New briefs: job-config-merge (one live merge of job, application profile, queue policy and project into effective values; simplified rules: job overrides profile, select ANDed, queue hard/task limits reject; queue is policy only, res_req deprecated), project-config (project quotas v1) and project-guarantee (backlog). application-profile: the profile carries the job's resreq (incl. span) and task limits; "frozen at submit" replaced by the live merge. parjob-placement: span constraints live in the fixed-N planner and the range search bisects over the planner's allowed counts; LSF 10.1 block, ptile, stripe and HOSTLIMIT_PER_JOB findings recorded. Roadmap: new specs, existing-spec updates, lsf-config-migration backlog. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
85 lines
4.2 KiB
Markdown
85 lines
4.2 KiB
Markdown
# Brief: project-config
|
|
|
|
(`/kiro-discovery`, 2026-10-09, Path E "configuration layering". Reference:
|
|
IBM Spectrum LSF 10.1 Administering LSF pp. 107 (default project), 446-453
|
|
(resource allocation limits, PROJECTS / PER_PROJECT); Using LSF on UNIX p21
|
|
(`bsub -P`).)
|
|
|
|
## Problem
|
|
Projects are meant to control **resource ownership**: who may charge work to
|
|
a project and how much of the cluster a project may use. Today a project is
|
|
only a free-text label the user types; anyone can name any project, project
|
|
limits cannot be configured end to end, and nothing shows usage per project.
|
|
|
|
## Current State
|
|
- Submit data carries `project_name` (`JobSubmitData`, `controller.capnp:582`
|
|
→ `job_submit_data_t`, `types.h:356-368`); `xcecli jobs submit -P`
|
|
(`xcecli/cmd/jobs/submit.go:60`); the apiserver passes it through
|
|
(`backend/convert.go:307-317`) without any check.
|
|
- The scheduler maps an empty name to `"default"` (`sched_get_project`,
|
|
`scheduler/src/core.c:617-623`). `SchedProject` (capnp:1260) has name and
|
|
priority but no config source.
|
|
- Limits already accept project scopes (`LimitConfig`, `limits.yaml`;
|
|
`CONSUMER_PROJECT`, `lib/xcelimit.c:23-26`, vres key `/p=<project>`), but
|
|
the scheduler's project-group manager `pgmgr` is created empty
|
|
(`core.c:185`) and never loaded (`scheduler.c:1144-1145` loads only
|
|
`ugmgr`/`hgmgr`), so project limits cannot be configured to work.
|
|
- No project fairshare, accounting view or access check exists.
|
|
|
|
## Desired Outcome (v1 = quota; user decision 2026-10-09)
|
|
- **Project definitions** in the cluster configuration: name, description,
|
|
members (users and user groups), priority; plus a cluster default project.
|
|
Schema, YAML, JSON schema, `xceadmin` mapping, live-upgrade replication.
|
|
- **Project selection** at submit: `-P` > the user's default project (if
|
|
defined) > the cluster default > `"default"` (LSF p107 order, without
|
|
environment variables).
|
|
- **Access check** at submit: the user must be a member of the project
|
|
(apiserver early check, controller authoritative), with a clear error;
|
|
`cluster_admin` keeps its existing bypass. Whether this is RBAC scope
|
|
(`project` scope type) or config membership is settled in requirements.
|
|
- **Quotas**: the scheduler loads project groups from config so
|
|
`PROJECTS` / `PER_PROJECT` limits in `limits.yaml` work end to end
|
|
(slots, jobs, mem, any resource), with pend reasons naming the project.
|
|
- **Visibility**: project shown in job details and listings; per-project
|
|
usage against its limits (REST, `xcecli`, REPL `info limit`).
|
|
- **Merge input**: the project is one level for `job-config-merge` (default
|
|
project; any project-level defaults decided in requirements).
|
|
|
|
## Approach
|
|
Follow the queue/usergroup config pattern (`config-design.md`): projects are
|
|
config entities pushed to controller and scheduler (`Conf4Sched`), the
|
|
scheduler fills `pgmgr` from them, and the existing limit machinery does the
|
|
rest. Projects already reach the scheduler on each job, so the WAL needs no
|
|
new job field.
|
|
|
|
## Scope
|
|
- **In**: project config model and pipeline; default project resolution;
|
|
submit access check; `pgmgr` loading; project limits end to end; display;
|
|
tests (unit, sim, live).
|
|
- **Out**: guaranteed shares, owned pools, borrowing and reclaim
|
|
(`project-guarantee`); fairshare; accounting history/billing.
|
|
|
|
## Boundary Candidates
|
|
- Config model and pipeline.
|
|
- Submit resolution and access check (apiserver, controller).
|
|
- Scheduler project groups and limits.
|
|
- Display surfaces.
|
|
|
|
## Out of Boundary
|
|
- Scheduling-order policy (queues), application settings (profiles).
|
|
|
|
## Upstream / Downstream
|
|
- **Upstream**: config pipeline (`admin-config-tool`, `config-upgrade-snapshot`),
|
|
limit plugin (`plugins/limit.c`, `lib/xcelimit.c`), RBAC (`rbac_check.go`).
|
|
- **Downstream**: `job-config-merge` (project level), `project-guarantee`,
|
|
`lsf-config-migration`, `rbac-persistence` (if access uses RBAC scopes).
|
|
|
|
## Existing Spec Touchpoints
|
|
- **Extends**: none.
|
|
- **Adjacent**: `scheduler-sim-diagnostics` (`info limit`),
|
|
`resource-distribution-mapping`, `rbac-persistence`.
|
|
|
|
## Constraints
|
|
- Middleware chain unchanged; additive Cap'n Proto; old jobs keep their
|
|
recorded project name even if the project is later removed from config
|
|
(requirements decide what a pending job with a removed project does).
|