docs(spec): projects own shares of their parent's resources; project-guarantee becomes project-borrowing
A project's `share` (absolute or a percentage of its parent's) is carved from the parent; siblings never add up to more than the parent. v1 is strict: a project holds at most its share. A project's own jobs and its children without a share draw from its unassigned part. Borrowing idle shares above a project's own (`max`) and reclaim are future work, in the renamed project-borrowing brief. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
136e4a0132
commit
f3d4dba970
7 changed files with 235 additions and 148 deletions
|
|
@ -73,7 +73,7 @@ The protocol **drive** is NOT fully event-driven. Root cause: `peer_ctrl_msg_han
|
|||
- [ ] controller-job-cleanup — NEW 2026-09-29 (`/kiro-discovery`, Path C). Primary-only periodic sweep (configurable `job_cleanup_period`, default 300 s) removes DONE/EXIT jobs finished longer than configurable `job_retention` (default 3600 s, 0 = off) with no dependents (stub; XCE has no job dependencies yet), via a NEW replicated WAL job-removal fact applied on every node (first-ever `jobmgr` delete; refcount-safe `xkvmgr` delete), propagated to scheduler and apiserver (`OnJobDeleted`). Retires the node-local snapshot-only `job_should_retain` finished-job filter (hard-coded 3600 s; causes snapshot/memory job-set drift). HA/persistence → full 3-phase workflow. Brief: `.kiro/specs/controller-job-cleanup/brief.md`. Dependencies: none (extends ha-entity-snapshots).
|
||||
- [ ] job-status-detail-display — NEW 2026-09-28 (`/kiro-discovery`, Path E). End users (REST/`xcecli`) see NO pend reason for ANY job today: the apiserver cgo conversion (`backend/convert.go`) never maps `xce_job_t.pend_reasons`, so `pending_reason` is always empty. Maps it to a structured JSON (including `alternatives` from day one) using libxce's code→text mapping, and renders it in `xcecli` job detail. Open for requirements: also map `run_usage` → `JobRusage` (same gap; reserved vs. used for RUN jobs). Brief: `.kiro/specs/job-status-detail-display/brief.md`. Dependencies: none for flat reasons; scheduler-alter-pend-reasons for per-alternative rendering.
|
||||
- [ ] job-config-merge — NEW 2026-10-09 (`/kiro-discovery`, Path E "configuration layering"; ref LSF 10.1 Admin Guide pp. 314-316, 435-437, 561-563, 643, 660). One merge function in `src/lib` turning the job's own values + application profile + queue policy + project into effective values (resreq incl. `span`, task range, limits, pre/post-exec, env). One owner per attribute (user decision; `.kiro/steering/job-config-layering.md`): the job may add to a profile's values but never change or remove them (additive rule, 2026-10-12, replacing *default*/*fixed* markers); queue ceilings and project quotas only check (reject if never satisfiable, pend if full now). **Live** merge (user decision): WAL keeps inputs + names; controller/apiserver validate at submit; scheduler computes effective values and re-evaluates pending jobs on config change. Queue is policy only: queue `res_req` and `pre_exec`/`post_exec` deprecated (pre/post-exec moves to the profile), dropped queue limits (cpulimit/memlimit/runlimit/task limit) wired. Brief: `.kiro/specs/job-config-merge/brief.md`. Dependencies: none.
|
||||
- [ ] project-config — NEW 2026-10-09 (configuration layering). Projects control resource ownership, v1 = quota: project definitions in cluster config (path identity, members, cluster default), default-project resolution, submit access check, scheduler `pgmgr` loaded from config, quotas written on the project (`quota`, `member_quota` over the subtree; user decision 2026-10-12, replacing `PROJECTS`/`PER_PROJECT` limits in `limits.yaml`) and compiled into the limit plugin, per-project usage display; `project:submit` RBAC implied by membership, project admins, authorization in the apiserver only. Brief: `.kiro/specs/project-config/brief.md`. Dependencies: none to ship; production rejection of direct submits comes from peer-authentication-hardening.
|
||||
- [ ] project-config — NEW 2026-10-09 (configuration layering). Projects control resource ownership, v1 = quota: project definitions in cluster config (path identity, members, cluster default), default-project resolution, submit access check, scheduler `pgmgr` loaded from config, owned shares written on the project (`share` of the parent's resources, absolute or percentage, strict in v1; children without a share draw from the parent's unassigned part; `member_quota` over the subtree; user decisions 2026-10-12, replacing `PROJECTS`/`PER_PROJECT` limits in `limits.yaml`) compiled into the limit plugin, per-project usage display; `project:submit` RBAC implied by membership, project admins, authorization in the apiserver only. Brief: `.kiro/specs/project-config/brief.md`. Dependencies: none to ship; production rejection of direct submits comes from peer-authentication-hardening.
|
||||
|
||||
- [ ] host-failure-job-recovery — NEW 2026-10-10 (`/kiro-discovery`, "Long-running services" phase 1; also a batch gap: RUN jobs on a dead host stay RUN forever, `controller/sched.c:631` FIXME). After a grace period, jobs on a lost host go to an unknown state, then are requeued (rerunnable, job option or profile default) or end EXIT "host lost"; multi-host jobs resolved as a whole; attempt numbers fence a stale copy when the host returns; manual requeue (REST/xcecli). Brief: `.kiro/specs/host-failure-job-recovery/brief.md`. Dependencies: none.
|
||||
- [ ] service-supervision — NEW 2026-10-10 (Long-running services phase 1). Application profile kind `service`: restart policy (always/on-failure/never, backoff, give-up), liveness and readiness probes (command/TCP/HTTP) run by each replica's own worker (no special replica), in-place restart, change-only health reports, display. Brief: `.kiro/specs/service-supervision/brief.md`. Dependencies: application-profile, host-failure-job-recovery.
|
||||
|
|
@ -119,7 +119,7 @@ Direction: `.kiro/steering/resource-model.md`. Decisions, proposals (PR-1 generi
|
|||
|
||||
## Future Specs / Backlog (dependency order)
|
||||
- [ ] gang-groups — NEW 2026-10-11; **demoted to backlog 2026-10-12** (user decision N24 in `resource-questions.md`). LLM training is one parallel job with topology requirements (PR-10), so it needs no gang. Re-scoped as a **co-start group** of separately managed jobs and services: RL post-training (trainer, rollout engines, reward and reference models), disaggregated serving (prefill and decode pools, router), streaming pipelines. All-or-nothing start, close placement, group failure policy. Revisit when RL or disaggregated serving reaches the roadmap. Brief: `.kiro/specs/gang-groups/brief.md`. Dependencies: resource-instances, resource-claims, service-replica-set.
|
||||
- [ ] project-guarantee — NEW 2026-10-09 (configuration layering, backlog). Project-owned guarantees (resource amount or host pool), borrowing of idle guaranteed capacity, reclaim through preemption. Brief: `.kiro/specs/project-guarantee/brief.md`. Dependencies: project-config, scheduler-preemption.
|
||||
- [ ] project-borrowing — NEW 2026-10-09 as project-guarantee (configuration layering, backlog); renamed and narrowed 2026-10-12: ownership moved into project-config's owned shares. A `max` above a project's share, borrowing idle capacity whose owner is not waiting for it, reclaim through preemption, owned/borrowed accounting. Brief: `.kiro/specs/project-borrowing/brief.md`. Dependencies: project-config, scheduler-preemption.
|
||||
- [ ] limits-to-owners — NEW 2026-10-12 (configuration layering, user decisions; backlog, no brief yet). Retire `limits.yaml` (not in production, removed without deprecation): the limit plugin becomes internal, fed only by generated limits. `USERS`/`PER_USER` caps become `member_quota` (root project for a cluster-wide per-user cap), a user group's total becomes a project quota, `HOSTS`/`PER_HOST` slot caps move to host supply in host config (LSF `MXJ`), queue aggregate caps and mixed tuples (user group × queue) are dropped. Touches `xceadmin` mapping and schema (`admin-config-tool`), `ClusterConfig.limits`, `src/configs/limits.yaml`, scheduler sim fixtures and the limit tests. Rules: `job-config-layering.md` "Resource control". Dependencies: project-config.
|
||||
- [ ] job-resize — NEW 2026-10-10 (from long-running services discovery; not on the services path). Grow/shrink a running parallel job's allocation (LSF resizable jobs, `bresize`) for elastic applications. Brief: `.kiro/specs/job-resize/brief.md`. Dependencies: parjob-placement, host-failure-job-recovery.
|
||||
- [ ] project-delegated-membership — NEW 2026-10-10 (from project-config requirements, user decision: future). Project managers edit their project's (and subtree's) members at runtime without an admin config push; needs a durable store that coexists with `xceadmin` config pushes (e.g. a separate delegated member list) and audit. Dependencies: project-config, rbac-persistence.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue