docs(specs): init job-config-merge; project-config requirements

job-config-merge: spec initialized from its discovery brief.

project-config: requirements generated (9 requirements): nested projects
named by path, membership-based submit check with no inheritance and
overlapping membership, group-default then cluster-default project, open
mode until projects are defined, pending jobs held with a reason after
project changes, roll-up quotas, project visibility, a project scope for
job permissions covering the subtree and a project:view permission,
compatibility.

Roadmap: project-delegated-membership backlog item (runtime membership
edits by project managers, deferred).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

.kiro/specs/job-config-merge/requirements.md

@@ -0,0 +1,42 @@
+# Requirements Document
+
+## Project Description (Input)
+Discovery brief: `.kiro/specs/job-config-merge/brief.md`. Governing rules: `.kiro/steering/job-config-layering.md` (2026-10-09).
+
+**Who has the problem:** users submitting jobs, and admins configuring queues, application profiles and projects. Once a job's settings come from several places (the job, its application profile, its queue, its project), every component (apiserver, controller, scheduler, display) needs the same answer to "what did this job actually ask for, and is it allowed?".
+
+**Current situation:**
+- No code combines settings from more than one level. The scheduler's only default turns an empty resreq into `"type == any"` (`scheduler/src/core_events.c:300-309`), and `apply_job_task_count()` (`lib/resreq.c:574-591`) copies the job's task counts into the resreq.
+- Queues carry a `resreq` that nothing reads, and `pre_exec`/`post_exec` that the controller reads at dispatch (`get_queue_exec()`, `controller/queue.c:199`, `job.c:1827`) and the worker runs as an outer layer (`worker/src/xcecd_worker.c:368,391`).
+- `xceadmin` drops queue cpulimit, memlimit, runlimit, task limit and other fields (`xceadmin/src/xcecommon/mapping.py:80-97`).
+- The controller persists the client's submit data unchanged in the WAL (`build_job_submit_payloads`, `controller/job.c:347-371`) and checks neither the queue nor any limit; the apiserver checks only resreq syntax.
+
+**What should change:**
+- One merge function in `src/lib`, shared by controller, scheduler and (through cgo) apiserver. Its inputs are the job's own values, the application profile, the queue's policy and the project; its output is the job's effective values: resreq (select / rusage / span / same / order), task range, execution limits, pre/post-exec, env.
+- **One owner per attribute.** The queue owns scheduling policy, the application profile owns what the application needs and how it runs, and the project owns resource ownership. Only two interactions exist:
+  - **Override:** the job may override a profile attribute marked *default*. It may not change one marked *fixed*, except `select`, where it may add ANDed terms.
+  - **Ceiling:** queue ceilings and project quotas never change a value. A value that can never be satisfied is rejected at submit; a quota that is only full right now makes the job pend with a reason naming the queue or project.
+- **Live merge.** The WAL keeps only the job's own values plus its queue, profile and project names.
+  - The controller (authoritative) and the apiserver (early) run the merge at submit to validate.
+  - The scheduler computes the effective values and re-evaluates pending jobs when config changes. Running jobs are unaffected.
+- **Queue is policy only.** Queue `res_req`, `pre_exec` and `post_exec` are deprecated (warn, then reject).
+  - Pre/post-exec moves to the profile: the profile's commands form the worker's outer layer, and the job's own commands are the inner layer, which the profile can allow or forbid.
+  - The queue limits `xceadmin` drops today are wired through as ceilings.
+- **Display.** Job details (REST, `xcecli`, REPL, decision trace) show each effective value and the level it came from.
+- **Divergence from LSF.** The requirements list each departure from LSF's merge rules, as input for `lsf-config-migration`.
+
+**Out of scope:**
+- the profile model (`application-profile`) and project definitions (`project-config`);
+- `span` placement (`parjob-placement`);
+- LSF compound/alternative resreq merge stages;
+- fairshare.
+
+**Constraints:**
+- additive Cap'n Proto only;
+- WAL/snapshot field completeness for the inputs;
+- live and simulator schedulers compute identical effective values;
+- config-change re-evaluation must not block the delaymgr;
+- full 3-phase workflow.
+
+## Requirements
+<!-- Will be generated in /kiro-spec-requirements phase -->

.kiro/specs/job-config-merge/spec.json

@@ -0,0 +1,24 @@
+{
+  "feature_name": "job-config-merge",
+  "created_at": "2026-10-10T03:23:10Z",
+  "updated_at": "2026-10-10T03:23:10Z",
+  "language": "en",
+  "phase": "initialized",
+  "approvals": {
+    "requirements": {
+      "generated": false,
+      "approved": false
+    },
+    "design": {
+      "generated": false,
+      "approved": false
+    },
+    "tasks": {
+      "generated": false,
+      "approved": false
+    }
+  },
+  "ready_for_implementation": false
+}
+
+

.kiro/specs/project-config/requirements.md

@@ -0,0 +1,167 @@
+# Requirements Document
+
+## Project Description (Input)
+Discovery brief: `.kiro/specs/project-config/brief.md`. Governing rules: `.kiro/steering/job-config-layering.md` (2026-10-09): the project owns resource ownership (membership, quotas, later guarantees, accounting) and never owns how a job runs or is ordered.
+
+**Who has the problem:** cluster admins who need to control which users may charge work to a project and how much of the cluster each project may use; users who need to know which projects they can use and why a job is held by a project limit.
+
+**Current situation:**
+- A project is a free-text label on the job (`-P`). Anyone can name any project. The apiserver only checks its length, and the scheduler maps an empty name to `"default"`.
+- No project is defined anywhere in cluster config.
+- Project-scoped limits exist in `limits.yaml`, but the scheduler's project-group table is never loaded. The host-group and project-group managers were passed to `build_limit` swapped (bug-1933, fixed in `6ecd418d`).
+- `xcecli jobs list -P` is rejected by the apiserver ("unsupported filter").
+- Nothing shows a project's usage.
+
+**What should change (v1 = quota):** nested project definitions in cluster config; default-project resolution; a membership check at submit; project limits working end to end and rolling up the tree; project visibility. Guarantees are `project-guarantee`.
+
+**Decisions (2026-10-10):**
+- Access is by project membership.
+- Open mode applies until any project is defined.
+- Default project: group default first, then the cluster default.
+- A pending job held by a project change pends with a reason.
+- Projects nest, named by path (e.g. `eng/hpc/cfd`).
+- A job may be charged to any project in the tree.
+- Membership is not inherited between parent and child, and a user may be a member of any number of projects.
+- Quotas roll up: a job must fit the limits of its project and of every ancestor.
+- RBAC interaction:
+  - a new `project` scope for job permissions, so a project manager can view and control the jobs of a project and its whole subtree;
+  - a `project:view` permission.
+  - Runtime (delegated) membership edits are left for later; membership changes only through cluster config.
+
+## Introduction
+This feature makes projects first-class cluster configuration entities that control resource ownership.
+- **Definitions:** admins define a tree of projects named by path, each with its own members, and say which user groups default to which project.
+- **Submit:** the system resolves each job's project, admits only members, and records the project on the job.
+- **Quotas:** existing project-scoped limits apply to the right jobs and roll up the tree, and a held job's reason names the project.
+- **Visibility:** users and admins can see projects and their usage against limits.
+- **RBAC:** roles can grant job permissions over a project subtree (project managers) and the right to view projects one does not belong to.
+- **Compatibility:** a cluster that defines no projects keeps today's behaviour.
+
+## Boundary Context
+- **In scope**:
+  - nested project definitions (path name, description, members, default-for groups) and a cluster default project;
+  - default-project resolution, membership check at submit, and an "open" mode when no projects are defined;
+  - pending-job behaviour when a project or membership changes;
+  - project-scoped limits applying end to end and rolling up the tree, with project-naming pend reasons;
+  - overlapping membership (a user in several projects);
+  - project listing (as a tree) and detail with usage against limits;
+  - RBAC: a `project` scope for job permissions (covering the project's subtree) and a `project:view` permission;
+  - job filtering by project.
+- **Out of scope**:
+  - guaranteed shares, owned pools, borrowing, reclaim (`project-guarantee`);
+  - fairshare;
+  - any project influence on job ordering or priority (queues own ordering);
+  - accounting history and billing of finished jobs;
+  - RBAC gating of which project a job is submitted under (membership decides; the existing queue permission still applies);
+  - runtime (delegated) membership edits by project managers (future; membership changes only through cluster config);
+  - per-user default projects in `users.yaml`;
+  - membership inherited between parent and child projects;
+  - moving or renaming a project while keeping its jobs' recorded name;
+  - wildcard or pattern matching of member names.
+- **Adjacent expectations**:
+  - The existing limit configuration (`limits.yaml`) stays the place where project quotas are written; this feature makes those limits take effect for defined projects.
+  - User groups come from the existing user-group configuration.
+  - Grants made through the role API are subject to the existing `rbac-persistence` gap (not durable across apiserver restart) until that spec lands.
+  - `job-config-merge` treats project quotas as ceilings: they hold or admit a job but never change its values.
+  - The defect that resolved limit host and project scopes against each other's group tables (it also broke host-group limits) was fixed separately as bug-1933 (`6ecd418d`); Requirement 6.7 keeps it as a requirement.
+
+## Requirements
+
+### Requirement 1: Project definitions
+**Objective:** As a cluster admin, I want to define projects and their members in the cluster configuration, so that project ownership is explicit and versioned with the rest of the cluster config.
+
+#### Acceptance Criteria
+1. The configuration tool shall accept project definitions consisting of a path name, an optional description, members (user names, user-group names, or `all`), optional excluded members, and an optional list of user groups for which the project is the default.
+2. The configuration tool shall treat a path name of one segment (e.g. `eng`) as a top-level project and a path of several `/`-separated segments (e.g. `eng/hpc/cfd`) as a child of the project named by the path without its last segment.
+3. The configuration tool shall accept an optional cluster default project naming one defined project.
+4. If a project definition repeats a path name, uses a reserved name, has an empty or malformed path segment, names a parent project that is not defined, names an undefined user group, or lists a default-for user group that is not covered by the project's members, then the configuration tool shall reject the configuration with an error identifying the project and the offending field.
+5. If the cluster default project names an undefined project, or a project whose members do not include every user, then the configuration tool shall reject the configuration with an error naming the cluster default project.
+6. When a configuration with changed project definitions is applied by live configuration upgrade, the Cluster shall apply the new definitions to all subsequent submissions and scheduling decisions without a restart.
+7. The Cluster shall keep project definitions across controller failover and restart identically to other cluster configuration.
+
+### Requirement 2: Project resolution at submit
+**Objective:** As a user, I want my job to get the right project when I do not name one, so that I do not have to pass `-P` for my usual project.
+
+#### Acceptance Criteria
+1. When a job is submitted with a project path name, the Cluster shall use that project for the job, whether it is a top-level, inner or leaf project.
+2. When a job is submitted without a project name and at least one defined project lists a user group the submitter belongs to as default-for, the Cluster shall use the first such project in configuration order.
+3. When a job is submitted without a project name and no default-for rule matches the submitter, the Cluster shall use the cluster default project if one is configured, and otherwise the project `default`.
+4. The Cluster shall record the resolved project name on the job at submit, show it in the job's details, and keep it unchanged across later configuration changes, failover and restart.
+5. The Cluster shall apply the same project resolution to every submit path (REST API, `xcecli`, harness and client library) and to every element of a job array.
+
+### Requirement 3: Membership check at submit
+**Objective:** As a cluster admin, I want only a project's members to submit work under it, so that projects control who may use their resources.
+
+#### Acceptance Criteria
+1. While at least one project is defined, when a job is submitted under a project whose members include the submitter, the Cluster shall accept the job's project.
+2. If, while at least one project is defined, a job is submitted under a project whose members do not include the submitter, then the Cluster shall reject the submission with an error naming the project and stating that the user is not a member.
+3. If, while at least one project is defined, a job is submitted under a project name that is not defined, then the Cluster shall reject the submission with an error naming the unknown project.
+4. Where the submitter has the cluster administrator role, the Cluster shall accept any defined project without a membership check.
+5. The Cluster shall treat a user as a member of a project if the user is listed directly in that project's own members, belongs to a user group listed there, or those members include `all`, and the user is not excluded there.
+6. The Cluster shall not infer membership of a project from membership of its parent, its children or any other project.
+7. Where a user is a member of several projects, the Cluster shall accept the user's jobs under any of those projects.
+8. The Cluster shall reject a non-member or unknown-project submission before the job is assigned a job ID or recorded, on every submit path.
+
+### Requirement 4: Open mode when no projects are defined
+**Objective:** As an operator of an existing cluster, I want clusters without project definitions to keep working as today, so that upgrading does not break existing submissions.
+
+#### Acceptance Criteria
+1. While no project is defined in the cluster configuration, the Cluster shall accept any project name at submit without a membership check.
+2. While no project is defined in the cluster configuration, when a job is submitted without a project name, the Cluster shall record the project `default`.
+3. When the first project definitions are applied to a cluster that had none, the Cluster shall apply Requirement 5 to pending jobs whose recorded project is undefined or does not include their owner.
+
+### Requirement 5: Pending jobs after project changes
+**Objective:** As a cluster admin, I want project changes to take effect on work that has not started, without disturbing running work, so that removing a project or a member is effective and reversible.
+
+#### Acceptance Criteria
+1. If, while projects are defined, a pending job's recorded project is no longer defined, then the Scheduler shall keep the job pending with a reason naming the project and stating that it is not defined.
+2. If a pending job's owner is no longer a member of the job's recorded project, then the Scheduler shall keep the job pending with a reason naming the project and stating that the owner is not a member.
+3. When a later configuration change makes a held job's project defined again, or restores its owner's membership, the Scheduler shall consider the job for scheduling again without user action.
+4. While a job is running, the Cluster shall not stop, requeue or alter the job because of a change to its project's definition or membership.
+5. The Cluster shall allow an admin to kill or modify a job held by Requirement 5 using the existing job controls.
+
+### Requirement 6: Project quotas
+**Objective:** As a cluster admin, I want limits scoped to projects to cap each project's usage, so that a project cannot use more than its share of the cluster.
+
+#### Acceptance Criteria
+1. When a limit is scoped to listed projects, the Scheduler shall count against that limit exactly the jobs whose recorded project is a listed project or a descendant of one (or any project, for `all`), and is neither an excluded project nor a descendant of one.
+2. When a limit is scoped per project, the Scheduler shall apply the limit value separately to each matching project, counting that project's own jobs and the jobs of its descendants.
+3. The Scheduler shall dispatch a job only if it fits every limit that applies to the job's recorded project or to any of its ancestors.
+4. The Scheduler shall count a job only against limits matched through its recorded project and that project's ancestors, never against limits of other projects its owner belongs to.
+5. If dispatching a job would exceed a project-scoped limit, then the Scheduler shall keep the job pending with a reason naming the limit and the project whose limit is full (the job's own project or the ancestor).
+6. When usage under a project-scoped limit falls below its value, the Scheduler shall consider jobs held by that limit for scheduling again.
+7. The Scheduler shall resolve each limit's user, queue, project and host scopes against the corresponding kind of names only, so that a project name never matches a host group and a host-group name in a host scope matches the hosts of that group.
+8. The live Scheduler and the scheduler simulator shall make identical project-limit decisions for the same input.
+
+### Requirement 7: Project visibility
+**Objective:** As a user or admin, I want to see projects, their members and their usage against limits, so that I know which projects I can use and why a job is held.
+
+#### Acceptance Criteria
+1. When a user lists projects through the REST API or `xcecli`, the Cluster shall return every project the user is a member of, with its path name and description, shown in tree order, and mark the project that would be the user's default.
+2. Where the requester has the cluster administrator role, or a `project:view` permission covering a project, the Cluster shall include that project in the list with its members, excluded members and default-for user groups.
+3. When a user requests a project's details, the Cluster shall show, for each limit that applies to the project or to any of its ancestors, the limit name, the project the limit is matched through, the limited resource, the limit value and the current usage counted against it (including descendants' jobs).
+4. If a user requests the details of a project they are not a member of, and they have neither the cluster administrator role nor a `project:view` permission covering the project, then the Cluster shall reject the request without revealing the project's members or usage.
+5. When a user lists jobs filtered by project through the REST API or `xcecli`, the Cluster shall return only jobs with that recorded project, or, when the user asks to include sub-projects, with that project or any descendant, subject to the user's existing job-visibility permissions.
+6. While no project is defined, the Cluster shall report that projects are not configured when projects are listed.
+7. The scheduler's diagnostic interface shall show, for a named project, the limits that apply to it and its current usage.
+
+### Requirement 8: Project-scoped access control
+**Objective:** As a cluster admin, I want to grant a project manager rights over the jobs and details of a project and its sub-projects, so that project leads can manage their project's work without being cluster administrators.
+
+#### Acceptance Criteria
+1. The Cluster shall accept roles that grant a job permission (view, kill, modify or any other existing job permission) with the scope type `project` and a list of project path names as scope values.
+2. The Cluster shall treat a `project`-scoped grant on a project as covering that project and every descendant project.
+3. When a user performs a job operation, the Cluster shall allow it if any of the user's grants for that operation covers the job (scope `all`, `self` for the user's own jobs, or `project` covering the job's recorded project), and shall reject it with a permission error otherwise.
+4. When a user lists jobs, the Cluster shall return exactly the jobs covered by the user's view grants.
+5. The Cluster shall accept roles that grant `project:view` with the scope type `all` or `project`; a `project` scope covers the named projects and their descendants.
+6. If, while projects are defined, a role is created or updated with a `project` scope value that names an undefined project, then the Cluster shall reject the change with an error naming the unknown project.
+7. While a `project` scope value names a project that is no longer defined, the Cluster shall treat that value as covering no project.
+8. The Cluster shall leave the behaviour of existing roles and grants unchanged; the default RBAC configuration shall grant no `project`-scoped permission to any role other than the cluster administrator.
+
+### Requirement 9: Compatibility and durability
+**Objective:** As an operator, I want projects added without data migration or downtime, so that existing jobs and records keep working.
+
+#### Acceptance Criteria
+1. When a job recorded before this feature is recovered, the Cluster shall keep its recorded project name, treating an empty name as `default`.
+2. The Cluster shall accept a cluster configuration without any project definitions and behave as Requirement 4 describes.
+3. When a configuration tool of an earlier version pushes a configuration without project definitions, the Cluster shall treat it as defining no projects.

.kiro/specs/project-config/spec.json

@@ -0,0 +1,22 @@
+{
+  "feature_name": "project-config",
+  "created_at": "2026-10-10T03:25:03Z",
+  "updated_at": "2026-10-10T03:48:51Z",
+  "language": "en",
+  "phase": "requirements-generated",
+  "approvals": {
+    "requirements": {
+      "generated": true,
+      "approved": false
+    },
+    "design": {
+      "generated": false,
+      "approved": false
+    },
+    "tasks": {
+      "generated": false,
+      "approved": false
+    }
+  },
+  "ready_for_implementation": false
+}

.kiro/steering/roadmap.md

@@ -82,6 +82,7 @@ The protocol **drive** is NOT fully event-driven. Root cause: `peer_ctrl_msg_han
 
 ## Future Specs / Backlog (dependency order)
 - [ ] project-guarantee — NEW 2026-10-09 (configuration layering, backlog). Project-owned guarantees (resource amount or host pool), borrowing of idle guaranteed capacity, reclaim through preemption. Brief: `.kiro/specs/project-guarantee/brief.md`. Dependencies: project-config, scheduler-preemption.
+- [ ] project-delegated-membership — NEW 2026-10-10 (from project-config requirements, user decision: future). Project managers edit their project's (and subtree's) members at runtime without an admin config push; needs a durable store that coexists with `xceadmin` config pushes (e.g. a separate delegated member list) and audit. Dependencies: project-config, rbac-persistence.
 - [ ] lsf-config-migration — NEW 2026-10-09 (user idea, backlog; no brief yet). A tool (likely an `xceadmin` subcommand) that converts LSF configuration (`lsb.queues`, `lsb.applications`, `lsb.resources` Limit sections, `lsb.users`/`lsb.hosts` groups, `lsb.params` defaults, project settings) into XCE `configs/*.yaml`, with a report of what does not map: queue `RES_REQ` (XCE queues are policy only; move into a profile or drop), LSF's full per-section merge rules vs XCE's simplified ones, compound/alternative resreqs, unsupported parameters. Dependencies: job-config-merge, application-profile, project-config, parjob-placement (span), admin-config-tool. Promote via `/kiro-discovery` when those land.
 
 - [ ] **xce-harness-component** — **PROMOTED to spec 2026-09-13** (`/kiro-discovery`; brief at `.kiro/specs/xce-harness-component/brief.md`). Elevate `xce_test` (the REPL-driven wire-protocol test client used by HA soak / multi-host / resource-value verification) out of `src/controller/testsuite/` into its own first-class `src/harness/` component — own `meson.build`, `CLAUDE.md`, and unit-test binary (carved out of `controller_unit_test`, which today recompiles ~14 of xce_test's own files a second time paired with white-box tests) — fully decoupled from `/xce-build controller` (today's implicit "controller always builds xce_test" coupling, `build-deployment.md:24`). Verified finding: none of xce_test's 15 source files include any controller-private header — only libxce public headers + generated capnp schema — so this is a pure relocation + build-graph split, not a dependency untangling problem, and no xce_test behavior changes. Touches `/xce-test-ha`, `/xce-deploy-ha`, `/xce-test-multihost`, `/xce-test-resources` (each must build `harness` explicitly once decoupled) and is adjacent to (not overlapping) the three existing behavior specs `xce-test-cli-ux`/`xce-test-e2e-primitives`/`xce-test-rc-file`, whose file-path references go stale after the move. Next: `/kiro-spec-init xce-harness-component`.