docs(specs): discovery for long-running service jobs

New briefs: host-failure-job-recovery (jobs on a lost host are requeued or
end EXIT after a grace period; stale copies fenced on rejoin; manual
requeue), service-supervision (profile kind service: restart policy and
liveness/readiness probes run by each replica's own worker, in-place
restart), service-replica-set (controller-held service object owning
independent replica jobs; event-driven replacement, scaling and rolling
update; no head agent, no promotion), service-endpoint (per-replica ports,
endpoints of ready replicas), job-resize (backlog, elastic jobs).

application-profile brief gains the service section; roadmap lists the new
specs, the existing-spec updates and the backlog item.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-10 07:15:46 +00:00
parent 253d92340f
commit 664ba565cc
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
7 changed files with 420 additions and 0 deletions

View file

@ -34,6 +34,15 @@ decisions):**
still taken from the profile at dispatch time; requirements decide whether
dispatch freezes them on the job record.
**Update 2026-10-10 (`/kiro-discovery`, long-running services):** a profile
may declare **kind `service`** with a service section: restart policy
(always / on-failure / never, backoff, give-up), liveness and readiness
probes (command / TCP / HTTP), named ports, default replica count, spread
rule and rollout settings. The section is defined with this spec and
consumed by `service-supervision` (worker probes and restarts),
`service-replica-set` (replicas, scaling, rolling update) and
`service-endpoint` (ports, endpoints).
## Problem
XCE users run many kinds of applications: trusted in-house codes, untrusted
scripts, vendor binaries, MPI solvers. Each needs a different way of running:

View file

@ -0,0 +1,86 @@
# Brief: host-failure-job-recovery
(`/kiro-discovery`, 2026-10-10, "Long-running services" phase 1. A
prerequisite for services, and a gap for batch jobs today.)
## Problem
When a host dies, the jobs running on it are never recovered. Users see them
as RUN indefinitely, their resources stay charged, and nothing requeues or
fails them. The only exception is a host that comes back: when its worker
rejoins, the jobs it reports as "unrecognized" are reconciled. Every
long-running service needs its replicas recovered after a host failure, but
batch users are hurt by this already.
## Current State
- `handle_worker_leave()` / `mark_worker_unavail()`
(`src/controller/worker.c:897,921`) mark the host UNAVAIL, record
`dead_since` and set its slots to 0. They do not touch the host's jobs.
- `controller/sched.c:631` says so:
`FIXME: need to handle job status in this worker`.
- Reconciliation exists only when a worker rejoins (`resync.c:692-710`): a
job it reports as "unrecognized" goes back to PEND. The rejoin check times
out after 15 s and then simply ends, leaving the jobs as they are
(`resync.c:259-290`).
- Job states `RUNKWN`/`UNKWN`/`TRANSIENT` are defined (`types.h:382-405`) but
no code sets them.
- `JOB_REQUEUE` exists only as an enum value; the apiserver's requeue control
is commented out (`jobs.go:2366-2382`). There is no rerunnable flag.
## Desired Outcome
- After a configurable grace period with the host unreachable, each job that
ran on it is moved to an "unknown" state and then resolved by policy:
- requeued (PEND, new attempt) if the job is rerunnable (job option or
application profile default);
- otherwise EXIT with a clear reason ("host <h> lost").
- A multi-host job that loses one of its hosts is resolved as a whole (its
surviving tasks are stopped). A service replica is an ordinary job: it is
resolved like any other, and `service-replica-set` replaces it.
- If the host comes back with the job still running after the job was
resolved, the stale copy is killed (fencing by attempt number).
- Resources charged to the lost job are released; limits and quotas recover.
- Users and admins see the lost-host reason and the attempt count;
`xcecli`/REST can requeue a job manually (`JOB_REQUEUE`).
- Behaviour is identical after controller failover (facts in the WAL).
## Approach
Controller-side, primary-only, trigger-only timer (delaymgr rule:
no blocking) that walks jobs of hosts unavailable longer than the grace
period, commits a "job lost" fact per job and applies the policy through the
existing commit-then-apply seam. The worker's rejoin resync gains a check
that kills jobs whose attempt number is stale.
## Scope
- **In**: grace period config; unknown state; rerunnable flag (job and
profile default) and requeue-on-host-loss; EXIT with reason otherwise;
multi-host job handling; attempt numbers and stale-copy fencing on rejoin;
manual requeue (REST, `xcecli`); display; WAL/snapshot field completeness;
sim/live scheduler consistency for released resources.
- **Out**: requeue on job exit codes (`REQUEUE_EXIT_VALUES`, possible
follow-up); checkpoint/restart migration; replacing service replicas
(`service-replica-set`).
## Boundary Candidates
- Host-loss detection and grace period (controller).
- Job-lost policy (requeue / exit) and attempt numbering.
- Stale-copy fencing on worker rejoin.
- Manual requeue control (apiserver, xcecli).
## Out of Boundary
- Restarting a process on the same host (`service-supervision`).
- Keeping a service's replica count (`service-replica-set`).
## Upstream / Downstream
- **Upstream**: worker liveness (`worker.c`), worker rejoin resync
(`worker-rejoin-resync`), commit/apply seam (`raft-apply-decoupling`).
- **Downstream**: `service-supervision`, `service-replica-set`,
`job-dependency` (lost jobs end as EXIT).
## Existing Spec Touchpoints
- **Extends**: none.
- **Adjacent**: `worker-rejoin-resync`, `controller-job-cleanup`,
`scheduler-preemption` (requeue semantics).
## Constraints
- HA/persistence: full 3-phase workflow; additive Cap'n Proto; the new facts
round-trip through WAL and snapshot (`/xce-audit-invariants`).
- The timer must never block the delaymgr (`ha-async-io.md`).

View file

@ -0,0 +1,66 @@
# Brief: job-resize
(`/kiro-discovery`, 2026-10-10, "Long-running services"; **backlog, not on
the services path**: services use independent replica jobs instead
(`service-replica-set`), so this stays an elastic-job feature. LSF
precedent: resizable jobs, `bresize`, autoresizable jobs; Admin Guide p. 339.)
## Problem
A running parallel job's allocation is fixed at dispatch. Elastic
applications (e.g. elastic MPI or task-farm frameworks) cannot grow onto
free hosts or give hosts back without being killed and resubmitted.
## Current State
- Allocation is decided once by the scheduler and committed at dispatch
(`scheduler-dispatch-commit`); job modify (`JOB_MODIFY`) is not
implemented.
- Job arrays have a fixed size; nothing adds or removes tasks of a running
job.
## Desired Outcome
- A running parallel job can **grow** (request K more tasks; the scheduler
places them under the job's constraints, e.g. one per host, and the
controller adds them to the allocation and starts them) and **shrink**
(release named tasks or hosts; their processes are stopped first).
- A grow request pends until it can be placed and can be cancelled; the
job's min/max task range bounds every resize.
- Quotas, limits and accounting follow the current allocation.
- Requests come from the job's owner (REST/xcecli) and from the job itself
(an in-job API for elastic frameworks).
- Resizes survive controller failover (facts in the WAL) and the scheduler
simulator reproduces them.
## Approach
A resize request is a pending allocation request attached to a running job;
the scheduler treats a grow like a parallel-job placement of K tasks with
the job's span constraints (`parjob-placement`) and existing hosts excluded
or preferred per policy; the controller commits the grown/shrunk allocation
and notifies the job's workers.
## Scope
- **In**: grow/shrink requests (REST, xcecli, in-job API), scheduling of
grow requests, allocation commit and worker launch/stop,
accounting, persistence, simulator support.
- **Out**: changing per-task resource sizes (vertical resize); resizing
serial jobs; resize-driven autoscaling policies.
## Boundary Candidates
- Resize request model and API.
- Scheduler placement of grow requests.
- Controller allocation change and worker task start/stop.
## Out of Boundary
- Deciding when to resize (the user or the application).
## Upstream / Downstream
- **Upstream**: `parjob-placement` (span constraints), scheduler allocation,
`host-failure-job-recovery` (lost tasks).
- **Downstream**: elastic-job frameworks; none of the service specs.
## Existing Spec Touchpoints
- **Extends**: none.
- **Adjacent**: `parjob-placement`, `scheduler-preemption`,
`job-sandbox-multinode` (remote task launch).
## Constraints
- HA/persistence and scheduler equivalence: full 3-phase workflow.

View file

@ -0,0 +1,59 @@
# Brief: service-endpoint
(`/kiro-discovery`, 2026-10-10, "Long-running services" phase 2.)
## Problem
Clients of a service need a stable way to reach its ready replicas. Today
users only see exec host names; there are no ports, no host addresses and no
registry, so every service invents its own discovery.
## Current State
- No port or network resource, endpoint, DNS or load-balancing concept;
`xce_worker_t` has no IP field and the host API marks `ip_addr` as
unsupported (`models/hosts.go:43`).
- Readiness will be reported by workers (`service-supervision`).
## Desired Outcome
- A service profile declares named ports (fixed or "any free"); each replica
is allocated its ports on its host like a resource, so two replicas never
collide; the job sees its ports in environment variables.
- Workers report host addresses; a service has an **endpoint**: its name
resolves to the list of `host:port` of ready replicas (readiness from
`service-supervision`), updated as replicas become ready or not.
- Users and clients read endpoints through REST and `xcecli` (and a simple
query interface usable from scripts); access follows RBAC (job view
scope).
- Out of v1 but kept possible: DNS records, a built-in proxy or load
balancer.
## Approach
Ports become a per-host consumable resource the scheduler allocates;
readiness changes already reach the controller; the controller (or the
apiserver from pushed state) maintains the endpoint table per service.
## Scope
- **In**: port declarations and allocation, host address reporting,
endpoint table from readiness, REST/xcecli endpoint queries, RBAC.
- **Out**: DNS, proxy/load balancing, TLS, ingress from outside the
cluster.
## Boundary Candidates
- Port resource and allocation (scheduler, worker).
- Host address reporting.
- Endpoint table and query surfaces.
## Out of Boundary
- Health probing itself (`service-supervision`).
## Upstream / Downstream
- **Upstream**: `service-supervision` (readiness), `application-profile`
(port declarations), resource model (`resource-accounting.md`).
- **Downstream**: `service-replica-set` (a service's endpoint covers all its
replica jobs; rollouts keep it populated).
## Existing Spec Touchpoints
- **Extends**: `application-profile` (ports in the service section).
- **Adjacent**: `resource-distribution-mapping`.
## Constraints
- Endpoint updates are change-driven; no per-probe traffic.

View file

@ -0,0 +1,100 @@
# Brief: service-replica-set
(`/kiro-discovery`, 2026-10-10, "Long-running services" phase 2. Replaces
the earlier `service-head-agent` idea by user decision: no special replica,
no head, no promotion. Kubernetes analogue: Deployment/ReplicaSet with
independent pods.)
## Problem
A service with N replicas needs N healthy replicas kept running: a replica
that gave up or was lost with its host must be replaced elsewhere, the
count must be changeable, and a new version must roll out without
downtime. Per-replica supervision (`service-supervision`) cannot do this
alone because no worker sees the other replicas.
## Current State
- Phase 1 gives per-replica probes and in-place restart on each worker
(`service-supervision`) and host-loss recovery that ends or requeues a
job (`host-failure-job-recovery`).
- There is no service object, no replica grouping and no rollout concept;
job arrays are fixed-size.
## Desired Outcome
- A **service** is a controller-held object: name, owner, project, queue,
application profile (kind `service`), desired replica count, version
(the profile/command revision it runs) and the list of its **replica
jobs**.
- **Replicas are independent, ordinary jobs.** Each is scheduled, limited,
charged, recovered and RBAC-checked like any job; the scheduler spreads a
service's replicas across hosts (anti-affinity rule from the profile,
e.g. at most one per host, best effort or strict).
- **No replica is special**: each replica's worker checks it
(`service-supervision`); there is no head, no promotion, no epoch.
- **Event-driven rules in the controller** (no reconcile loop, no new
timers on the delaymgr), applied when a fact commits:
- a replica job ends (gave up, lost with its host, killed outside the
service) while the service wants more → submit a replacement replica
job, with backoff when replacements keep failing (a crash-looping
service pends with a reason instead of churning);
- the desired count changes (scale through REST/xcecli) → submit or kill
replica jobs (scale-down picks not-ready replicas first, then newest);
- **rolling update** (new version) → replace replicas in batches limited
by max-unavailable / max-surge, each step advancing when the new
replicas report ready; pause, resume and rollback; a progress deadline
marks a stuck rollout.
- Deleting a service kills its replicas; killing one replica directly just
triggers its replacement.
- Users see service state: desired / running / ready counts, version,
rollout progress, last events; `xcecli services` commands and REST.
- Everything survives controller failover (service object and rollout state
are WAL facts); the scheduler simulator handles replica jobs as jobs.
## Approach
A new controller entity with its own WAL facts and snapshot frames; rules
attached to the existing commit-then-apply seam (replica end, readiness
change, desired-count change, version change). Replica jobs carry their
service name and replica index. Considered and set aside: a head agent per
service (needs promotion and fencing) and a separate cluster-wide service
manager (one more HA component).
## Scope
- **In**: service object and API (create, scale, update, pause/resume,
rollback, delete, list/show) in REST, xcecli and libxce; replica jobs
linked to the service; replacement with backoff; scaling; rolling update;
spread rule (with the scheduler); display; persistence (WAL, snapshot,
field completeness); RBAC on services (follows job permissions and
project scopes).
- **Out**: metric-based autoscaling; canary/blue-green beyond rolling
update; endpoints (`service-endpoint`); per-replica probes
(`service-supervision`); elastic resize of one job (`job-resize`).
## Boundary Candidates
- Service entity, facts and API.
- Replacement and scaling rules.
- Rollout state machine.
- Spread placement rule (scheduler).
- Display surfaces.
## Out of Boundary
- How a replica is probed or restarted on its host.
- Endpoint table mechanics.
## Upstream / Downstream
- **Upstream**: `service-supervision` (readiness, gave-up events),
`host-failure-job-recovery` (lost replicas), `application-profile`,
`job-config-merge`, `project-config` (quotas cap replicas).
- **Downstream**: `service-endpoint` (the endpoint covers all replica
jobs), future autoscaling.
## Existing Spec Touchpoints
- **Extends**: `application-profile` (default replicas, spread rule,
rollout settings in the service section).
- **Adjacent**: `job-array-single-wal-record` (grouped submission
precedent), `controller-job-cleanup` (finished replica jobs).
## Constraints
- Rules run inside the commit/apply path and must stay bounded and
non-blocking (`ha-async-io.md`); replacement backoff without new delaymgr
work beyond trigger-only timers.
- HA/persistence: full 3-phase workflow; `/xce-audit-invariants` for the new
entity.

View file

@ -0,0 +1,90 @@
# Brief: service-supervision
(`/kiro-discovery`, 2026-10-10, "Long-running services" phase 1.)
## Problem
Users and admins want to run long-lived services on XCE next to batch work:
model servers, notebooks, databases and web apps for users; license servers,
workflow engines and monitoring agents for admins. Today a job that exits or
hangs simply ends or keeps running unnoticed: nothing restarts it and nothing
checks that it still works.
## Current State
- A job runs in a systemd transient unit `xcecd-worker-<jobid>` with no
`Restart=` (`src/worker/src/xcecd_cgroups.c`); the `xcecd_worker` process
forks the job and reports its exit (`xcecd_worker.c:467-523`); the
controller ends it DONE or EXIT (`job.c:2937-2951`).
- No health, liveness or readiness checks exist.
- Nothing kills an indefinitely running job unless the user sets `-W`
(`RuntimeMaxUSec` only when > 0), so running forever already works.
- A running job writes a usage WAL record every 10 s; checkpoints keep the
WAL bounded (no growth with job age).
## Desired Outcome
- An application profile can declare **kind `service`** with:
- a restart policy: `always` / `on-failure` / `never`, backoff (initial,
maximum), and a give-up rule (max restarts in a window);
- a liveness probe: command, TCP connect or HTTP GET, with interval,
timeout, failure threshold and initial delay;
- a readiness probe (same forms), reported but not yet used for routing
(`service-endpoint`);
- whether the service is rerunnable after host loss (default yes,
`host-failure-job-recovery`).
- The worker supervises each service task on its host: runs the probes
locally, restarts the process in place on exit or failed liveness (keeping
its allocation), applies backoff, and gives up per policy (the job then
ends EXIT with the reason, or is requeued if the policy says so).
- The worker reports only changes to the controller: ready / not ready,
restarted (count, last reason), gave up. Users see state, restart count
and last failure in job details and `xcecli`.
- Stopping a service is the normal job kill; a service never ends DONE on
its own unless its policy is `never` and it exits 0.
- Service jobs are excluded from no-op expectations of batch jobs: no run
limit by default, not cleaned up while running.
## Approach
Supervision lives in `xcecd_worker` (already the job's parent): a restart
loop around the launch with backoff, probe timers on the worker's event
loop, and change-only status reports over the existing worker→controller
channel. The profile's service section flows to the worker through the job's
effective values (`job-config-merge`). Using systemd `Restart=` was
considered and rejected for v1: it would rerun `xcecd_worker` itself
(including profile pre-exec) and hide restarts from XCE.
## Scope
- **In**: profile service section (kind, restart policy, probes, rerunnable);
worker restart loop and probes; change-only health reports and their
persistence; display (state, restarts, last failure); xcecli/REST
submission of service jobs through profiles; tests incl. `xcecd_virtual`
simulation of probes and restarts.
- **Out**: replicas, scaling, replacement on other hosts, rolling update
(`service-replica-set`); endpoints and ports
(`service-endpoint`); host failure (`host-failure-job-recovery`).
## Boundary Candidates
- Profile schema for services (with `application-profile`).
- Worker supervision loop and probe runner.
- Health/restart reporting and persistence (controller).
- Display surfaces.
## Out of Boundary
- Any decision that needs a cluster-wide view (more replicas, new hosts).
- No replica is special: every replica's own worker checks it (user
decision 2026-10-10), so there is no head and no promotion.
## Upstream / Downstream
- **Upstream**: `application-profile`, `job-config-merge` (effective
values), `host-failure-job-recovery` (host loss), worker launch path.
- **Downstream**: `service-replica-set` (replaces replicas that gave up,
gates rollouts on readiness), `service-endpoint` (readiness).
## Existing Spec Touchpoints
- **Extends**: `application-profile` (service section of the profile).
- **Adjacent**: `job-sandbox-execution` (execution backends must support the
restart loop), `controller-job-cleanup`.
## Constraints
- Health reports must be change-only (no per-probe WAL traffic).
- Real `xcecd` needs systemd/dbus to verify; `xcecd_virtual` must simulate
probe outcomes and restarts for CI.
- Persistence of health state: field completeness (`persistence.md`).

View file

@ -73,6 +73,15 @@ The protocol **drive** is NOT fully event-driven. Root cause: `peer_ctrl_msg_han
- [ ] job-config-merge — NEW 2026-10-09 (`/kiro-discovery`, Path E "configuration layering"; ref LSF 10.1 Admin Guide pp. 314-316, 435-437, 561-563, 643, 660). One merge function in `src/lib` turning the job's own values + application profile + queue policy + project into effective values (resreq incl. `span`, task range, limits, pre/post-exec, env). One owner per attribute (user decision; `.kiro/steering/job-config-layering.md`): job overrides a profile attribute only if it is *default* (for `select` only by adding ANDed terms); queue ceilings and project quotas only check (reject if never satisfiable, pend if full now). **Live** merge (user decision): WAL keeps inputs + names; controller/apiserver validate at submit; scheduler computes effective values and re-evaluates pending jobs on config change. Queue is policy only: queue `res_req` and `pre_exec`/`post_exec` deprecated (pre/post-exec moves to the profile), dropped queue limits (cpulimit/memlimit/runlimit/task limit) wired. Brief: `.kiro/specs/job-config-merge/brief.md`. Dependencies: none.
- [ ] project-config — NEW 2026-10-09 (configuration layering). Projects control resource ownership, v1 = quota: project definitions in cluster config (members, priority, cluster default), default-project resolution, submit access check, scheduler `pgmgr` loaded from config so `PROJECTS`/`PER_PROJECT` limits work end to end, per-project usage display. Brief: `.kiro/specs/project-config/brief.md`. Dependencies: none.
- [ ] host-failure-job-recovery — NEW 2026-10-10 (`/kiro-discovery`, "Long-running services" phase 1; also a batch gap: RUN jobs on a dead host stay RUN forever, `controller/sched.c:631` FIXME). After a grace period, jobs on a lost host go to an unknown state, then are requeued (rerunnable, job option or profile default) or end EXIT "host lost"; multi-host jobs resolved as a whole; attempt numbers fence a stale copy when the host returns; manual requeue (REST/xcecli). Brief: `.kiro/specs/host-failure-job-recovery/brief.md`. Dependencies: none.
- [ ] service-supervision — NEW 2026-10-10 (Long-running services phase 1). Application profile kind `service`: restart policy (always/on-failure/never, backoff, give-up), liveness and readiness probes (command/TCP/HTTP) run by each replica's own worker (no special replica), in-place restart, change-only health reports, display. Brief: `.kiro/specs/service-supervision/brief.md`. Dependencies: application-profile, host-failure-job-recovery.
- [ ] service-replica-set — NEW 2026-10-10 (Long-running services phase 2; user decision: no head agent, no promotion). Controller-held service object owning independent replica jobs; event-driven rules on commit (replace ended replicas with backoff, scale, rolling update with max-unavailable/surge, pause/resume/rollback); spread rule; REST/xcecli `services`. Brief: `.kiro/specs/service-replica-set/brief.md`. Dependencies: service-supervision, host-failure-job-recovery, application-profile.
- [ ] service-endpoint — NEW 2026-10-10 (Long-running services phase 2). Named ports allocated per replica as host resources, host addresses reported by workers, endpoint = `host:port` of ready replicas from readiness changes, REST/xcecli queries; DNS/proxy later. Brief: `.kiro/specs/service-endpoint/brief.md`. Dependencies: service-supervision, service-replica-set.
## Existing Spec Updates (2026-10-10 long-running services discovery)
- [ ] application-profile — profile kind `service` section: restart policy, probes, ports, default replicas, spread rule, rollout settings (consumed by service-supervision, service-replica-set, service-endpoint).
- [ ] job-config-merge — service attributes follow the ownership rule (profile owns them; the job may override only `default` ones, e.g. replica count).
## Existing Spec Updates (2026-10-09 configuration layering discovery)
- [ ] application-profile — the profile carries the job's resource requirement (select/rusage/`span`/same) and task limits (min/default/max); "resolved and frozen at submit" replaced by the live merge of `job-config-merge` (brief updated). Dependencies: job-config-merge.
- [ ] parjob-placement — `span` read from the effective resreq; span constraints in the fixed-N planner; range search bisects over the planner's allowed counts (block: multiples; ptile: multiples + max); LSF 10.1 findings recorded in its requirements.md notes; `span[stripe]` candidate. Dependencies: job-config-merge (for `span` from profiles; job-level `-R` span can land first).
@ -82,6 +91,7 @@ The protocol **drive** is NOT fully event-driven. Root cause: `peer_ctrl_msg_han
## Future Specs / Backlog (dependency order)
- [ ] project-guarantee — NEW 2026-10-09 (configuration layering, backlog). Project-owned guarantees (resource amount or host pool), borrowing of idle guaranteed capacity, reclaim through preemption. Brief: `.kiro/specs/project-guarantee/brief.md`. Dependencies: project-config, scheduler-preemption.
- [ ] job-resize — NEW 2026-10-10 (from long-running services discovery; not on the services path). Grow/shrink a running parallel job's allocation (LSF resizable jobs, `bresize`) for elastic applications. Brief: `.kiro/specs/job-resize/brief.md`. Dependencies: parjob-placement, host-failure-job-recovery.
- [ ] project-delegated-membership — NEW 2026-10-10 (from project-config requirements, user decision: future). Project managers edit their project's (and subtree's) members at runtime without an admin config push; needs a durable store that coexists with `xceadmin` config pushes (e.g. a separate delegated member list) and audit. Dependencies: project-config, rbac-persistence.
- [ ] lsf-config-migration — NEW 2026-10-09 (user idea, backlog; no brief yet). A tool (likely an `xceadmin` subcommand) that converts LSF configuration (`lsb.queues`, `lsb.applications`, `lsb.resources` Limit sections, `lsb.users`/`lsb.hosts` groups, `lsb.params` defaults, project settings) into XCE `configs/*.yaml`, with a report of what does not map: queue `RES_REQ` (XCE queues are policy only; move into a profile or drop), LSF's full per-section merge rules vs XCE's simplified ones, compound/alternative resreqs, unsupported parameters. Dependencies: job-config-merge, application-profile, project-config, parjob-placement (span), admin-config-tool. Promote via `/kiro-discovery` when those land.