docs(specs): discovery for long-running service jobs

New briefs: host-failure-job-recovery (jobs on a lost host are requeued or
end EXIT after a grace period; stale copies fenced on rejoin; manual
requeue), service-supervision (profile kind service: restart policy and
liveness/readiness probes run by each replica's own worker, in-place
restart), service-replica-set (controller-held service object owning
independent replica jobs; event-driven replacement, scaling and rolling
update; no head agent, no promotion), service-endpoint (per-replica ports,
endpoints of ready replicas), job-resize (backlog, elastic jobs).

application-profile brief gains the service section; roadmap lists the new
specs, the existing-spec updates and the backlog item.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

2026-10-10 07:15:46 +00:00

2.8 KiB

Raw Blame History

Brief: job-resize

(/kiro-discovery, 2026-10-10, "Long-running services"; backlog, not on the services path: services use independent replica jobs instead (service-replica-set), so this stays an elastic-job feature. LSF precedent: resizable jobs, bresize, autoresizable jobs; Admin Guide p. 339.)

Problem

A running parallel job's allocation is fixed at dispatch. Elastic applications (e.g. elastic MPI or task-farm frameworks) cannot grow onto free hosts or give hosts back without being killed and resubmitted.

Current State

Allocation is decided once by the scheduler and committed at dispatch (scheduler-dispatch-commit); job modify (JOB_MODIFY) is not implemented.
Job arrays have a fixed size; nothing adds or removes tasks of a running job.

Desired Outcome

A running parallel job can grow (request K more tasks; the scheduler places them under the job's constraints, e.g. one per host, and the controller adds them to the allocation and starts them) and shrink (release named tasks or hosts; their processes are stopped first).
A grow request pends until it can be placed and can be cancelled; the job's min/max task range bounds every resize.
Quotas, limits and accounting follow the current allocation.
Requests come from the job's owner (REST/xcecli) and from the job itself (an in-job API for elastic frameworks).
Resizes survive controller failover (facts in the WAL) and the scheduler simulator reproduces them.

Approach

A resize request is a pending allocation request attached to a running job; the scheduler treats a grow like a parallel-job placement of K tasks with the job's span constraints (parjob-placement) and existing hosts excluded or preferred per policy; the controller commits the grown/shrunk allocation and notifies the job's workers.

Scope

In: grow/shrink requests (REST, xcecli, in-job API), scheduling of grow requests, allocation commit and worker launch/stop, accounting, persistence, simulator support.
Out: changing per-task resource sizes (vertical resize); resizing serial jobs; resize-driven autoscaling policies.

Boundary Candidates

Resize request model and API.
Scheduler placement of grow requests.
Controller allocation change and worker task start/stop.

Out of Boundary

Deciding when to resize (the user or the application).

Upstream / Downstream

Upstream: parjob-placement (span constraints), scheduler allocation, host-failure-job-recovery (lost tasks).
Downstream: elastic-job frameworks; none of the service specs.

Existing Spec Touchpoints

Extends: none.
Adjacent: parjob-placement, scheduler-preemption, job-sandbox-multinode (remote task launch).

Constraints

HA/persistence and scheduler equivalence: full 3-phase workflow.

2.8 KiB Raw Blame History