4.1 KiB
Brief: gang-groups
(Resource-model discovery, 2026-10-11. Demoted to the backlog 2026-10-12
by user decision N24 in .kiro/steering/resource-questions.md. LLM training
is one parallel job with topology requirements (PR-10 there), not a gang.
This spec is re-scoped as a co-start group of separately managed jobs
and services, to revisit when RL post-training or disaggregated serving
reaches the roadmap.)
Problem
Some workloads are several cooperating parts that must start together, yet each part needs its own identity: application profile or container, owner, lifecycle, or independent scaling. One parallel job cannot express that. Starting only some parts wastes accelerators, and can deadlock the cluster when parts wait on each other.
Use cases (review 2026-10-11, "Gang use cases" in resource-questions.md):
- RL post-training (RLHF, GRPO): a training job, rollout engines (e.g. vLLM), and reward and reference models. Each is a different app; rollouts restart and scale on their own.
- Disaggregated LLM serving with independent pools: prefill and decode
pools that scale independently behind a global router. Fixed xPyD units
are one compound job each, replicated by
service-replica-set(PR-12), and need no gang. - Streaming pipelines: a live producer feeding a consumer (preprocessing into training, online evaluation) with different profiles or owners.
Not this spec:
- data-, tensor- or pipeline-parallel training; MPMD codes; coordinator plus
workers: one parallel job with a compound resreq and topology
requirements (PR-10,
resource-instances,parjob-placement); - a multi-host replica of a served model: one parallel job per replica in
service-replica-set; - a disaggregated serving unit (router, prefill and decode in a fixed ratio): one compound job per unit (PR-12);
- jobs that only need to be placed near each other: placement groups (PR-11), which give co-placement without a co-start.
Current State
- One parallel job is already a gang: all its tasks are placed together
(
parjob-placement), and compound resreqs (lib/xce_resreq.peg:60-83) mix task shapes within it. - There is no all-or-nothing start for a group of separate jobs or services.
Desired Outcome
- Co-start group: a named group whose members are batch jobs and/or service replica sets.
- Placement: all members are placed all-or-nothing, in one session decision or through a reservation that accumulates until complete, with a timeout and release. Placement is packed by topology when requested, reusing PR-11 placement groups.
- Start barrier: no member starts until every member is placed and all their claims are READY.
- Failure policy: after the start, members keep their own lifecycles (restart, scale). When a member fails, the group policy decides: fail the group, restart the member, or tolerate.
- Display: in xcecli/REST, with the group state, member states and the pend reason.
Approach
A group object in the controller; a scheduler co-placement step on top of the allocator framework and placement groups; a group start barrier in the dispatch path.
Scope
- In: group definition, mixing jobs and service replica sets; atomic placement; start barrier; failure policy; display; simulator support.
- Out: RL and serving frameworks themselves; single-job training (PR-10); claim and instance mechanics.
Boundary Candidates
- Group object.
- Scheduler co-placement step.
- Dispatch barrier.
- Failure policy.
Out of Boundary
- Application-level rendezvous (torchrun, Ray).
Upstream / Downstream
- Upstream:
resource-instances,resource-claims,service-replica-set, placement groups (PR-11),scheduler-framework. - Downstream: RL post-training and disaggregated-serving workflows.
Existing Spec Touchpoints
- Adjacent:
job-dependency: ordering, not co-start;scheduler-preemption;service-replica-set: replicas are independent; a co-start group can contain replica sets;job-resize.
Constraints
- Deterministic for sim/replay.
- Group state durable and field-complete in snapshots.
- Full 3-phase spec workflow (HA/persistence).