docs(steering): one owner per job attribute; pre/post-exec moves to the app profile

New steering job-config-layering.md: queue = scheduling policy, application
profile = what the application needs and how it runs (resreq, task range,
execution settings, pre/post-exec, limits), project = resource ownership.
Only two interactions: the job overrides a profile attribute marked
default (fixed: not at all; select: only ANDed terms), and queue ceilings
or project quotas check effective values (reject if never satisfiable,
pend if full now). Replaces the simplified job > app > queue rules.

Queue pre_exec/post_exec are deprecated with queue res_req: the profile's
pre/post-exec fills the worker's existing outer layer, the job's own stays
the inner layer, allowed or forbidden by the profile.

job-config-merge and application-profile briefs and the roadmap follow;
CLAUDE.md lists the new steering file.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

2026-10-10 03:19:20 +00:00

4.7 KiB

Raw Blame History

Job Configuration Layering

How a job's settings are split between the job, its application profile, its queue and its project, and how they interact. Decided 2026-10-09 (/kiro-discovery, configuration layering). Specs that follow this: job-config-merge, application-profile, project-config, project-guarantee, parjob-placement, lsf-config-migration.

Why not LSF's rules

LSF lets the queue, the application profile and the job set the same attributes (RES_REQ, limits, pre/post-exec, job starter, ...) and then needs a rule per section to decide which wins (select ANDed, order/span overridden, rusage merged with the queue as ceiling, two-stage compound/alternative merges, queue pre-exec before app pre-exec, ...; Admin Guide pp. 314-316, 435-437, 581). The roles of queue, application profile and project overlap. XCE instead gives every attribute exactly one owning level.

Ownership

Level	Owns	Never owns
Queue: scheduling policy	priority and ordering, host partition, dispatch/run windows, preemption policy, policy ceilings (max run time, max tasks, max hosts, ...)	resource requirement, anything about how a job executes (pre/post-exec, env, launch, limits enforced on the job)
Application profile: what the application needs and how it runs	resreq (select / rusage / span / same / order), task range (min / default / max), execution mode and options, launch type, env, pre/post-exec, execution limits (mem, runtime, CPU, tasks)	priority, queue choice, quotas
Project: resource ownership	membership, quotas / limits, later guarantees, accounting	how a job runs or is ordered
Job	its own values, only for attributes its profile allows	n/a

Access to each is separate: queue and profile by RBAC permission, project by membership (project-config). Any allowed queue combines with any allowed profile and project.

The only two interactions

Override: job vs. application profile. Each profile attribute is either
- default: the job's value replaces it, or
- fixed: the job cannot change it. Exception: for select, the job may add terms (narrow the host set, ANDed) but never drop the profile's.
Ceiling: queue or project vs. the job's effective values. A queue ceiling or project quota never changes the job's values, it only checks them:
- never satisfiable (runtime above the queue's maximum, task range above its max tasks): rejected at submit, never silently clipped;
- not satisfiable right now (project quota in use): the job pends with a reason naming the queue or project.

There is no third case: no queue resreq, no rusage merge, no ordering of queue against profile settings, no compound/alternative merge stages.

Pre/post-exec

Moves from the queue to the application profile. Queue pre_exec / post_exec are deprecated (xceadmin config build warns, then rejects).
Two layers remain, each with one owner:
1. Profile pre/post-exec: admin-defined, always runs (fixed); the job cannot replace or skip it.
2. Job pre/post-exec: the user's own (start_info today); allowed or forbidden by the profile (e.g. a sandbox profile may forbid it).
Order: profile pre → job pre → job → job post → profile post. A failed profile pre-exec skips the rest; profile post-exec still runs for cleanup.
The worker already runs exactly two layers (queue_pre_exec() / queue_post_exec(), worker/src/xcecd_worker.c; stages JOB_EXEC_STAGE_Q_PRE_EXEC / Q_POST_EXEC), so the worker keeps its structure; the controller fills the outer layer from the profile instead of the queue (get_queue_exec(), controller/queue.c:199, called at dispatch controller/job.c:1827) and the stages are renamed.

Timing

Live (job-config-merge): the WAL keeps the job's own values plus the names of its queue, profile and project. The controller and apiserver validate at submit; the scheduler computes effective values and re-evaluates pending jobs when config changes; running jobs are unaffected. Execution-side profile values (pre/post-exec, env, mode) are read at dispatch, as queue pre/post-exec is today.

Edge cases

Overlapping limits: the profile's runtime limit (what the application needs, enforced on the job) and the queue's max run time (policy ceiling) stay separate; the ceiling checks the profile/job value.
Site-wide host restrictions for a queue use the queue's host partition, not a resreq.
LSF migration (lsf-config-migration): an LSF queue RES_REQ that only selects hosts becomes a queue host partition; anything else moves into a profile or is reported. LSF queue PRE_EXEC/POST_EXEC move into profiles.

4.7 KiB Raw Blame History