docs(spec): split execution modes from application-profile into job-sandbox-execution
- application-profile keeps the profile core: config, access and submit, dispatch record, outer pre/post-exec layer; mode: accepts only direct - sandbox mode + options, I/O directories, capability advertisement and placement by mode move to job-sandbox-execution - order: application-profile requirements/design before job-config-merge design; implementation interleaves Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
8900f8876b
commit
d1e03055e4
3 changed files with 60 additions and 29 deletions
|
|
@ -45,6 +45,18 @@ consumed by `service-supervision` (worker probes and restarts),
|
|||
`service-endpoint` (ports, endpoints).
|
||||
|
||||
**Update 2026-10-12 (discussion; user decisions):**
|
||||
- **Split: execution modes move to `job-sandbox-execution`.** This spec is
|
||||
the profile core: config, access and submit, job state (name, dispatch
|
||||
record), the outer pre/post-exec layer. Its schema has `mode:` but
|
||||
accepts only `direct`. The `sandbox` mode with its options, I/O
|
||||
directories, worker capability advertisement and scheduler placement by
|
||||
mode belong to `job-sandbox-execution`, since nothing uses a non-`direct`
|
||||
mode before the sandbox exists. The worker's clear launch error for a
|
||||
missing `direct` exec user stays here.
|
||||
- **Order with `job-config-merge`:** requirements and design of this spec
|
||||
first (it fixes the profile schema), then `job-config-merge`'s design;
|
||||
implementation tasks interleave (schema/config, merge library, submit
|
||||
validation and dispatch record).
|
||||
- **Own configuration file.** Profiles live in a new
|
||||
`src/configs/applications.yaml` with its own JSON schema, mapped into
|
||||
`ClusterConfig` and `Conf4Sched` like queues and limits. See "Profile
|
||||
|
|
@ -144,9 +156,9 @@ fits XCE's LSF-style job model.
|
|||
- **Admins define named application profiles** in the cluster configuration. The
|
||||
first version contains the following (user decision; the format must be
|
||||
extensible for "maybe others"):
|
||||
1. **Execution mode and its options.** `direct` or `sandbox` now, with room for
|
||||
`container`/`vm`. Sandbox options include the allowed input/output
|
||||
prefixes and isolation extras.
|
||||
1. **Execution mode.** The `mode:` field, accepting only `direct` in this
|
||||
spec; `job-sandbox-execution` adds `sandbox` with its options (allowed
|
||||
input/output prefixes, isolation extras), later `container`/`vm`.
|
||||
2. **Launch type.** Serial, `openmpi` or `mpich`.
|
||||
3. **Environment and pre/post-exec.** Environment variables the application
|
||||
needs, and the outer pre/post-exec commands.
|
||||
|
|
@ -210,9 +222,9 @@ fits XCE's LSF-style job model.
|
|||
user doesn't exist on the node, the worker fails the job at launch with a
|
||||
clear, user-visible reason, instead of today's opaque systemd failure.
|
||||
- Account-less users are served by profiles whose mode is `sandbox`.
|
||||
- **Placement.** A job is placed only on workers whose advertised capabilities
|
||||
support its effective mode (sandbox needs systemd 242 or later and
|
||||
`nss-systemd`).
|
||||
- **Placement by mode** (moved to `job-sandbox-execution`): a job is placed
|
||||
only on workers whose advertised capabilities support its mode (sandbox
|
||||
needs systemd 242 or later and `nss-systemd`).
|
||||
- **Ownership.** The submitter remains the owner for RBAC, limits and accounting
|
||||
in every mode.
|
||||
|
||||
|
|
@ -248,9 +260,9 @@ fits XCE's LSF-style job model.
|
|||
launch type, env, profile pre/post-exec) are read from the profile at
|
||||
dispatch, as queue pre/post-exec is today (`get_queue_exec()`,
|
||||
`controller/queue.c:199`).
|
||||
- **Workers advertise their supported execution modes.** The scheduler filters
|
||||
hosts by the job's effective mode. The design picks the mechanism, for example
|
||||
a host attribute or boolean resource.
|
||||
- **Workers advertise their supported execution modes** and the scheduler
|
||||
filters hosts by mode: `job-sandbox-execution` (the design picks the
|
||||
mechanism, e.g. a host attribute or boolean resource).
|
||||
|
||||
### Profile configuration
|
||||
- **One file per entity type**, as for hosts, queues, resources, limits,
|
||||
|
|
@ -527,9 +539,13 @@ Decided 2026-10-12 (user), so a rename never strands jobs:
|
|||
- RBAC permission for profile use.
|
||||
- Submit surfaces: REST, `xcecli --app`, harness, libxce.
|
||||
- The profile name and the job's own values persisted on the job.
|
||||
- Execution-mode and I/O-directory fields end to end.
|
||||
- Capability advertisement and placement.
|
||||
- The `mode:` schema field (only `direct`).
|
||||
- A worker launch error when a `direct` exec user is missing.
|
||||
- Dispatch record (scheduling and execution values, content hash) and the
|
||||
retryable dispatch failure for a missing or changed profile.
|
||||
- Profile pre/post-exec as the outer layer (moved from the queue), with
|
||||
a clean environment; the reserved `XCE_` prefix.
|
||||
- Aliases, live-upgrade warning, pend reason for a missing profile.
|
||||
- Profile name and mode shown in job details.
|
||||
- Listing available profiles (REST and xcecli).
|
||||
- Profile `--dry-run` support: the effective values a job would get
|
||||
|
|
@ -552,12 +568,12 @@ Decided 2026-10-12 (user), so a rename never strands jobs:
|
|||
the outer layer and reserved `XCE_` instead of an env denylist; moving
|
||||
pending jobs deferred to `job-modify`, profile aliases in v1; override granularity (no markers; additive rule),
|
||||
limits a job may lower (profile limits are maximums), personal presets
|
||||
(client side), `app:override` and inheritance (deferred).
|
||||
- Splitting the spec: execution mode, I/O directories, capability
|
||||
advertisement and placement as a separate spec (or folded into
|
||||
`job-sandbox-execution`), leaving this spec the profile core.
|
||||
(client side), `app:override` and inheritance (deferred); execution modes
|
||||
split into `job-sandbox-execution`.
|
||||
- **Out**:
|
||||
- The worker sandbox mechanics (`job-sandbox-execution`).
|
||||
- The `sandbox` mode and its options, I/O directories, capability
|
||||
advertisement, placement by mode and the worker sandbox mechanics
|
||||
(`job-sandbox-execution`).
|
||||
- Multi-node sandbox (`job-sandbox-multinode`).
|
||||
- Container or vm execution itself (only room for them in the schema).
|
||||
- Auth backends and JWT claims.
|
||||
|
|
@ -569,7 +585,6 @@ Decided 2026-10-12 (user), so a rename never strands jobs:
|
|||
- Submit and listing surfaces (REST, xcecli, harness, libxce).
|
||||
- Persisting the profile name and the job's own values (wire, WAL submit
|
||||
fact, snapshot).
|
||||
- Capability advertisement and scheduler placement.
|
||||
|
||||
## Out of Boundary
|
||||
- Worker execution backends and sandbox mechanics.
|
||||
|
|
@ -587,9 +602,10 @@ Decided 2026-10-12 (user), so a rename never strands jobs:
|
|||
- The config pipeline (`clusterconfig-schema-split`, `config-upgrade-snapshot`,
|
||||
`admin-config-tool`).
|
||||
- RBAC (`rbac_check.go`).
|
||||
- Resource/attribute reporting from worker to scheduler.
|
||||
- **Downstream**:
|
||||
- `job-sandbox-execution` consumes the effective mode, options and directories.
|
||||
- `job-sandbox-execution` adds the `sandbox` mode, its options and I/O
|
||||
directories, capability advertisement and placement, on top of this
|
||||
spec's `mode:` field and dispatch record.
|
||||
- `job-sandbox-multinode`.
|
||||
- Future `container`/`vm` modes, which add profile attributes.
|
||||
- `job-status-detail-display`, which can show the profile and mode.
|
||||
|
|
@ -618,6 +634,6 @@ Decided 2026-10-12 (user), so a rename never strands jobs:
|
|||
- Input validation (`security.md`): the job's additions and paths are validated
|
||||
server-side. No path traversal, and mount prefixes are restricted by the
|
||||
profile.
|
||||
- Size: this is likely 20 or more tasks. If requirements confirm that, split
|
||||
as listed under "Open for requirements".
|
||||
- Size: still large after the split (config, access and submit, job state,
|
||||
outer layer); one coherent feature, testable end to end with `direct`.
|
||||
- HA/persistence spec: use the full 3-phase workflow.
|
||||
|
|
|
|||
|
|
@ -1,5 +1,16 @@
|
|||
# Brief: job-sandbox-execution
|
||||
|
||||
**Update 2026-10-12 (application-profile split; user decision):** this spec
|
||||
now also owns the submission side of execution modes, moved out of
|
||||
`application-profile` (whose schema keeps `mode:` accepting only `direct`):
|
||||
the `sandbox` mode value and its options (allowed input/output prefixes,
|
||||
isolation extras), the job's I/O-directory fields end to end (wire, WAL
|
||||
submit fact and snapshot, field completeness), worker capability
|
||||
advertisement, and scheduler placement by mode. The mode and options reach
|
||||
the worker through `application-profile`'s dispatch record. Sandbox-specific
|
||||
env exposure is here; the outer pre/post-exec clean environment and the
|
||||
reserved `XCE_` prefix are `application-profile`'s.
|
||||
|
||||
## Problem
|
||||
A job whose execution mode is `sandbox` (from `application-profile`) must run on a
|
||||
worker node:
|
||||
|
|
@ -103,7 +114,8 @@ Code exploration, 2026-09-29.
|
|||
- **Capability**:
|
||||
- The worker detects its sandbox capability (systemd 242 or later, `systemd`
|
||||
in `nsswitch.conf` passwd/group, and D-Bus property support) and advertises
|
||||
it to `application-profile`'s placement.
|
||||
it; the scheduler places sandbox jobs only on capable workers (this
|
||||
spec).
|
||||
- It refuses, with a clear error, a sandbox job it cannot honour.
|
||||
- **Direct mode is unchanged.**
|
||||
|
||||
|
|
@ -150,11 +162,14 @@ Viability check (2026-09-29, systemd NEWS, systemd.exec(5), v239 sources):
|
|||
- Input/output mounts.
|
||||
- `StandardOutput=file:` output and retrieval.
|
||||
- Helper authentication by unit or cgroup.
|
||||
- Capability detection and advertisement.
|
||||
- Capability detection and advertisement, and scheduler placement by mode.
|
||||
- The `sandbox` mode value and options in the profile schema, and the
|
||||
job's I/O-directory fields end to end (wire, WAL, snapshot).
|
||||
- Single-node MPI in one unit.
|
||||
- Tests: unit tests plus a live test on a systemd 242+ node.
|
||||
- **Out**:
|
||||
- The submission side: mode field, policy, placement (`application-profile`).
|
||||
- The profile core (`application-profile`): config, `mode:` field,
|
||||
access, dispatch record.
|
||||
- Multi-node sandbox launch (`job-sandbox-multinode`).
|
||||
- Non-systemd backends and container/vm modes (the interface only, not the
|
||||
implementations).
|
||||
|
|
@ -169,15 +184,15 @@ Viability check (2026-09-29, systemd NEWS, systemd.exec(5), v239 sources):
|
|||
- Helper authentication.
|
||||
|
||||
## Out of Boundary
|
||||
- Job mode selection and policy.
|
||||
- Profile selection and access (`application-profile`).
|
||||
- Cross-node uid consistency and remote launch.
|
||||
- Worker cgroup resource limits, which are unchanged and simply apply to sandbox
|
||||
units too.
|
||||
|
||||
## Upstream / Downstream
|
||||
- **Upstream**:
|
||||
- `application-profile`: mode and directory fields on `job_startinfo_t`, and
|
||||
capability-based placement.
|
||||
- `application-profile`: the profile core, `mode:` field and the dispatch
|
||||
record that carries mode and options to the worker.
|
||||
- The remote-task root fix.
|
||||
- **Downstream**:
|
||||
- `job-sandbox-multinode`.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue