From 73dbe2535eb55bef82df9d80bbd9cf7039745c19 Mon Sep 17 00:00:00 2001 From: Rongsong Shen Date: Mon, 12 Oct 2026 04:28:06 +0000 Subject: [PATCH] docs(steering): resource log: shared volume with per-job subdirectories as the job I/O default (PR-6) Co-Authored-By: Claude Opus 5.5 (1M context) --- .kiro/steering/resource-questions.md | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/.kiro/steering/resource-questions.md b/.kiro/steering/resource-questions.md index e45b1cf6..59e2e37a 100644 --- a/.kiro/steering/resource-questions.md +++ b/.kiro/steering/resource-questions.md @@ -92,6 +92,12 @@ silently. - **Placement in the job:** the sandbox mounts the claims at fixed points (`/xce/in`, `/xce/out`) or at profile-allowed paths. Direct-mode jobs get the paths through environment variables. - **v1 provider: a directory provider on a shared filesystem.** Create is `mkdir` with an owner and a quota, publish is a bind mount or path, delete is removal. CSI, cloud block and burst-buffer volumes plug into the same contract later. - **Default:** jobs without I/O options keep today's behaviour. +- **Refinement (user, 2026-10-11): a shared volume with per-job subdirectories is the default (status: leaning).** Jobs do not each get a volume; several jobs share one: `shared_volume///{in,out,err}`, array elements `//...`. + - **The shared volume:** a long-lived, admin-provisioned standalone claim (read-write-many, retain). Its capacity is a pool counter in the supply map; the hosts that mount it are an L1 attribute, so placement only uses hosts that reach it. + - **A job's directory:** a lightweight template claim of kind "subdirectory". Create is `mkdir` plus ownership, with no storage call; release is retention or cleanup. + - **Per-job volumes** (burst buffer, large scratch) stay possible when a profile asks for one. + - **Settles:** N9 (read-write-many, so multi-host jobs work) and N10 (a subdirectory per element). Reframes N8 as per-user or per-project quota on the shared volume (filesystem quota, or usage charged to the pool counter after the fact, consistent with `max(reserved, actual)`). + - Prior art: Kubernetes `subPath` and the nfs-subdir provisioner; the shared output trees common at LSF and Slurm sites. - **Affects:** - `resource-claims` (adds job I/O as its first use, ahead of checkpoints); - `job-sandbox-execution` (mount points); @@ -176,13 +182,18 @@ Source: the CSI specification, from general knowledge; to be checked against the | N4 | User RBAC in the apiserver vs component authorization in the controller (which daemon roles may call which opcodes): confirm both layers exist and where each check lives. | leaning | User 2026-10-11: XCE delegates access control to the API server, and the controller trusts it. Implication: the trust is only sound if the controller knows a request really comes from the apiserver (or another legitimate component). Today `validate_host()` accepts every peer (`src/controller/join.c:13-16`), so anything that reaches the controller port bypasses RBAC. Component authentication with a per-role opcode allowlist (`peer-authentication-hardening`, P3) is what makes the delegation safe, especially once proxies can spend money. | | N6 | PR-6: with I/O claims, should `-o/-e/-i` and `out_dir` be accepted again even when no output claim is configured (e.g. a default per-job output directory claim), or stay rejected? | open | | | N7 | PR-6: where do uploaded inputs live between submit and job start (provider volume, apiserver staging area), and is there a size limit? | open | Large inputs should be datasets (PR-5). | -| N8 | PR-6: retention and quota defaults for output claims; who is charged (pool counter per project)? | open | | -| N9 | PR-6: multi-host jobs: a read-write-many output (shared filesystem), or per-host outputs collected after the job? | open | | -| N10 | PR-6: job arrays: one output claim with a subdirectory per element, or one claim per element? | open | Leaning: one claim, subdirectories (cheap). | +| N8 | PR-6: retention and quota defaults for output claims; who is charged (pool counter per project)? | discussing | With the shared volume: per-user or per-project quota on that volume (filesystem quota, or usage charged to the pool counter after the fact). Retention: see N14. | +| N9 | PR-6: multi-host jobs: a read-write-many output (shared filesystem), or per-host outputs collected after the job? | leaning | Read-write-many shared volume (PR-6 refinement). | +| N10 | PR-6: job arrays: one output claim with a subdirectory per element, or one claim per element? | leaning | Subdirectory per element: `//...` (PR-6 refinement). | | N11 | PR-6: requeue and restart: reuse the same output claim or create a new one? How does it relate to checkpoint claims? | open | | +| N12 | PR-6 shared volume: permissions and creation. `` 0700 and user-owned under a root-owned top? Is the job directory created by the worker (as root, before dropping privileges) or by a provider? Path-safe encoding of array IDs with brackets? | open | | +| N13 | PR-6 shared volume: a requeued job keeps its ID. Reuse `/` or add an attempt subdirectory (`/attempt-2`)? (Refines N11.) | open | | +| N14 | PR-6 shared volume: retention per job directory (TTL, after fetch), the cleaner (a provider task), and orphan directories whose job no longer exists. | open | | +| N15 | PR-6 shared volume: with several shared volumes, which one a job uses: by application profile, queue or project? | open | | | C3 | CSI compatibility: nice-to-have or a must? If a must, which side: the controller plugin through the proxy, the node plugin on the worker, or both? | discussing | Background and recommendation in "CSI notes" below (2026-10-11). | ## History +- 2026-10-11: user proposed a shared volume with per-user/per-job subdirectories (`shared_volume///{in,out,err}`) instead of a volume per job → PR-6 refinement (leaning). N9 and N10 leaning; N8 reframed; N12–N15 added. - 2026-10-11: user asked whether volume provisioning can carry job input and output → PR-6; questions N6–N11 added. - 2026-10-11: user input on N1 (hosts join directly; asked how to unify → PR-4), N2 (multiple proxies → leaning), N3 (a simplified model first → PR-5), N4 (access control delegated to the API server, the controller trusts it → leaning, with the component-authentication implication), C3 (asked for more on CSI → CSI notes). N2a and N2b added. - 2026-10-11: created from the resource-model discussion. User input recorded for PR-1, PR-2 and PR-3 (proposals) and for R3, R4, R5, R6 and R8. Questions N1–N4 and C3 added.