docs(steering): resource log: simple submit options (PR-7), data dependencies (PR-8), user isolation and sharing (PR-9)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

.kiro/steering/resource-questions.md

@@ -105,6 +105,33 @@ silently.
   - `application-profile` (default I/O claims, allowed paths);
   - PR-5 (inputs from datasets).
 
+### PR-7: keep job submission options simple (status: leaning)
+- **User (2026-10-11):** keep the submit options simple.
+- **Proposal:**
+  - **No volume or claim names for ordinary jobs.** The application profile or queue selects the shared volume and the directory layout (resolves N15).
+  - **LSF options come back**, resolved against the job's directory (PR-6): `-i`, `-o`, `-e` (with `%J`/`%I`), `-outdir`, and `-f` (copy local files in, or results back out). Resolves N6: the bug-1936 rejections are lifted once the job directory exists.
+  - **At most two advanced options:** `--data name@ver` (input dataset, PR-5) and `--claim name:mode` (standalone claim, e.g. checkpoints or model weights). Everything else lives in profiles.
+
+### PR-8: job dependencies by data produced and consumed (status: discussing)
+- **User (2026-10-11):** consider job dependencies based on producing and consuming data.
+- **Proposal:** an extension of `job-dependency`. Prior art: LSF Data Manager data tags, Nextflow, Snakemake, Argo.
+  - **Producer:** declares an output dataset (`--produce results@auto`). On successful completion its output directory is **sealed** as an immutable dataset version (PR-5).
+  - **Consumer:** declares `--data results@latest` or a specific version. A dependency condition `data(results@v)` keeps it pending until the version is available. Then the version is mounted read-only as an input, and placement prefers hosts near it.
+- **Open:** see N16–N19.
+
+### PR-9: isolation between users; data sharing later (status: leaning)
+- **User (2026-10-11):** consider isolation between users; a mechanism for sharing data across users may be low priority.
+- **Isolation (v1 must):**
+  - Each user's tree is 0700 and user-owned; jobs run as the user's uid.
+  - Claims and datasets are owned by a user or project.
+  - The API server lets users list and fetch only their own data; admins see everything (N4: access control lives in the API server).
+  - **Filesystem permissions (uid, gid, ACLs) must enforce isolation as well**, because jobs read the shared filesystem directly on workers. API server RBAC alone is not enough.
+- **Sharing (low priority):**
+  - A sealed, immutable dataset (PR-5) gets a visibility of `private`, `project` or `public`, always **read-only**.
+  - The API server checks the visibility; the worker mounts the dataset read-only, or a group ACL grants access on the filesystem.
+  - Shared writable space, if ever needed, is a project-owned directory with group permissions.
+  - Sharing is what enables cross-user PR-8.
+
 ## CSI notes (2026-10-11, for C3 and R4)
 Source: the CSI specification, from general knowledge; to be checked against the current spec version during design.
 
@@ -180,19 +207,25 @@ Source: the CSI specification, from general knowledge; to be checked against the
 | N2b | Can one proxy serve several kinds (cloud and storage), or one kind each? | open | Leaning: any mix; the lease is per provider, not per kind. |
 | N3 | Dataset model (PR-3): identity and versions, read-only vs updatable, capacity accounting at each location, eviction policy, who triggers staging (scheduler demand, user, admin)? | discussing | User 2026-10-11: start with a simplified model. Proposal **PR-5** below. |
 | N4 | User RBAC in the apiserver vs component authorization in the controller (which daemon roles may call which opcodes): confirm both layers exist and where each check lives. | leaning | User 2026-10-11: XCE delegates access control to the API server, and the controller trusts it. Implication: the trust is only sound if the controller knows a request really comes from the apiserver (or another legitimate component). Today `validate_host()` accepts every peer (`src/controller/join.c:13-16`), so anything that reaches the controller port bypasses RBAC. Component authentication with a per-role opcode allowlist (`peer-authentication-hardening`, P3) is what makes the delegation safe, especially once proxies can spend money. |
-| N6 | PR-6: with I/O claims, should `-o/-e/-i` and `out_dir` be accepted again even when no output claim is configured (e.g. a default per-job output directory claim), or stay rejected? | open | |
+| N6 | PR-6: with I/O claims, should `-o/-e/-i` and `out_dir` be accepted again even when no output claim is configured (e.g. a default per-job output directory claim), or stay rejected? | leaning | PR-7: accepted again, resolved against the job directory the profile or queue provides. Rejected only where no shared volume is configured. |
 | N7 | PR-6: where do uploaded inputs live between submit and job start (provider volume, apiserver staging area), and is there a size limit? | open | Large inputs should be datasets (PR-5). |
 | N8 | PR-6: retention and quota defaults for output claims; who is charged (pool counter per project)? | discussing | With the shared volume: per-user or per-project quota on that volume (filesystem quota, or usage charged to the pool counter after the fact). Retention: see N14. |
 | N9 | PR-6: multi-host jobs: a read-write-many output (shared filesystem), or per-host outputs collected after the job? | leaning | Read-write-many shared volume (PR-6 refinement). |
 | N10 | PR-6: job arrays: one output claim with a subdirectory per element, or one claim per element? | leaning | Subdirectory per element: `<jobid>/<index>/...` (PR-6 refinement). |
-| N11 | PR-6: requeue and restart: reuse the same output claim or create a new one? How does it relate to checkpoint claims? | open | |
+| N11 | PR-6: requeue and restart: reuse the same output claim or create a new one? How does it relate to checkpoint claims? | leaning (details deferred) | User 2026-10-11: requeue and restart can use the previous one; details later. |
 | N12 | PR-6 shared volume: permissions and creation. `<user>` 0700 and user-owned under a root-owned top? Is the job directory created by the worker (as root, before dropping privileges) or by a provider? Path-safe encoding of array IDs with brackets? | open | |
-| N13 | PR-6 shared volume: a requeued job keeps its ID. Reuse `<jobid>/` or add an attempt subdirectory (`<jobid>/attempt-2`)? (Refines N11.) | open | |
+| N13 | PR-6 shared volume: a requeued job keeps its ID. Reuse `<jobid>/` or add an attempt subdirectory (`<jobid>/attempt-2`)? (Refines N11.) | leaning (details deferred) | Reuse the previous directory (user 2026-10-11); whether earlier attempts' files are kept, overwritten or rotated is to be discussed later. |
 | N14 | PR-6 shared volume: retention per job directory (TTL, after fetch), the cleaner (a provider task), and orphan directories whose job no longer exists. | open | |
 | N15 | PR-6 shared volume: with several shared volumes, which one a job uses: by application profile, queue or project? | open | |
+| N16 | PR-8: what counts as "complete" for sealing an output: job DONE with exit 0, or an explicit seal by the job? | open | |
+| N17 | PR-8: partial outputs from a failed or killed producer: discard, keep unsealed, or seal as a failed version? | open | |
+| N18 | PR-8: version numbering (`@auto` increment, timestamps, user tags) and `@latest` resolution: at submit or at dispatch? | open | |
+| N19 | PR-8: a consumer whose producer fails or never seals: pend forever, or fail like an unsatisfiable `job-dependency` condition? | open | Follow `job-dependency`'s rule for never-satisfied conditions. |
+| N20 | PR-9: sharing granularity: dataset visibility (private/project/public, read-only) is enough, or are explicit per-user grants and writable shared space needed? How is visibility enforced on the filesystem (group ACLs, read-only mounts)? | open | Low priority (user 2026-10-11). |
 | C3 | CSI compatibility: nice-to-have or a must? If a must, which side: the controller plugin through the proxy, the node plugin on the worker, or both? | discussing | Background and recommendation in "CSI notes" below (2026-10-11). |
 
 ## History
+- 2026-10-11: user input: keep submit options simple → PR-7 (N6 leaning); data producer/consumer dependencies → PR-8 (N16–N19); isolation between users, sharing low priority → PR-9 (N20); requeue/restart reuse the previous directory, details later → N11/N13 leaning.
 - 2026-10-11: user proposed a shared volume with per-user/per-job subdirectories (`shared_volume/<user>/<jobid>/{in,out,err}`) instead of a volume per job → PR-6 refinement (leaning). N9 and N10 leaning; N8 reframed; N12–N15 added.
 - 2026-10-11: user asked whether volume provisioning can carry job input and output → PR-6; questions N6–N11 added.
 - 2026-10-11: user input on N1 (hosts join directly; asked how to unify → PR-4), N2 (multiple proxies → leaning), N3 (a simplified model first → PR-5), N4 (access control delegated to the API server, the controller trusts it → leaning, with the component-authentication implication), C3 (asked for more on CSI → CSI notes). N2a and N2b added.