docs(steering): resource log: hosts as supply (PR-4), simple datasets (PR-5), CSI notes, N2/N4 input

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

.kiro/steering/resource-questions.md

@@ -50,6 +50,69 @@ silently.
 - **Affects:** `resource-distribution-mapping`, `custom-resource-reporting` (string-set locality may become map entries instead), `resource-claims` (scope shrinks to job-private data), R9.
 - **Open:** see N3.
 
+### PR-4: hosts as supply entries; the worker connection only confirms them (status: discussing)
+- **User:** hosts get into the controller directly today; how could they be unified with the rest of the supply?
+- **Today:** a worker connects to the controller and is accepted only if its name matches a host config entry (`find_matched_hostconfig`, `src/controller/join.c:~76`). The host record is created by that first join (`init_worker`). `elastic-hosts` plans "expected host" records matched at join.
+- **Proposal:** separate *who decides a host should exist* (control plane) from *the worker's connection* (data plane):
+  - **Host supply entries:** a host is an entry in the supply map with a source and a lifecycle state:
+    - `config`: a static `hosts.yaml` entry or pattern;
+    - `provisioned`: created by a proxy as an expected host, with instance id, template and deadline;
+    - `discovered`: an unknown host that joins, admitted only if a policy allows it.
+
+    Lifecycle: `expected → joining → available → draining → released`, plus `lost`.
+  - **Join confirms:** the worker still connects directly. Job control, heartbeats and resync stay on that long-lived connection. A join confirms an existing entry (or is admitted by policy) instead of being the only way a host appears.
+  - **Host resources:** `slots`, `mem` and so on stay L2 counters at the host's location, as today.
+  - **Bring-up:** `xce-node-manager` can act as the node-side provider that starts the worker on a provisioned machine.
+- **Gains:** one place for static, elastic (cloud) and discovered hosts. Orphan rules apply uniformly (expected but never joined; joined but unknown). `provisioning-demand` sees hosts and other resources the same way.
+- **Affects:** `elastic-hosts` (its expected-host records become supply entries), `resource-distribution-mapping` (PR-1), `xce-node-manager`, the controller join path.
+
+### PR-5: simplified dataset model, v1 (status: discussing)
+- **User:** give a simplified dataset model first.
+- **Proposal:**
+  - **Identity:** a dataset is `name@version` and is **immutable**; a change is a new version. No update consistency problem.
+  - **Copies:** supply-map entries of the dataset at a location (host, host group or storage pool) with a state `staging → available → evicting` and a size. The size is charged as an L2 counter on the location's cache or scratch capacity.
+  - **Jobs:** a job names datasets, e.g. `data[imagenet@3]`. The scheduler **prefers** hosts with an available copy, as a locality attribute from its cached snapshot. v1 does not make placement wait for a copy.
+  - **Staging:** in v1 staging is explicit. A user or admin asks for a copy at a location (`xcecli data stage`); a proxy provider copies it and reports `available`. Eviction is explicit too.
+  - **Later:** scheduler-driven staging (shortfall as provisioning demand), automatic LRU eviction, dataset quotas, and writable datasets.
+- **Open:** whether v1 job placement should *require* a copy (a hard constraint) or only prefer one; and what the dataset registry is (config, REST object, or both).
+
+## CSI notes (2026-10-11, for C3 and R4)
+Source: the CSI specification, from general knowledge; to be checked against the current spec version during design.
+
+- **What it is:** the Container Storage Interface, a CNCF specification: a gRPC API between a *container orchestrator* and a *storage driver*. It is orchestrator-neutral (used by Kubernetes, Nomad, Mesos and Cloud Foundry), though most drivers are written, packaged and tested for Kubernetes.
+- **Three services:**
+  - **Identity** (every plugin): `GetPluginInfo`, `GetPluginCapabilities`, `Probe`.
+  - **Controller** (runs once, central): `CreateVolume`/`DeleteVolume`, `ControllerPublishVolume`/`Unpublish` (attach to a node), `ValidateVolumeCapabilities`, `ListVolumes`, `GetCapacity`, `CreateSnapshot`/`DeleteSnapshot`/`ListSnapshots`, `ControllerExpandVolume`, `ControllerGetVolume`.
+  - **Node** (one per host, privileged): `NodeStageVolume`/`Unstage` (mount once per node at a staging path), `NodePublishVolume`/`Unpublish` (bind-mount into the workload's path), `NodeGetInfo` (node id, max volumes, accessible topology), `NodeGetCapabilities`, `NodeExpandVolume`, `NodeGetVolumeStats`.
+- **Key concepts:**
+  - Volume capability: block or filesystem, plus fs type and mount flags.
+  - Access modes: single-node writer or reader, multi-node reader-only, single-writer or multi-writer.
+  - Topology: driver-reported segments such as `zone=a` or `rack=r1`; `CreateVolume` takes requisite and preferred topology.
+  - Opaque parameters (a storage class); per-call secrets; volume context returned by create.
+- **How Kubernetes runs it:**
+  - The controller plugin runs as a Deployment, with sidecars (external-provisioner, attacher, snapshotter, resizer) that turn PVCs into CSI calls.
+  - The node plugin runs as a privileged DaemonSet, with node-driver-registrar, over a unix socket.
+  - The sidecars are Kubernetes-specific; the CSI calls themselves are not.
+- **Mapping to XCE:**
+  - **Controller side → PR-2 proxy:** `CreateVolume` when a claim or dataset copy is created; `ControllerPublish` when it is bound to hosts; `Delete` on release; `GetCapacity` → pool counters in the map; snapshots → checkpoint snapshots; secrets stay in the proxy.
+  - **Node side → worker/xcecd:** `NodeStage` on first use on a host; `NodePublish` into the job's sandbox (or container) path; unpublish and unstage at job end. `NodeGetInfo` topology → host attributes (L1).
+  - **Timing:** CSI topology plus "create after placement" matches the rule that the scheduler only selects (Kubernetes `WaitForFirstConsumer`).
+  - **Access modes** map one-to-one onto claim access modes.
+- **Costs:**
+  - gRPC clients: natural in Go (the proxy; P2), awkward in C (the worker). A small Go node helper, started with `posix_spawn` or run under `xce-node-manager`, could make the node calls.
+  - The node plugin is a privileged daemon on every host, needing mount propagation.
+  - Some drivers assume Kubernetes APIs or container packaging, so each target driver needs testing outside Kubernetes.
+- **Relevant drivers:**
+  - HPC parallel filesystems: IBM Spectrum Scale/GPFS CSI, BeeGFS CSI, Lustre (e.g. AWS FSx, DDN), WEKA, VAST.
+  - General storage: NFS, Ceph RBD/CephFS, NetApp Trident, Portworx, cloud disks (EBS, GCE PD, Azure Disk), local path.
+  - Object storage: S3 via Mountpoint for S3 or JuiceFS.
+  - Dataset caching: Fluid (Alluxio/JuiceFS), a Kubernetes dataset-cache project relevant to PR-5.
+- **Recommendation:**
+  - Make XCE's own claim and provider contract **CSI-shaped**: the same verbs (create, publish, stage, node publish, capacity, snapshot, expand), capabilities, access modes, topology and per-call secrets.
+  - v1 ships simple native JSON providers (local scratch, NFS or parallel-filesystem directories).
+  - Add a **CSI adapter** later as one provider type in the proxy, plus a node helper for the worker.
+  - That keeps CSI a cheap "must later" rather than a v1 cost. If containers (R6) or cloud block storage become early targets, CSI moves up.
+
 ## Resource model questions (from `resource-model.md`)
 
 | ID | Question | Status | User input / notes |
@@ -82,11 +145,14 @@ silently.
 
 | ID | Question | Status | Notes |
 |---|---|---|---|
-| N1 | Does the supply map (PR-1) also cover hosts provisioned by `elastic-hosts`, or only resources on hosts and pools? | open | Hosts have their own lifecycle (join, drain, decommission); they may stay separate, with the map referencing them. |
-| N2 | One proxy (PR-2) for every provider kind, or one per kind (cloud, license, storage)? How does it map onto the provisioner's per-provider leases? | open | The brief already allows several instances, each owning providers by lease. |
-| N3 | Dataset model (PR-3): identity and versions, read-only vs updatable, capacity accounting at each location, eviction policy, who triggers staging (scheduler demand, user, admin)? | open | |
-| N4 | User RBAC in the apiserver vs component authorization in the controller (which daemon roles may call which opcodes): confirm both layers exist and where each check lives. | open | Raised by the user's RBAC position (R3) and P3. |
-| C3 | CSI compatibility: nice-to-have or a must? If a must, which side: the controller plugin through the proxy, the node plugin on the worker, or both? | open | The node plugin implies gRPC on the worker and mount semantics for the sandbox or container mode. |
+| N1 | Does the supply map (PR-1) also cover hosts provisioned by `elastic-hosts`, or only resources on hosts and pools? | discussing | User 2026-10-11: hosts join the controller directly today; how could they be unified? Proposal **PR-4** below. |
+| N2 | One proxy (PR-2) for every provider kind, or one per kind (cloud, license, storage)? How does it map onto the provisioner's per-provider leases? | leaning | User 2026-10-11: there may be multiple proxies. Matches the provisioner brief: several instances, each provider owned by exactly one active proxy at a time through a controller lease, with standbys taking over. Still open: N2a, N2b. |
+| N2a | How does a proxy declare which providers it serves: config pushed by the controller, or self-declared at join (and checked against config)? | open | |
+| N2b | Can one proxy serve several kinds (cloud and storage), or one kind each? | open | Leaning: any mix; the lease is per provider, not per kind. |
+| N3 | Dataset model (PR-3): identity and versions, read-only vs updatable, capacity accounting at each location, eviction policy, who triggers staging (scheduler demand, user, admin)? | discussing | User 2026-10-11: start with a simplified model. Proposal **PR-5** below. |
+| N4 | User RBAC in the apiserver vs component authorization in the controller (which daemon roles may call which opcodes): confirm both layers exist and where each check lives. | leaning | User 2026-10-11: XCE delegates access control to the API server, and the controller trusts it. Implication: the trust is only sound if the controller knows a request really comes from the apiserver (or another legitimate component). Today `validate_host()` accepts every peer (`src/controller/join.c:13-16`), so anything that reaches the controller port bypasses RBAC. Component authentication with a per-role opcode allowlist (`peer-authentication-hardening`, P3) is what makes the delegation safe, especially once proxies can spend money. |
+| C3 | CSI compatibility: nice-to-have or a must? If a must, which side: the controller plugin through the proxy, the node plugin on the worker, or both? | discussing | Background and recommendation in "CSI notes" below (2026-10-11). |
 
 ## History
+- 2026-10-11: user input on N1 (hosts join directly; asked how to unify → PR-4), N2 (multiple proxies → leaning), N3 (a simplified model first → PR-5), N4 (access control delegated to the API server, the controller trusts it → leaning, with the component-authentication implication), C3 (asked for more on CSI → CSI notes). N2a and N2b added.
 - 2026-10-11: created from the resource-model discussion. User input recorded for PR-1, PR-2 and PR-3 (proposals) and for R3, R4, R5, R6 and R8. Questions N1–N4 and C3 added.