myxce/.kiro/specs/resource-distribution-mapping/research.md

# Research: resourcemap / distributions / hostBased

## Status: scope decision pending — do not generate requirements.md until a scope option below is chosen

## What triggered this spec

`xceadmin config build` warns `resources.yaml: field 'resourcemap' on '<name>' is schema-valid but has no ClusterConfig destination -- ignored` for every resource entry that uses `resourcemap`. The initial framing (a quick capnp-field-uncomment-and-wire task) turned out to understate the real scope once the C-side consumers were traced — see findings below.

## Findings (verified against code, file:line cited)

- `res_distribution_t` (`include/xce/types.h:127-131`): `{value: xce_val_t*, nhosts: int, hosts: char**}` — maps 1:1 to one `resourcemap` YAML entry `{value, location: [...]}`. Commented-out capnp counterpart: `src/schema/controller.capnp:195-198,208-209`.
- `xce_resource_t.hbased_defined` is typed `xce_val_t*` (a value, not a bool flag) — plausibly a fallback/default value for hosts not covered by any explicit `distributions` entry (matches the `location: [default]` sentinel used by `dynres1`/`strres1`/`staticres3`/`host_quota`). Inference only — nothing in the codebase reads this field to confirm.
- **The fields are dead on the C side too, not just uncommented in capnp.** `resobj_init()` (`src/lib/resobj.c:37-51`) explicitly zeroes `dst->distributions`/`dst->hbased_defined` with `/* FIXME: */` comments, even under a hypothetical populated input. `scheduler.c:787-813`'s resource update handler copies only `valtype`/`ordtype`/`flags`/`interval`, with a comment explaicitly excluding these two as "deep-structs." No other file in scheduler/, worker/, controller/, lib/ reads `res_distribution_t`, `->distributions`, or `->hbased_defined` at all (repo-wide grep, zero other hits).
- **A separate, fully-live per-host resource-value pipeline already exists**, independent of `resourcemap`, via `hosts.yaml`'s host-level `resource: [{name, value}]` list. Full trace: `mapping.py:_build_host_config` → `HostConfig.resources` (capnp) → controller `apply_join_state()` → `mark_host_res()` (`src/controller/worker.c:163-221`) → `update_val_res()` writes `w->host.resvals[idx]` → pushed to scheduler via `collect_resval()`/`Worker.resvals` → `schedobj.c:286-304` `zarray_put(&(worker->resvals), ...)` → read by `sched_check_host_satisfy_()` (`src/scheduler/src/resource.c:330,357-379`) during job placement.
- `mark_host_res()` (`worker.c:190-193`) **actively rejects** a resource flagged `XCE_RESF_SHARED`: `if (r->flags & XCE_RESF_SHARED) { ERROR(...); return -EINVAL; }`. So "shared pool" and "per-host static value" are mutually exclusive today, by construction.
- `XCE_RESF_SHARED`/`GLOBAL` pooling machinery (`sched->sharedInst`, `sched_check_shared_satisfy_()`, `src/scheduler/src/resource.c:232-282`) is real and live, but exclusively serves job-limit virtual resources (`src/lib/xcelimit.c:399-404`'s `add_vres()`), never a physical `resources.yaml` resource. The scheduler plumbing for "N consumers draw from one pooled value" is architecturally reusable, just not reachable from `resourcemap` today.
- `interval`/dynamic-resource semantics have **zero enforcement logic anywhere** — `last_report` is never read; `interval` is only copied, never acted on. The only real dynamic-reporting mechanism is `src/worker/src/resource_sync.c`'s 11 hardcoded built-in load indices (`init_builtin_load_indices()`, lines 59-94). A plugin hook exists to register a *custom* dynamic resource (`resource_sync_register_load_index()`, line 104) but nothing calls it from any YAML-driven path — a user-defined dynamic resource (`dynres1`/`dynres2`-style) has no live way to ever acquire a value today.
- `src/configs/resources.yaml`'s own example comments are internally inconsistent about "share" semantics: `staticres1` (one `resourcemap` entry, two hosts in `location:`) says "host1 and host2 **share** the 100 res"; `staticres2` (two separate entries, one host each) says "each of host1 and host2 **shares** the 100 res" — opposite shapes, same word. Not a reliable semantic source as-is.

## Three candidate scopes (sizes verified against the above, not guessed)

1. **Static replication to N hosts** (`staticres3`-style: one `resourcemap` declaration → the same static value written into N named hosts' per-host `resvals`, reusing the existing `mark_host_res()`/`update_val_res()` path). Pure DRY/ergonomics win — functionally redundant with just repeating `{name, value}` across N `hosts.yaml` host entries, since `hosts.yaml` already does the "one host, one static value" case live end-to-end. Small: capnp field un-comment + mapping.py translation + a controller-side loop over the host list at config-load/push time. No scheduler changes, no new runtime concept.

2. **Static replication + shared pool** (adds `staticres1`/`staticres4`-style: N hosts collectively draw down ONE pooled value). Requires: allowing `XCE_RESF_SHARED` on a physical resource (currently `mark_host_res` refuses this), wiring `sched->sharedInst` for resources.yaml-sourced resources (today limits-only), and defining job-placement semantics for a shared physical resource (distinct from the existing shared *limit* semantics — a resource being consumed to zero is a scheduling constraint, not a rejection). Real new capability, touches job-placement code.

3. **All three, plus dynamic per-host custom resource reporting** (`dynres1`/`dynres2`-style). Needs a new worker-side mechanism: something must call `resource_sync_register_load_index()` (or an equivalent) per YAML-declared dynamic resource, decide how/when it's collected per host, and thread it into the same `resvals` pipewith proper interval-based refresh — none of which exists today even as a partial scaffold. Largest scope.

## Decision

Not yet made — user chose to record the options and decide later rather than commit now. **Do not proceed to requirements.md generation until one of the three scopes (or a variant) is chosen.** When resumed, re-run `/kiro-spec-requirements resource-distribution-mapping` with the chosen scope stated explicitly, since the EARS requirements (and everything downstream) differ substantially by option.