feat(resource-value-verification): scaffold xce-test-resources skill (task 1.2)

Documentation-only stub: usage, VERDICT line format, and PASS/FAIL/cleanup contract per design.md, explicitly marked not-yet-functional. The actual run-resource-verify.sh orchestration script is built incrementally by tasks 3.1-7.1. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-17 23:11:52 +00:00 · 2026-08-17 23:11:52 +00:00 · 642e23621f
commit 642e23621f
parent 98b8c34bec
2 changed files with 190 additions and 1 deletions
--- a/.claude/skills/xce-test-resources/SKILL.md
+++ b/.claude/skills/xce-test-resources/SKILL.md
@ -0,0 +1,189 @@
+---
+name: xce-test-resources
+description: "End-to-end resource-value propagation verification: inject a synthetic worker resource value via xce_test's resupdate and confirm it is correctly visible at the controller, live scheduler, and Go apiserver. Usage: /xce-test-resources [resource=value ...]"
+model: haiku
+---
+
+# XCE Resource-Value Verification
+
+## Status: scaffold only, not yet functional
+
+This `SKILL.md` is task 1.2 of the `resource-value-verification` spec
+(`.kiro/specs/resource-value-verification/`) — a documentation-only stub that
+makes this skill **discoverable and invocable** and pins its contract ahead of
+implementation. **`run-resource-verify.sh`, the orchestration script this file
+describes, does not exist yet.** Invoking `/xce-test-resources` today will not
+run a live verification — follow this file to confirm that, and report the gap
+rather than fabricate a result.
+
+The real orchestration logic is built incrementally by later tasks in the same
+spec:
+- **Task 2.1/2.2** — `resupdate` accepts any registered resource name and
+  reports per-name accept/reject (closes the gap this whole feature exists to
+  close; `resval_typeresolve.c`'s fallback path).
+- **Task 3.1** — `xce-deploy-ha`'s `deploy-ha.sh` grows a conditional apiserver
+  step + a combined `start-all.sh` (controller → scheduler → apiserver).
+- **Task 4.1/4.2** — one trace-log line each in the scheduler
+  (`core_events.c`, `XCE_CORE_EVENT_RESVALS_DELTA`) and the apiserver
+  (`handler.go`'s `OnResourceLocationUpdated`) — what this skill will
+  tail/grep for.
+- **Task 5.1** — wires `start-all.sh` + trap-based cleanup into this skill.
+- **Task 6.1–6.4** — worker join + injection, and the controller/scheduler/
+  apiserver checks themselves.
+- **Task 7.1** — verdict aggregation, the VERDICT line, and cleanup-on-every-path.
+
+Once those land, `run-resource-verify.sh` will exist alongside this file and
+this section (and the rest of this document's hedging) should be removed.
+
+## What this skill will do (target behavior)
+
+Bug-1569 required deploying real worker/controller/scheduler/apiserver
+daemons and waiting on periodic reporting cycles just to confirm a
+resource-value routing fix. This skill turns that multi-minute manual
+investigation into one command:
+
+1. **Deploy** controller + scheduler + apiserver into a fresh scratch
+   directory by invoking `xce-deploy-ha`'s `deploy-ha.sh` (Step 3) and then
+   its generated `start-all.sh` — no manual per-component sequencing lives in
+   this skill; `start-all.sh` already gates scheduler/apiserver startup on
+   `xce-deploy-ha`'s existing controller health-check.
+2. **Inject** — join a synthetic worker through `xce_test`'s worker context
+   (no real worker daemon process) and run `resupdate -n <name> -v <value>`
+   for one or more resource names.
+3. **Verify all three layers**:
+   - **Controller** — query via `xce_test`'s client-context `resval` and
+     compare against the injected value.
+   - **Scheduler** — tail/grep the live scheduler daemon's own log file for
+     the trace line task 4.1 adds, with a short poll interval up to a
+     generous bounded wait (no record/replay/REPL).
+   - **Apiserver** — tail/grep the apiserver's own log file for the trace
+     line task 4.2 adds, same bounded-wait posture.
+   Each layer's observed value is reported distinctly, so a mismatch is
+   attributable to a specific layer, not just "something failed."
+4. **Report** a single VERDICT line plus the standard terminal PASS/FAIL
+   oracle line and exit code.
+5. **Clean up** every process and runtime/data-directory artifact this run
+   created — unconditionally, on PASS, FAIL, or a mid-deployment abort alike
+   — unless the operator explicitly asked for the deployment to be left
+   running. **Log files under the scratch directory are always preserved**,
+   independent of that flag, so a FAIL verdict's underlying trace-log
+   evidence stays inspectable after the run (a deliberate divergence from
+   `xce-deploy-ha`'s own `cleanup.sh`, which wipes `logs/*.log`).
+
+**Idempotency & recovery**: every invocation is safe to re-run — a fresh
+scratch directory per run, following `xce-deploy-ha`'s existing "every
+invocation is a fresh, self-contained deploy" convention — with no shared
+mutable state carried across runs.
+
+## Usage
+
+```
+/xce-test-resources [resource=value ...]
+```
+
+- No arguments: defaults to Requirement 1's gap-resource set — the 7 names
+  `resupdate` is currently missing (`r15m`, `pg`, `ls`, `it`, `swp`, `io`,
+  `tmp`) — plus a couple of already-supported names for regression coverage
+  (e.g. `mem` for the numeric path, `type` for the string path), so a default
+  run also proves the fix didn't disturb existing behavior.
+- One or more `resource=value` pairs: checks exactly those, e.g.
+  `/xce-test-resources r15m=1.5 pg=3`.
+- Also accepts an optional scratch/deploy directory and a "keep running" flag
+  (leaves the deployment up instead of cleaning up on exit) — exact flag/env
+  names are pinned during implementation (tasks 5.1/7.1), following
+  `xce-test-ha`'s `[rounds] [dir]` positional-argument convention and
+  `XCE_HA_TOP`-style env vars as the closest precedent.
+
+## VERDICT line and PASS/FAIL oracle
+
+Following `xce-test-ha`'s machine-parseable VERDICT convention:
+
+```
+[xce-test-resources] VERDICT resources=7 layers_checked=3 mismatches=0 result=PASS
+```
+
+- `resources` — count of resource names checked this run (7 for the
+  canonical Requirement-1-gap-only run shown above; a default no-args run
+  that also includes the regression names, or an explicit `resource=value`
+  list, reports whatever count was actually checked).
+- `layers_checked` — always 3 (controller, scheduler, apiserver) for a full
+  run.
+- `mismatches` — count of (resource, layer) pairs where the observed value
+  did not match the injected value, or the layer was unreachable
+  (trace-log grep timed out). `result=PASS` iff `mismatches == 0`.
+- `result` — `PASS` or `FAIL`.
+
+Per-resource, per-layer detail (injected vs. observed value) is reported
+above the VERDICT line for anything that mismatched, distinguishing a genuine
+**mismatch** (wrong value observed) from **unreachable** (bounded wait timed
+out with no matching trace line) — the same distinction Requirement 4.3/5.4
+and design.md's Error Handling require, since the two point at different bugs
+(a value-propagation defect vs. a layer that never came up or never logged).
+
+Alongside the VERDICT line, the skill emits the usual terminal oracle line so
+a goal-loop or CI-style caller can branch on it without parsing the VERDICT
+line's fields:
+
+```
+[xce-test-resources] PASS: 7 resources, 3 layers checked, zero mismatches
+[xce-test-resources] FAIL: resource=r15m layer=scheduler mismatch (injected=1.5 observed=timeout)
+```
+
+PASS ⇒ exit `0`. Any FAIL condition ⇒ non-zero exit.
+
+## Error handling this skill will implement
+
+Per design.md's Error Handling section:
+
+- **Unrecognized resource name** — `resupdate`'s own follow-up
+  `xcecli_resval()` verification (task 2.2) reports it as rejected; the skill
+  treats this as a run-level failure and names the specific rejected
+  resource(s) in its report.
+- **Layer mismatch** — reported with both the injected and observed value,
+  attributed to the specific layer (controller/scheduler/apiserver).
+- **Layer unreachable** (e.g. apiserver never came up, or the expected
+  trace line never appears before the bounded wait elapses) — treated the
+  same as a mismatch for verdict purposes (FAIL), but labeled "timeout"
+  distinctly from "mismatch" in the report, never a silent skip.
+- **Deployment failure** (controller/scheduler/apiserver fails to start) —
+  the skill aborts before attempting injection, with a clear "deployment
+  failed at step X" message. Cleanup still runs on this path: a trap (or
+  equivalent) is installed covering every deployment step from the first
+  process start onward, not just the post-injection verification path —
+  deliberately stronger than `xce-deploy-ha`'s own precedent (a separate,
+  manually-invoked `cleanup.sh` that a mid-deploy failure would otherwise
+  leave for the caller to run by hand).
+
+## Not using `context: fork` for the eventual orchestration script
+
+Neither this skill's frontmatter nor the invocation instructions below will
+set `context: fork`, for the same reason `xce-test-ha` and `xce-deploy-ha`
+both call out explicitly in their own `SKILL.md`s: this skill's
+`run-resource-verify.sh` will invoke `xce-deploy-ha`'s `start-all.sh`
+internally, starting real backgrounded controller/scheduler/apiserver
+processes for the duration of the run. `context: fork`'s own teardown sends
+SIGHUP through the whole process group on return — the same confirmed-broken
+mechanism `xce-test-ha`'s soak driver and `xce-deploy-ha` both hit (bug
+confirmed 2026-07-08) — that would silently kill those processes mid-run,
+before cleanup or the verdict line ever fires. Dispatch it directly via Bash,
+or via the `Agent` tool
+(empirically verified safe for this exact pattern — see `xce-test-ha`'s and
+`xce-deploy-ha`'s own `SKILL.md`s), not via a forked Skill-tool context.
+
+## Dependencies (not yet built)
+
+- `xce_test`'s worker-context `resupdate`/`join` and client-context `resval`
+  — existing, but `resupdate`'s resource-name table is currently incomplete
+  (task 2.1 closes this).
+- `xce-deploy-ha`'s `deploy-ha.sh` / `start-all.sh` — controller deployment
+  exists today; the apiserver step and combined `start-all.sh` are task 3.1.
+- Scheduler and apiserver trace-log lines — tasks 4.1 and 4.2.
+- This skill's own `run-resource-verify.sh` — tasks 5.1–7.1.
+
+## Requirements traceability
+
+This stub documents the contract for design.md's "`xce-test-resources`
+skill" component (Components and Interfaces → Test Tooling / Bash), covering
+Requirements 3.1, 3.2, 4.1, 4.2, 4.3, 5.1, 5.2, 5.4, 6.1, 6.2, 6.3, 6.4, 6.5.
+See `.kiro/specs/resource-value-verification/requirements.md` and `design.md`
+for the full text.
--- a/.kiro/specs/resource-value-verification/tasks.md
+++ b/.kiro/specs/resource-value-verification/tasks.md
@ -8,7 +8,7 @@
  - Observable completion: the new module and test target both compile and link via `meson`, and the new test target fails (RED) because the fallback-to-numeric and type-override behavior don't exist yet
  - _Requirements: 1.1, 1.4_

- [ ] 1.2 (P) Scaffold the xce-test-resources skill directory
+- [x] 1.2 (P) Scaffold the xce-test-resources skill directory
  - Create the skill directory with a `SKILL.md` stub describing the intended usage (`/xce-test-resources [resource=value ...]`) and the VERDICT line format, following the `xce-test-ha`/`xce-deploy-ha` SKILL.md convention
  - Observable completion: the skill is discoverable/invocable (even though its underlying script is not yet functional) and its documented contract matches design.md's Batch/Job Contract for the skill
  - _Requirements: 6.1_