From 642e23621f12f5eb2302d3b153ce9d8ad8863a44 Mon Sep 17 00:00:00 2001 From: Rongsong Shen Date: Mon, 17 Aug 2026 23:11:52 +0000 Subject: [PATCH] feat(resource-value-verification): scaffold xce-test-resources skill (task 1.2) Documentation-only stub: usage, VERDICT line format, and PASS/FAIL/cleanup contract per design.md, explicitly marked not-yet-functional. The actual run-resource-verify.sh orchestration script is built incrementally by tasks 3.1-7.1. Co-Authored-By: Claude Sonnet 5 --- .claude/skills/xce-test-resources/SKILL.md | 189 ++++++++++++++++++ .../resource-value-verification/tasks.md | 2 +- 2 files changed, 190 insertions(+), 1 deletion(-) create mode 100644 .claude/skills/xce-test-resources/SKILL.md diff --git a/.claude/skills/xce-test-resources/SKILL.md b/.claude/skills/xce-test-resources/SKILL.md new file mode 100644 index 00000000..7c401103 --- /dev/null +++ b/.claude/skills/xce-test-resources/SKILL.md @@ -0,0 +1,189 @@ +--- +name: xce-test-resources +description: "End-to-end resource-value propagation verification: inject a synthetic worker resource value via xce_test's resupdate and confirm it is correctly visible at the controller, live scheduler, and Go apiserver. Usage: /xce-test-resources [resource=value ...]" +model: haiku +--- + +# XCE Resource-Value Verification + +## Status: scaffold only, not yet functional + +This `SKILL.md` is task 1.2 of the `resource-value-verification` spec +(`.kiro/specs/resource-value-verification/`) — a documentation-only stub that +makes this skill **discoverable and invocable** and pins its contract ahead of +implementation. **`run-resource-verify.sh`, the orchestration script this file +describes, does not exist yet.** Invoking `/xce-test-resources` today will not +run a live verification — follow this file to confirm that, and report the gap +rather than fabricate a result. + +The real orchestration logic is built incrementally by later tasks in the same +spec: +- **Task 2.1/2.2** — `resupdate` accepts any registered resource name and + reports per-name accept/reject (closes the gap this whole feature exists to + close; `resval_typeresolve.c`'s fallback path). +- **Task 3.1** — `xce-deploy-ha`'s `deploy-ha.sh` grows a conditional apiserver + step + a combined `start-all.sh` (controller → scheduler → apiserver). +- **Task 4.1/4.2** — one trace-log line each in the scheduler + (`core_events.c`, `XCE_CORE_EVENT_RESVALS_DELTA`) and the apiserver + (`handler.go`'s `OnResourceLocationUpdated`) — what this skill will + tail/grep for. +- **Task 5.1** — wires `start-all.sh` + trap-based cleanup into this skill. +- **Task 6.1–6.4** — worker join + injection, and the controller/scheduler/ + apiserver checks themselves. +- **Task 7.1** — verdict aggregation, the VERDICT line, and cleanup-on-every-path. + +Once those land, `run-resource-verify.sh` will exist alongside this file and +this section (and the rest of this document's hedging) should be removed. + +## What this skill will do (target behavior) + +Bug-1569 required deploying real worker/controller/scheduler/apiserver +daemons and waiting on periodic reporting cycles just to confirm a +resource-value routing fix. This skill turns that multi-minute manual +investigation into one command: + +1. **Deploy** controller + scheduler + apiserver into a fresh scratch + directory by invoking `xce-deploy-ha`'s `deploy-ha.sh` (Step 3) and then + its generated `start-all.sh` — no manual per-component sequencing lives in + this skill; `start-all.sh` already gates scheduler/apiserver startup on + `xce-deploy-ha`'s existing controller health-check. +2. **Inject** — join a synthetic worker through `xce_test`'s worker context + (no real worker daemon process) and run `resupdate -n -v ` + for one or more resource names. +3. **Verify all three layers**: + - **Controller** — query via `xce_test`'s client-context `resval` and + compare against the injected value. + - **Scheduler** — tail/grep the live scheduler daemon's own log file for + the trace line task 4.1 adds, with a short poll interval up to a + generous bounded wait (no record/replay/REPL). + - **Apiserver** — tail/grep the apiserver's own log file for the trace + line task 4.2 adds, same bounded-wait posture. + Each layer's observed value is reported distinctly, so a mismatch is + attributable to a specific layer, not just "something failed." +4. **Report** a single VERDICT line plus the standard terminal PASS/FAIL + oracle line and exit code. +5. **Clean up** every process and runtime/data-directory artifact this run + created — unconditionally, on PASS, FAIL, or a mid-deployment abort alike + — unless the operator explicitly asked for the deployment to be left + running. **Log files under the scratch directory are always preserved**, + independent of that flag, so a FAIL verdict's underlying trace-log + evidence stays inspectable after the run (a deliberate divergence from + `xce-deploy-ha`'s own `cleanup.sh`, which wipes `logs/*.log`). + +**Idempotency & recovery**: every invocation is safe to re-run — a fresh +scratch directory per run, following `xce-deploy-ha`'s existing "every +invocation is a fresh, self-contained deploy" convention — with no shared +mutable state carried across runs. + +## Usage + +``` +/xce-test-resources [resource=value ...] +``` + +- No arguments: defaults to Requirement 1's gap-resource set — the 7 names + `resupdate` is currently missing (`r15m`, `pg`, `ls`, `it`, `swp`, `io`, + `tmp`) — plus a couple of already-supported names for regression coverage + (e.g. `mem` for the numeric path, `type` for the string path), so a default + run also proves the fix didn't disturb existing behavior. +- One or more `resource=value` pairs: checks exactly those, e.g. + `/xce-test-resources r15m=1.5 pg=3`. +- Also accepts an optional scratch/deploy directory and a "keep running" flag + (leaves the deployment up instead of cleaning up on exit) — exact flag/env + names are pinned during implementation (tasks 5.1/7.1), following + `xce-test-ha`'s `[rounds] [dir]` positional-argument convention and + `XCE_HA_TOP`-style env vars as the closest precedent. + +## VERDICT line and PASS/FAIL oracle + +Following `xce-test-ha`'s machine-parseable VERDICT convention: + +``` +[xce-test-resources] VERDICT resources=7 layers_checked=3 mismatches=0 result=PASS +``` + +- `resources` — count of resource names checked this run (7 for the + canonical Requirement-1-gap-only run shown above; a default no-args run + that also includes the regression names, or an explicit `resource=value` + list, reports whatever count was actually checked). +- `layers_checked` — always 3 (controller, scheduler, apiserver) for a full + run. +- `mismatches` — count of (resource, layer) pairs where the observed value + did not match the injected value, or the layer was unreachable + (trace-log grep timed out). `result=PASS` iff `mismatches == 0`. +- `result` — `PASS` or `FAIL`. + +Per-resource, per-layer detail (injected vs. observed value) is reported +above the VERDICT line for anything that mismatched, distinguishing a genuine +**mismatch** (wrong value observed) from **unreachable** (bounded wait timed +out with no matching trace line) — the same distinction Requirement 4.3/5.4 +and design.md's Error Handling require, since the two point at different bugs +(a value-propagation defect vs. a layer that never came up or never logged). + +Alongside the VERDICT line, the skill emits the usual terminal oracle line so +a goal-loop or CI-style caller can branch on it without parsing the VERDICT +line's fields: + +``` +[xce-test-resources] PASS: 7 resources, 3 layers checked, zero mismatches +[xce-test-resources] FAIL: resource=r15m layer=scheduler mismatch (injected=1.5 observed=timeout) +``` + +PASS ⇒ exit `0`. Any FAIL condition ⇒ non-zero exit. + +## Error handling this skill will implement + +Per design.md's Error Handling section: + +- **Unrecognized resource name** — `resupdate`'s own follow-up + `xcecli_resval()` verification (task 2.2) reports it as rejected; the skill + treats this as a run-level failure and names the specific rejected + resource(s) in its report. +- **Layer mismatch** — reported with both the injected and observed value, + attributed to the specific layer (controller/scheduler/apiserver). +- **Layer unreachable** (e.g. apiserver never came up, or the expected + trace line never appears before the bounded wait elapses) — treated the + same as a mismatch for verdict purposes (FAIL), but labeled "timeout" + distinctly from "mismatch" in the report, never a silent skip. +- **Deployment failure** (controller/scheduler/apiserver fails to start) — + the skill aborts before attempting injection, with a clear "deployment + failed at step X" message. Cleanup still runs on this path: a trap (or + equivalent) is installed covering every deployment step from the first + process start onward, not just the post-injection verification path — + deliberately stronger than `xce-deploy-ha`'s own precedent (a separate, + manually-invoked `cleanup.sh` that a mid-deploy failure would otherwise + leave for the caller to run by hand). + +## Not using `context: fork` for the eventual orchestration script + +Neither this skill's frontmatter nor the invocation instructions below will +set `context: fork`, for the same reason `xce-test-ha` and `xce-deploy-ha` +both call out explicitly in their own `SKILL.md`s: this skill's +`run-resource-verify.sh` will invoke `xce-deploy-ha`'s `start-all.sh` +internally, starting real backgrounded controller/scheduler/apiserver +processes for the duration of the run. `context: fork`'s own teardown sends +SIGHUP through the whole process group on return — the same confirmed-broken +mechanism `xce-test-ha`'s soak driver and `xce-deploy-ha` both hit (bug +confirmed 2026-07-08) — that would silently kill those processes mid-run, +before cleanup or the verdict line ever fires. Dispatch it directly via Bash, +or via the `Agent` tool +(empirically verified safe for this exact pattern — see `xce-test-ha`'s and +`xce-deploy-ha`'s own `SKILL.md`s), not via a forked Skill-tool context. + +## Dependencies (not yet built) + +- `xce_test`'s worker-context `resupdate`/`join` and client-context `resval` + — existing, but `resupdate`'s resource-name table is currently incomplete + (task 2.1 closes this). +- `xce-deploy-ha`'s `deploy-ha.sh` / `start-all.sh` — controller deployment + exists today; the apiserver step and combined `start-all.sh` are task 3.1. +- Scheduler and apiserver trace-log lines — tasks 4.1 and 4.2. +- This skill's own `run-resource-verify.sh` — tasks 5.1–7.1. + +## Requirements traceability + +This stub documents the contract for design.md's "`xce-test-resources` +skill" component (Components and Interfaces → Test Tooling / Bash), covering +Requirements 3.1, 3.2, 4.1, 4.2, 4.3, 5.1, 5.2, 5.4, 6.1, 6.2, 6.3, 6.4, 6.5. +See `.kiro/specs/resource-value-verification/requirements.md` and `design.md` +for the full text. diff --git a/.kiro/specs/resource-value-verification/tasks.md b/.kiro/specs/resource-value-verification/tasks.md index 0aabd35b..55ad1965 100644 --- a/.kiro/specs/resource-value-verification/tasks.md +++ b/.kiro/specs/resource-value-verification/tasks.md @@ -8,7 +8,7 @@ - Observable completion: the new module and test target both compile and link via `meson`, and the new test target fails (RED) because the fallback-to-numeric and type-override behavior don't exist yet - _Requirements: 1.1, 1.4_ -- [ ] 1.2 (P) Scaffold the xce-test-resources skill directory +- [x] 1.2 (P) Scaffold the xce-test-resources skill directory - Create the skill directory with a `SKILL.md` stub describing the intended usage (`/xce-test-resources [resource=value ...]`) and the VERDICT line format, following the `xce-test-ha`/`xce-deploy-ha` SKILL.md convention - Observable completion: the skill is discoverable/invocable (even though its underlying script is not yet functional) and its documented contract matches design.md's Batch/Job Contract for the skill - _Requirements: 6.1_