myxce/.kiro/specs/xce-harness-component/tasks.md
Rongsong Shen 973d38f20b
test(xce-harness-component): verify xce_test behavioral parity after relocation
Validation-only task, no source changes. Confirmed via: file-identity
spot-checks (7 of 15 production files across two independent passes,
all zero content drift since relocation beyond the one documented
ctrlstatus_fmt.h include-path edit); a fresh harness suite run
reproducing known-good counts (4/20/5/12/132 tests, 0 failures);
byte-diffing a genuine pre-relocation binary (src/build.dbg/controller/
xce_test, predates the move by 3 days) against the harness-built binary
for --help/--usage/piped REPL commands -- identical output, confirmed
independently by two separate passes.

Found a real, non-blocking follow-up (pre-scoped as out of this spec's
boundary): two integration scripts outside the 15 relocated files
(test_rcfile_integration.sh, clean_failover_preservation.sh) hardcode
a stale default xce_test path. Logged in Implementation Notes for
whoever owns those scripts next.

Task 4.1 of xce-harness-component.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RLSNJhNoULvVcUhXEFafQC
2026-09-14 08:10:48 +00:00

108 lines
16 KiB
Markdown

# Implementation Plan: xce-harness-component
- [ ] 1. Foundation: relocate xce_test itself and establish the component
- [x] 1.1 Relocate xce_test's production sources, wire its standalone build, and strip every controller-side reference to the moved files
- Move all 15 xce_test source files (`test.c`, `waldump.c`, `walcompare.c`, `snapverify.c`, `snapdiff.c`, `configcompare.c`, `resval_typeresolve.c`, `joblist_display.c`, `join_worker_args.c`, `rcfile.c`, `flagspec.c`, `interruptcall.c`, `confirm.c`, `listfmt_json.c`, `listrow_json.c`) and their headers out of `src/controller/testsuite/` into `src/harness/`, content unchanged
- Create `src/harness/meson.build` defining only the `xce_test` executable so far (same sources/dependencies as today, paths updated); add `subdir('harness')` to `src/meson.build`
- In the same task, from `src/controller/meson.build` remove: the `xce_test_src`/`xce_test_deps`/`xce_test` executable block; every `controller_unit_test_src` entry that references one of the 15 moved files, which is 10 production+test pairs (`waldump`, `walcompare`, `snapverify`, `snapdiff`, `configcompare`, `rcfile`, `flagspec`, `interruptcall`, `confirm`, `listfmt_json`, each with its `test_*.c`) plus `test_xcetest_interrupt.c` (extra `interruptcall.c` coverage) — 11 test files total; and the four entire standalone `executable()`/`test()` blocks (`resval_typeresolve_unit_test`, `joblist_display_unit_test`, `join_worker_args_unit_test`, `listrow_json_unit_test`), since each references one of the 15 moved production files directly
- Also remove those same 11 test files' `extern`/`RUN_TEST` entries from `controller_test.c`. Leave `test_test_cli.c`'s and `test_xcetest_process_entry.c`'s entries in place — neither links any of the 15 moved files (they spawn the built `xce_test` binary as a subprocess instead), so they stay valid in `controller_unit_test` until task 2.1 relocates them
- The 11 removed test files' `.c` sources are left on disk at their current `src/controller/testsuite/` path for now (referenced by no build target) — task 2.1 physically relocates them together with `test_test_cli.c`/`test_xcetest_process_entry.c`
- Observable: a harness-scoped build produces an `xce_test` binary; a controller-scoped build run immediately after this single task (no later tasks applied) still succeeds, `controller_unit_test`'s reported test count has dropped by exactly the 11 removed entries, and none of the four standalone binaries are defined anymore
- _Requirements: 1.1, 1.2, 1.3, 2.1, 2.2, 2.4_
- [x] 1.2 Promote the shared ctrlstatus_fmt.h header and repoint its current consumers
- Move `ctrlstatus_fmt.h` to `src/include/xce/`; update the include path in harness's `test.c` and `joblist_display.c` (already relocated in 1.1) and in controller's `test_ctrlstatus.c`
- `joblist_display.c`'s own test file, `test_joblist_display.c`, still lives in `src/controller/testsuite/` at this point and is intentionally left untouched here — its include-path fix happens in task 2.2, when that file relocates
- Observable: a repo-wide search finds no remaining relative `#include "ctrlstatus_fmt.h"`; all three current consumers include it from `xce/ctrlstatus_fmt.h`
- _Requirements: 1.4_
- [x] 1.3 Write the harness component guide
- New `src/harness/CLAUDE.md` describing the component's purpose, build command, and structure, matching the style of existing component guides (e.g. `src/controller/CLAUDE.md`)
- Observable: `src/harness/CLAUDE.md` exists and documents the build command and file layout
- _Requirements: 6.1_
- [ ] 2. Core: migrate the five harness test binaries
- [x] 2.1 Consolidate the controller-embedded harness tests into a new binary and finish clearing them from the controller's own test binary
- Physically relocate the 13 test files for this group into `src/harness/testsuite/`: the 11 whose `controller_unit_test_src`/`controller_test.c` references were already removed in task 1.1 (`test_waldump.c` through `test_xcetest_interrupt.c`, left on disk at their old path until now), plus `test_test_cli.c` and `test_xcetest_process_entry.c` (still validly part of `controller_unit_test` until this task)
- Write a new Unity runner `harness_test.c` (modeled on `controller_test.c`) carrying all 13 `extern`/`RUN_TEST` entries; register the new `harness_unit_test` executable and its `meson test()` under suite `harness` in `src/harness/meson.build`; declare its build-order dependency on the `xce_test` executable target (two of its tests spawn that binary over a pty)
- In the same task, remove `test_test_cli.c`'s and `test_xcetest_process_entry.c`'s entries from `controller_unit_test_src` and their `extern`/`RUN_TEST` lines from `controller_test.c` — the only two of the 13 not already removed in task 1.1 (`test_ctrlstatus`'s entry stays — it tests controller's own `ctrlstatus.c`)
- Observable: `meson test --suite harness` runs `harness_unit_test` with the same pass count these 13 files reported before the move; `meson test --suite controller` run immediately after this same task still runs `controller_unit_test` successfully, with its test count reduced by exactly these final 2 entries (11 were already reflected in task 1.1's count drop)
- _Requirements: 3.1, 3.2, 3.3, 3.4_
- [x] 2.2 Relocate the four already-standalone harness test binaries
- Physically relocate the `test_resval_typeresolve.c`, `test_joblist_display.c`, `test_join_worker_args.c`, and `test_listrow_json.c` files into `src/harness/testsuite/` (left on disk at their old path since task 1.1 removed their controller-side `executable()`/`test()` blocks); re-create each of the four `executable()`/`test()` blocks, unchanged in shape, in `src/harness/meson.build` under suite `harness`, referencing their production files already relocated to `src/harness/` in task 1.1
- Update `test_joblist_display.c`'s `ctrlstatus_fmt.h` include path to `xce/ctrlstatus_fmt.h` as part of this same move
- Observable: `meson test --suite harness` lists all four binaries alongside `harness_unit_test`, each reporting the same pass count as before the move
- _Requirements: 1.4, 3.1, 3.2, 3.4_
- [ ] 3. Integration: decouple build orchestration, downstream workflows, and documentation
- [x] 3.1 Add the harness build entry and decouple it from the controller build
- In `src/tools/xce-build.sh`, drop `controller/xce_test` from the `controller)` case's compile target list; add a new `harness)` case following the identical three-part `setup_build` / `meson compile` / conditional `meson test --suite harness` template already used by `scheduler)`/`worker)`
- Observable: `/xce-build harness` on a fresh build dir (no prior controller build) produces the `xce_test` binary and runs the `harness` suite; `/xce-build controller` alone does not produce `xce_test`
- _Requirements: 2.1, 2.2, 2.3, 2.4_
- [x] 3.2 (P) Update the HA soak workflow's harness prerequisite
- Update `.claude/skills/xce-test-ha/run-ha-soak.sh` and `SKILL.md` so the soak ensures the harness is built (or fails with a clear message) instead of assuming a controller build produced it
- Observable: running the workflow from a build dir where only controller was built either builds the harness itself or reports a clear, actionable message instead of a raw spawn/connect failure
- _Requirements: 5.1, 5.4_
- _Boundary: xce-test-ha skill_
- _Depends: 3.1_
- [x] 3.3 (P) Extend the local HA deploy workflow's soft-skip handling to the harness
- Update `.claude/skills/xce-deploy-ha/SKILL.md` to cover the harness binary under its existing worker/scheduler/xceadmin soft-skip pattern
- Observable: deploying from a build dir without the harness built reports the same clear soft-skip style already used for a missing worker/scheduler/xceadmin binary
- _Requirements: 5.2, 5.4_
- _Boundary: xce-deploy-ha skill_
- _Depends: 3.1_
- [x] 3.4 (P) Update the multi-host verification workflow's harness prerequisite
- Update `.claude/skills/xce-test-multihost/run-multihost-verify.sh` and `SKILL.md`
- Observable: running the workflow from a build dir where only controller was built either builds the harness itself or reports a clear, actionable message
- _Requirements: 5.3, 5.4_
- _Boundary: xce-test-multihost skill_
- _Depends: 3.1_
- [x] 3.5 (P) Update the resource-value verification workflow's harness prerequisite
- Update `.claude/skills/xce-test-resources/run-resource-verify.sh` and `SKILL.md`
- Observable: running the workflow from a build dir where only controller was built either builds the harness itself or reports a clear, actionable message
- _Requirements: 5.3, 5.4_
- _Boundary: xce-test-resources skill_
- _Depends: 3.1_
- [x] 3.6 (P) Update root guide and build/deployment steering for the harness component
- Add `harness` to root `CLAUDE.md`'s `/xce-build` component list and Subdirectory Guidelines; remove `build-deployment.md`'s "controller always builds xce_test" claim and add harness to its `/xce-build` command table and Meson Structure tree
- Observable: `CLAUDE.md` lists `harness` among buildable components; `build-deployment.md` no longer claims the controller build produces `xce_test`
- _Requirements: 6.1, 6.2_
- _Boundary: CLAUDE.md, build-deployment.md_
- _Depends: 3.1_
- [x] 3.7 (P) Update structure steering to list the harness component
- Add `harness` to `structure.md`'s component table and to its flat-root Exceptions list alongside controller
- Observable: `structure.md`'s component tree and Exceptions section both mention harness
- _Requirements: 6.3_
- _Boundary: structure.md_
- _Depends: 1.1_
- [ ] 4. Validation: prove behavioral parity and end-to-end correctness
- [x] 4.1 Verify xce_test's behavioral parity after relocation
- Rerun the existing `xce-test-cli-ux` / `xce-test-e2e-primitives` / `xce-test-rc-file` test coverage against the harness-built `xce_test` binary and diff its command/flag/output behavior against a pre-move capture
- Observable: no behavioral difference is found; every previously-passing scenario in that coverage still passes unchanged
- _Requirements: 4.1, 4.2, 4.3_
- _Depends: 3.1_
- [ ] 4.2 Verify the harness_unit_test build-order dependency actually holds
- Deliberately remove the built `xce_test` binary, run `harness_unit_test`, and confirm `test_test_cli.c`/`test_xcetest_process_entry.c` fail loudly rather than silently pass; rebuild normally and confirm the suite passes
- Observable: the deliberate-removal run reports a clear failure from the two pty-spawning tests; the normal run reports the full suite green
- _Requirements: 3.1, 3.2_
- _Depends: 2.1, 3.1_
- [ ] 4.3 Verify independent and combined build behavior
- Run `/xce-build harness` alone on a clean build dir and confirm `xce_test` is produced; run `/xce-build controller` alone and confirm `xce_test` is absent; run `/xce-build all` and confirm both exist
- Observable: each of the three build invocations produces exactly the binaries described above
- _Requirements: 2.1, 2.2, 2.4_
- _Depends: 3.1_
- [ ] 4.4 Verify every downstream workflow from a clean build state
- Run the HA soak, local HA deploy, multi-host verification, and resource-value verification workflows starting from a build directory where the harness has never been built
- Observable: none of the four workflows fails with a raw/unrelated error; each either succeeds after building the harness itself or reports the expected clear, actionable message
- _Requirements: 5.1, 5.2, 5.3, 5.4_
- _Depends: 3.2, 3.3, 3.4, 3.5_
## Implementation Notes
- **Task 1.1**: discovered a real gap in the discovery/design audit — `test_config_repl.c`, `test_ha_core_safety.c`, `test_upgrade_snapshot_apply.c`, `test_bug857_regression.c` (controller's own domain tests, out of this spec's boundary) call functions directly from `waldump.c`/`walcompare.c`/`snapverify.c`/`snapdiff.c`. Fixed by having `controller_unit_test_src` recompile those four files a second time from `../harness/`. This is a permanent, sanctioned controller→harness dependency, not a migration artifact — recorded in design.md's Allowed Dependencies. Later tasks: do not assume harness has zero inbound dependency from controller; only assume harness has zero *outbound* dependency on controller (Requirement 1.4).
- **Task 1.1**: `test_test_cli.c`/`test_xcetest_process_entry.c` locate `xce_test` as a build-dir sibling via `resolve_xce_test_path()`. Since they still live in `src/controller/testsuite/` until task 2.1, a temporary `xce_test_controller_link` custom_target symlink bridge was added to `src/controller/meson.build` (documented in-code as removable once task 2.1 relocates both files). Task 2.1 must remove this custom_target and the `test()`'s `depends:` on it as part of that relocation.
- **Task 1.1 gotcha**: a meson `custom_target` command's `@INPUT0@`/`@OUTPUT0@` tokens expand to build-root-relative paths, but a symlink's target string is resolved relative to the *symlink's own directory* at dereference time, not the invocation cwd or the string's own reference frame — using `@INPUT0@` directly as an `ln -sf` target from a different subdirectory produces a broken link. Use an explicit `../`-relative (or absolute) path instead.
- **Build dir note**: this repo's default build dir resolves to `src/xce.build/` (relative to `src/tools/xce-build.sh`'s own cwd), not a bare `xce.build/` at the repo root.
- **Task 4.1**: Confirmed behavioral parity conclusively — found a genuine pre-relocation binary (`src/build.dbg/controller/xce_test`, predates commit `cf55d691` by 3 days) and byte-diffed its `--help`/`--usage`/piped-REPL output against the harness-built binary: identical (only `--version`'s embedded build-hash/timestamp differs, as expected). Real, non-blocking follow-up found (out of this spec's boundary, pre-scoped by requirements.md's Adjacent Expectations for the `xce-test-rc-file`-owned one): `src/controller/testsuite/test_rcfile_integration.sh` and `clean_failover_preservation.sh` (the latter actually belongs to `ha-term-safety` task 5.2, not one of the three named xce-test-* specs) both hardcode a stale default binary path (`src/xce.build/controller/xce_test`) and the latter's failure message still says "run /xce-build controller" — should be repointed to `src/xce.build/harness/xce_test` by whoever owns each script next.
- **Task 3.7**: Major Task 3 (Integration) is now fully complete. Task 3.6's review round caught a real doc-consistency miss (a partial fix left a contradicting stale claim elsewhere in the same file); the lesson (full end-to-end file re-read, not just the two named spots) was applied successfully in 3.7 and independently confirmed by review. Non-blocking follow-up noted for a future pass: `structure.md` line 92's generic "`test.c` provides the test runner harness" line is doubly stale (controller's runner is `controller_test.c`, harness's is `harness_test.c`) and also coincidentally collides with `src/harness/test.c`, which is real production code (xce_test's REPL entry point), not a test runner — worth a one-line correction sometime, out of this spec's boundary.
- **Task 2.2**: Major Task 2 (Core) is now fully complete — all 5 harness test binaries exist under `suite: 'harness'` (4/20/5/12/132 tests, all green), controller's suite is fully clean at 1199 tests. Minor non-blocking cleanup found for a future pass: `src/controller/meson.build:211`'s `struct2json_dep` comment still cites `test_listfmt_json.c` (relocated to harness in task 2.1) as a consumer, and its own dependency may now be unused in `controller_unit_test_deps` — worth checking whether it can be dropped entirely once someone audits it.
- **Task 2.1**: confirmed `resolve_xce_test_path()` (`/proc/self/exe` dirname + sibling lookup) needed zero code change once both `xce_test` and `harness_unit_test` build under the same `src/harness/meson.build` — they land in the same build-dir subdir automatically. Test counts cross-checked and consistent end to end: 1331 (pre-move) → 1228 (after task 1.1's 11-entry removal) → 1199 controller + 132 harness = 1331 (after task 2.1). Minor non-blocking nit for a future pass: `test_xcetest_process_entry.c`'s header comment still describes the old controller-embedded build layout — stale prose only, not a functional issue.