Validation task 4.3 found a real, deterministic conflict with the original requirement 2.2 wording, not a flake: /xce-build controller alone's default asan profile transitively rebuilds harness/xce_test, because Meson's test-runner rebuilds every registered test's build- order dependencies project-wide before filtering by --suite, and harness_unit_test (task 2.1) depends on it. Confirmed via build-log step-count analysis (explicit compile: 209 steps, clean; subsequent meson test: 844 steps, resurrects it) and independently reproduced twice, with an exact step-count match both times. User decision (2026-09-14): accept as a documented Meson limitation rather than auditing every component's test-target list (controller's own suite already relies on this same mechanism for 4 of its own test binaries) or weakening harness_unit_test's build-order guarantee (verified load-bearing in task 4.2). Requirement 2.2 narrowed to what the build system directly controls: its explicit compile-target list never names xce_test. requirements.md/design.md/tasks.md updated to match and cross-reference each other; logged as bug-1806 (accepted, not fixed) for the record. Also commits the xce-harness-component spec's remaining docs (requirements.md, brief.md, research.md, spec.json) and the roadmap.md promotion entry from discovery, none of which had been committed yet. Task 4.3 of xce-harness-component. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLSNJhNoULvVcUhXEFafQC
349 lines
33 KiB
Markdown
349 lines
33 KiB
Markdown
# Design Document: xce-harness-component
|
|
|
|
## Overview
|
|
|
|
**Purpose**: This feature relocates `xce_test` — an already-decoupled wire-protocol test client — out of `src/controller/testsuite/` into its own standalone `src/harness/` component, giving it an independent build target, its own unit-test binary, and its own documentation, with zero change to its runtime behavior.
|
|
|
|
**Users**: XCE engineers who build the project, run HA soak / multi-host / resource-verification workflows, or maintain `xce_test` itself.
|
|
|
|
**Impact**: Changes `src/controller/meson.build` (removes the `xce_test` executable, 13 test-file entries from `controller_unit_test_src`, and 4 standalone test-binary blocks), `controller_test.c` (removes 13 extern/RUN_TEST entries), `src/tools/xce-build.sh` (decouples the `controller` build case, adds a `harness` case), four downstream skills, and three steering/documentation files. No change to `xce_test`'s command surface, output, or the controller's own domain test coverage.
|
|
|
|
### Goals
|
|
- `xce_test` becomes buildable, testable, and documented as an independent component (`src/harness/`).
|
|
- Building `controller` no longer implicitly builds `xce_test`, and vice versa.
|
|
- Every white-box/integration test that validates one of xce_test's own source files keeps running, now attributed to the harness's own test suite.
|
|
- Every downstream workflow that assumed the old coupling keeps working.
|
|
|
|
### Non-Goals
|
|
- No new `xce_test` command, flag, or output change.
|
|
- No rename of the `xce_test` binary.
|
|
- No change to `xcecli`, `xceadmin`, the Go frontend, or controller's own domain unit tests (HA, persistence, commit, replication, job).
|
|
- No nix/AppImage packaging change (xce_test is not part of either today).
|
|
|
|
## Boundary Commitments
|
|
|
|
### This Spec Owns
|
|
- The `src/harness/` component directory: its `meson.build`, `CLAUDE.md`, production sources, and test sources.
|
|
- The `xce_test` executable and its five test binaries (`harness_unit_test`, `resval_typeresolve_unit_test`, `joblist_display_unit_test`, `join_worker_args_unit_test`, `listrow_json_unit_test`) and their meson `test()` registration.
|
|
- The `controller)` and new `harness)` cases in `src/tools/xce-build.sh`, and root `CLAUDE.md`'s component list.
|
|
- The relocation of the one genuinely shared header, `ctrlstatus_fmt.h`, to `src/include/xce/` (the existing cross-daemon shared-header prefix, e.g. `cluster.h`, `types.h`, `errnox.h` — not the main libxce API headers).
|
|
- Updates to the four downstream skills (`xce-test-ha`, `xce-deploy-ha`, `xce-test-multihost`, `xce-test-resources`) so they no longer assume the old coupling.
|
|
- Updates to `structure.md` and `build-deployment.md` describing the new component.
|
|
|
|
### Out of Boundary
|
|
- Any xce_test behavioral change (owned by `xce-test-cli-ux`, `xce-test-e2e-primitives`, `xce-test-rc-file`, and the still-open roadmap item for further WAL/snapshot inspectors).
|
|
- Controller's own domain test files (`test_ha_*.c`, `test_persist_*.c`, `test_commit_*.c`, etc.) — these stay in `src/controller/testsuite/` untouched.
|
|
- Any change to `xcecli`/`xceadmin`/Go frontend build wiring.
|
|
- Re-litigating which controller tests belong where beyond the 17 files whose production counterpart is physically moving.
|
|
|
|
### Allowed Dependencies
|
|
- The harness component may depend on libxce (`src/lib/`) and its existing dependency set (`librtti`, `struct2json`, `libedit`, `tinylog`, `zlib`, `c_capnproto`, `openssl`, `thread`) — unchanged from today.
|
|
- The harness component may depend on `src/include/xce/ctrlstatus_fmt.h` once promoted there — the one small, header-only, link-free utility shared with controller's own test suite. Placed under the existing `xce/` cross-daemon shared-header prefix (not bare `src/include/`) so it doesn't read as part of libxce's real runtime API surface. This is the **only** sanctioned harness→controller-adjacent dependency; no other controller header or source file may be referenced.
|
|
- `harness_unit_test` may declare a build-order dependency on the `xce_test` executable target (two of its tests spawn that binary directly) — this is an intra-harness dependency, not a cross-component one.
|
|
- **Sanctioned exception, discovered during task 1.1 (not part of the original design)**: `controller_unit_test` may compile `waldump.c`, `walcompare.c`, `snapverify.c`, and `snapdiff.c` a second time directly from `src/harness/`, because four controller-domain regression tests (`test_config_repl.c`, `test_ha_core_safety.c`, `test_upgrade_snapshot_apply.c`, `test_bug857_regression.c`) call their functions directly and are themselves out of this spec's boundary (controller's own domain tests, untouched). This is a **controller→harness** dependency — the reverse direction from the `ctrlstatus_fmt.h` exception above — and does not violate Requirement 1.4 (which constrains harness's dependencies on controller, not the reverse). It is permanent, not a migration artifact to be removed by a later task.
|
|
|
|
### Revalidation Triggers
|
|
- Any future xce_test source file that grows a `#include` of a controller-private header (anything under `src/controller/` other than `src/include/xce/ctrlstatus_fmt.h`) breaks this spec's boundary and must be caught in review.
|
|
- Any new downstream skill/script that assumes xce_test exists without an explicit harness-build step must be added to Requirement 5's coverage.
|
|
- If `ctrlstatus_fmt.h` ever grows beyond a header-only `static inline` function (e.g. needs a `.c` file with real controller state), the shared-header decision must be revisited.
|
|
|
|
### Invariant Cross-Check
|
|
This spec touches zero controller/scheduler runtime code paths (async I/O thread-role placement, fail-loud/never-self-exit process lifecycle, consensus/replication safety, commit-before-materialize ordering) and adds no persistence/save-restore mechanism — it moves build/test infrastructure and one header-only formatting helper only. `xce-audit-invariants`' controller/scheduler dimension lists and `persistence.md`'s save/restore field-completeness rule are therefore both **not applicable**; noted here so a later reviewer does not have to re-derive that determination.
|
|
|
|
## Architecture
|
|
|
|
### Existing Architecture Analysis
|
|
|
|
Today, `xce_test`'s 15 source files live in `src/controller/testsuite/` and are compiled into **six** separate binaries by `src/controller/meson.build`: the `xce_test` executable itself; 10 of the 15 production files recompiled a second time into `controller_unit_test` alongside their white-box tests (plus 3 more test files with no separate production pairing — `test_xcetest_interrupt.c`, `test_test_cli.c`, `test_xcetest_process_entry.c` — 13 test files total in that group); and the remaining 4 production files (`resval_typeresolve.c`, `joblist_display.c`, `join_worker_args.c`, `listrow_json.c`) each recompiled into their own already-standalone test binary (`resval_typeresolve_unit_test`, `joblist_display_unit_test`, `join_worker_args_unit_test`, `listrow_json_unit_test`). `src/tools/xce-build.sh`'s `controller)` case compiles all of these together and is the sole mechanism that currently makes `xce_test` buildable. A confirmed dependency audit (see `research.md`) found one genuinely shared header (`ctrlstatus_fmt.h`, header-only) between controller's own tests and xce_test's sources. **Correction discovered during task 1.1 implementation**: the audit's "already fully decoupled at the source level" claim was incomplete — four controller-domain regression tests (`test_config_repl.c`, `test_ha_core_safety.c`, `test_upgrade_snapshot_apply.c`, `test_bug857_regression.c`) call functions directly from four of xce_test's own production files (`snapdiff_paths`/`waldump_path_visit`/`walcompare`/`snapverify_path`, i.e. `waldump.c`, `walcompare.c`, `snapverify.c`, `snapdiff.c`). This is a real, permanent controller→harness compile dependency, not a temporary migration artifact — `controller_unit_test_src` now recompiles those four files a second time via a relative path into `src/harness/`. See the updated Allowed Dependencies below. This spec otherwise formalizes the harness/controller split into the build graph and directory layout, preserving the existing 5-test-binary shape (`harness_unit_test` plus the four already-standalone binaries) rather than consolidating it.
|
|
|
|
### Architecture Pattern & Boundary Map
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph Before
|
|
CtrlBuildOld[controller build case]
|
|
XceTestOld[xce_test executable]
|
|
CtrlUnitOld[controller_unit_test executable]
|
|
SharedHeaderOld[ctrlstatus_fmt header]
|
|
CtrlBuildOld --> XceTestOld
|
|
CtrlBuildOld --> CtrlUnitOld
|
|
CtrlUnitOld --> SharedHeaderOld
|
|
XceTestOld --> SharedHeaderOld
|
|
end
|
|
subgraph After
|
|
CtrlBuildNew[controller build case]
|
|
HarnessBuildNew[harness build case]
|
|
XceTestNew[xce_test executable]
|
|
HarnessUnitNew[harness_unit_test executable]
|
|
CtrlUnitNew[controller_unit_test executable]
|
|
SharedHeaderNew[shared include ctrlstatus_fmt]
|
|
CtrlBuildNew --> CtrlUnitNew
|
|
HarnessBuildNew --> XceTestNew
|
|
HarnessBuildNew --> HarnessUnitNew
|
|
HarnessUnitNew --> XceTestNew
|
|
HarnessUnitNew --> SharedHeaderNew
|
|
CtrlUnitNew --> SharedHeaderNew
|
|
end
|
|
```
|
|
|
|
**Architecture Integration**:
|
|
- **Selected pattern**: Directory/build-graph split following the project's existing C-daemon component pattern, with the flat-root variant controller already established as an exception.
|
|
- **Domain/feature boundaries**: harness owns everything that is xce_test-specific (its 15 sources + 17 tests, across 5 test binaries — see below); controller keeps everything that is controller-domain (its own daemon sources + its own domain tests); the one shared header moves to the pre-existing neutral `xce/` shared-header prefix (`src/include/xce/`) rather than living inside either component.
|
|
- **Existing patterns preserved**: component-per-directory with its own `meson.build`/`CLAUDE.md`; `/xce-build <component>` dispatch pattern in `xce-build.sh`; meson `test(..., suite: '<name>')` registration pattern; the project's existing practice of splitting a lean, dependency-light module into its own standalone Unity binary rather than folding it into one monolith (already used today for `resval_typeresolve_unit_test`, `joblist_display_unit_test`, `join_worker_args_unit_test`, `listrow_json_unit_test`) — **preserved as-is, not consolidated** (user decision, validate-design review).
|
|
- **New components rationale**: `src/harness/` is needed because no existing component directory can host xce_test without re-creating the same coupling this spec removes. Five distinct test-binary targets move with it: the ~13-file group that lives inside `controller_unit_test` today becomes a new `harness_unit_test` binary (needs its own new Unity runner, since `controller_test.c` stays with controller's own remaining tests), and the four already-standalone binaries (`resval_typeresolve_unit_test`, `joblist_display_unit_test`, `join_worker_args_unit_test`, `listrow_json_unit_test`) relocate verbatim with only their `suite:` tag changing from `controller` to `harness`.
|
|
- **Steering compliance**: matches `structure.md`'s "component-per-directory, own Meson build + CLAUDE.md" philosophy; matches `build-deployment.md`'s three-part `setup_build` / `meson compile` / `meson test` dispatch pattern per component.
|
|
|
|
### Technology Stack
|
|
|
|
| Layer | Choice / Version | Role in Feature | Notes |
|
|
|-------|------------------|------------------|-------|
|
|
| Build | Meson ≥ 1.5 (unchanged) | New `src/harness/meson.build`; `subdir('harness')` added to `src/meson.build` | Same build system, no version change |
|
|
| Build orchestration | `src/tools/xce-build.sh` (bash) | New `harness)` case; `controller)` case trimmed | Mirrors the existing `scheduler)`/`worker)` case template exactly |
|
|
| C toolchain | gnu17 (unchanged) | Harness sources compile with the same standard/flags as controller | No new dependency |
|
|
| Test framework | Unity (unchanged) | New `harness_unit_test` Unity runner, modeled on `controller_test.c` | Reuses the existing `-DXCE_TESTING` pattern where applicable |
|
|
|
|
No new external dependencies are introduced; the harness reuses xce_test's existing dependency set unchanged (`libxce`, `librtti`, `struct2json`, `libedit`, `tinylog`, `zlib`, `c_capnproto`, `openssl`, `thread`).
|
|
|
|
## File Structure Plan
|
|
|
|
### Directory Structure
|
|
```
|
|
src/
|
|
├── harness/ # NEW component (this spec)
|
|
│ ├── meson.build # xce_test executable + FIVE test executables (see below)
|
|
│ ├── CLAUDE.md # component guide
|
|
│ ├── include/ # the 15 xce_test-local headers (waldump.h, rcfile.h, ...)
|
|
│ ├── test.c # REPL/dispatch entry point (unchanged content, new location)
|
|
│ ├── waldump.c ... listrow_json.c # remaining 14 production files (unchanged content)
|
|
│ └── testsuite/ # the 17 white-box / integration test files, across 5 binaries:
|
|
│ ├── harness_test.c # NEW Unity runner (modeled on controller_test.c) for the
|
|
│ │ # 13-file group below -- this is the ONE binary that is new
|
|
│ ├── test_waldump.c, test_walcompare.c, test_snapverify.c, test_snapdiff.c,
|
|
│ │ test_configcompare.c, test_rcfile.c, test_flagspec.c, test_xcetest_interrupt.c,
|
|
│ │ test_interruptcall.c, test_confirm.c, test_listfmt_json.c, test_test_cli.c,
|
|
│ │ test_xcetest_process_entry.c # 13 files -> new "harness_unit_test" binary
|
|
│ ├── test_resval_typeresolve.c # -> "resval_typeresolve_unit_test" (relocated as-is)
|
|
│ ├── test_joblist_display.c # -> "joblist_display_unit_test" (relocated as-is)
|
|
│ ├── test_join_worker_args.c # -> "join_worker_args_unit_test" (relocated as-is)
|
|
│ └── test_listrow_json.c # -> "listrow_json_unit_test" (relocated as-is)
|
|
├── controller/
|
|
│ ├── meson.build # MODIFIED: remove xce_test_src/xce_test_deps/xce_test executable;
|
|
│ │ # remove the moved entries from controller_unit_test_src;
|
|
│ │ # remove the 4 standalone test-binary blocks that move
|
|
│ │ # (resval_typeresolve_unit_test, joblist_display_unit_test,
|
|
│ │ # join_worker_args_unit_test, listrow_json_unit_test)
|
|
│ └── testsuite/
|
|
│ └── controller_test.c # MODIFIED: remove the 13 extern + RUN_TEST entries for the
|
|
│ # functions moving into the new harness_unit_test runner
|
|
│ # (test_ctrlstatus stays -- it tests controller's own ctrlstatus.c)
|
|
└── include/
|
|
└── xce/
|
|
└── ctrlstatus_fmt.h # MODIFIED location: promoted here from controller/testsuite/
|
|
```
|
|
|
|
> `src/harness/` follows controller's own documented flat-root exception (files directly in the component root, not nested under `src/`) rather than the general worker/scheduler `src/`+`include/` nested pattern — see `research.md`'s Architecture Pattern Evaluation for why.
|
|
|
|
### Modified Files
|
|
- `src/controller/meson.build` — remove the `xce_test_src`/`xce_test_deps`/`xce_test` executable block; remove the 13 harness-owned entries from `controller_unit_test_src`; remove the four standalone test-binary blocks that move (`resval_typeresolve_unit_test`, `joblist_display_unit_test`, `join_worker_args_unit_test`, `listrow_json_unit_test`, each currently its own `executable()`+`test()` pair).
|
|
- `src/controller/testsuite/controller_test.c` — remove the 13 `extern`/`RUN_TEST` entries (lines 68-80 today: `test_waldump` through `test_test_cli`) for the functions moving into the new harness Unity runner; `test_ctrlstatus`'s extern/RUN_TEST stays (it tests controller's own `ctrlstatus.c`).
|
|
- `src/meson.build` — add `subdir('harness')`.
|
|
- `src/tools/xce-build.sh:292-299` — drop `controller/xce_test` from the `controller)` case's compile target list; add a new `harness)` case following the identical `setup_build` / `meson compile` / conditional `meson test --suite harness` template already used by `scheduler)`/`worker)`.
|
|
- `CLAUDE.md` (root) — add `harness` to the `/xce-build` component list; add `src/harness/` to Subdirectory Guidelines pointing at its new `CLAUDE.md`.
|
|
- `.kiro/steering/build-deployment.md` — remove the "controller always builds xce_test" claim (current line 24); add `harness` to the `/xce-build` command table and the Meson Structure tree.
|
|
- `.kiro/steering/structure.md` — add `harness` to the component table and to the flat-root Exceptions list alongside controller.
|
|
- `.claude/skills/xce-test-ha/run-ha-soak.sh`, `.claude/skills/xce-test-ha/SKILL.md` — ensure the harness is built before the soak runs.
|
|
- `.claude/skills/xce-deploy-ha/SKILL.md` — document the harness under the existing soft-skip pattern (same treatment as worker/scheduler/xceadmin).
|
|
- `.claude/skills/xce-test-multihost/run-multihost-verify.sh`, `.claude/skills/xce-test-multihost/SKILL.md` — ensure the harness is built before verification runs.
|
|
- `.claude/skills/xce-test-resources/run-resource-verify.sh`, `.claude/skills/xce-test-resources/SKILL.md` — ensure the harness is built before verification runs.
|
|
|
|
## System Flows
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Engineer
|
|
participant XceBuild as xce build script
|
|
participant Meson
|
|
participant XceTestBin as xce_test binary
|
|
participant HarnessTests as harness_unit_test
|
|
|
|
Engineer->>XceBuild: request harness build
|
|
XceBuild->>Meson: compile harness targets
|
|
Meson->>XceTestBin: build xce_test executable
|
|
Meson->>HarnessTests: build harness_unit_test
|
|
Note over Meson,HarnessTests: harness_unit_test depends on<br/>XceTestBin already being built
|
|
XceBuild->>Meson: run test suite harness
|
|
HarnessTests->>XceTestBin: spawn over pty for CLI and process entry tests
|
|
HarnessTests-->>XceBuild: pass fail report under suite harness
|
|
```
|
|
|
|
**Key decision**: `harness_unit_test`'s two subprocess-spawning tests (`test_test_cli.c`, `test_xcetest_process_entry.c`) require `xce_test` to already be built — `src/harness/meson.build` declares this as an explicit target dependency so meson's own build graph enforces the order, rather than relying on incidental compile-order luck.
|
|
|
|
## Requirements Traceability
|
|
|
|
| Requirement | Summary | Components | Flows |
|
|
|-------------|---------|------------|-------|
|
|
| 1.1, 1.2 | Standalone directory matching worker/scheduler-style structure | src/harness/ layout | — |
|
|
| 1.3 | Binary name unchanged | xce_test executable target | — |
|
|
| 1.4 | No controller-private dependency | ctrlstatus_fmt.h relocation to src/include/xce/ | — |
|
|
| 2.1, 2.2 | Independent, decoupled build | xce-build.sh harness)/controller) cases | Build & test sequence |
|
|
| 2.3 | Harness is a selectable component | xce-build.sh, root CLAUDE.md | — |
|
|
| 2.4 | All-components build includes harness | src/meson.build subdir('harness') | — |
|
|
| 3.1, 3.2, 3.4 | Unit tests move and report under harness's own suite, as 5 separate binaries | harness unit-test suite (5 binaries), meson test(suite: harness) | Build & test sequence |
|
|
| 3.3 | Controller's own suite unaffected | controller_unit_test + controller_test.c (trimmed, not gutted) | — |
|
|
| 4.1, 4.2, 4.3 | Behavioral parity | xce_test executable (content unchanged) | — |
|
|
| 5.1, 5.2, 5.3 | Downstream workflows keep working | xce-test-ha, xce-deploy-ha, xce-test-multihost, xce-test-resources | — |
|
|
| 5.4 | Clear, actionable failure when harness missing | same four skills | — |
|
|
| 6.1, 6.2, 6.3 | Documentation consistency | src/harness/CLAUDE.md, build-deployment.md, structure.md | — |
|
|
|
|
## Components and Interfaces
|
|
|
|
| Component | Domain/Layer | Intent | Req Coverage | Key Dependencies (P0/P1) | Contracts |
|
|
|-----------|--------------|--------|---------------|--------------------------|-----------|
|
|
| harness component (xce_test) | C CLI / build | Standalone wire-protocol test client, structurally unchanged | 1.1-1.4, 4.1-4.3 | libxce (P0), src/include/xce/ctrlstatus_fmt.h (P1) | Batch |
|
|
| harness unit-test suite (5 binaries) | C test binaries | Runs the 17 moved white-box/integration tests, preserved as 5 separate binaries (not consolidated) under the `harness` suite | 3.1-3.4 | xce_test executable (P0, build-order, `harness_unit_test` only) | Batch |
|
|
| xce-build.sh dispatch | Build orchestration | Independent `harness` build entry, decoupled `controller` entry | 2.1-2.4 | Meson (P0) | Batch |
|
|
| Downstream verification skills | Workflow automation | Build/verify harness exists before use, with a clear failure path | 5.1-5.4 | harness component (P0) | Batch |
|
|
| Component/steering docs | Documentation | Describe harness accurately | 6.1-6.3 | — | — |
|
|
|
|
### Build & Test Domain
|
|
|
|
#### Harness Component (xce_test)
|
|
|
|
| Field | Detail |
|
|
|-------|--------|
|
|
| Intent | Wire-protocol REPL test client, relocated with no behavioral change |
|
|
| Requirements | 1.1, 1.2, 1.3, 1.4, 4.1, 4.2, 4.3 |
|
|
|
|
**Responsibilities & Constraints**
|
|
- Owns all 15 xce_test production source files and their headers.
|
|
- Must not include any controller-private header or source file (Allowed Dependencies lists the one sanctioned exception).
|
|
- Must produce byte-identical CLI/REPL output to today's binary.
|
|
|
|
**Dependencies**
|
|
- Inbound: `harness_unit_test` (P0) — two of its tests spawn this binary directly.
|
|
- Outbound: libxce (P0), `src/include/xce/ctrlstatus_fmt.h` (P1, via `test.c`/`joblist_display.c`).
|
|
- External: libedit, OpenSSL, zlib, Cap'n Proto runtime — unchanged from today.
|
|
|
|
**Contracts**: Batch [x]
|
|
|
|
##### Batch / Job Contract
|
|
- Trigger: `/xce-build harness` (new) or `/xce-build all`.
|
|
- Input / validation: none beyond today's existing CLI argument parsing (unchanged).
|
|
- Output / destination: `<build_dir>/harness/xce_test` binary.
|
|
- Idempotency & recovery: a rebuild simply recompiles; no state to recover.
|
|
|
|
**Implementation Notes**
|
|
- Integration: `src/harness/meson.build` defines this executable exactly as `src/controller/meson.build` does today, just relocated.
|
|
- Validation: rerun the existing `xce-test-cli-ux`/`xce-test-e2e-primitives`/`xce-test-rc-file` test coverage against the relocated binary and diff output against a pre-move capture.
|
|
- Risks: none beyond the header relocation already resolved in `research.md`.
|
|
|
|
#### Harness Unit-Test Suite (5 binaries)
|
|
|
|
| Field | Detail |
|
|
|-------|--------|
|
|
| Intent | Unity-based unit/integration test coverage for harness's own sources, preserved as 5 separate binaries matching today's structure (validate-design review: consolidation rejected in favor of file-for-file fidelity) |
|
|
| Requirements | 3.1, 3.2, 3.3, 3.4 |
|
|
|
|
**Responsibilities & Constraints**
|
|
- **`harness_unit_test`** (new binary, new name) — the 13 files that live inside `controller_unit_test` today (`test_waldump.c` through `test_test_cli.c`, listed in the File Structure Plan) plus their production pairs, with a **new** Unity runner `harness_test.c` (modeled on `controller_test.c`, carrying only these 13 `extern`/`RUN_TEST` entries).
|
|
- **`resval_typeresolve_unit_test`**, **`joblist_display_unit_test`**, **`join_worker_args_unit_test`**, **`listrow_json_unit_test`** — relocate verbatim from `src/controller/meson.build` (each already a self-contained 2-3 file `executable()`/`test()` pair with no shared runner); only their `include_directories`/`dependencies` source path and `suite:` tag change (`controller` → `harness`).
|
|
- All five register under the meson suite name `harness`, distinct from `controller`.
|
|
- `harness_unit_test` specifically must not silently pass if the `xce_test` executable its two pty-spawning tests (`test_test_cli.c`, `test_xcetest_process_entry.c`) rely on is stale or missing — that build-order dependency must be a hard prerequisite, not an informal expectation. The other four binaries have no such dependency (they never spawn `xce_test`).
|
|
|
|
**Dependencies**
|
|
- Inbound: none (leaf targets).
|
|
- Outbound: `xce_test` executable (P0, build-order — `harness_unit_test` only, declared via meson's target-dependency mechanism), `src/include/xce/ctrlstatus_fmt.h` (P1 — `joblist_display_unit_test` only, since `joblist_display.c` is its production file).
|
|
|
|
**Contracts**: Batch [x]
|
|
|
|
##### Batch / Job Contract
|
|
- Trigger: `/xce-build harness`'s test step, or `meson test -C <build_dir> --suite harness`.
|
|
- Input / validation: none.
|
|
- Output / destination: Unity `PASS`/`FAIL` report per test, aggregate suite result per binary.
|
|
- Idempotency & recovery: rerunning is safe; no persisted state.
|
|
|
|
**Implementation Notes**
|
|
- Integration: the four standalone binaries keep the exact registration pattern they already have today (`test('<name> test', <binary>, suite: 'harness', timeout: 60, env: [...])`); `harness_unit_test` mirrors `controller_unit_test`'s own pattern (`suite: 'harness'`, `timeout: 300`).
|
|
- Validation: verify via meson's own dependency graph (not just documentation) that `harness_unit_test` cannot start before `xce_test` finishes building; per `testing.md`'s standing rule, cross-check each standalone binary's own stdout `:FAIL:` count, not just meson's aggregate `Ok:N Fail:0`, before treating a run as green.
|
|
- Risks: a future edit could accidentally drop the build-order dependency — `research.md`'s follow-up recommends one regression check that the two spawning tests fail loudly (not silently pass) if `xce_test` is absent. A second risk specific to this corrected design: missing one of the four standalone `executable()`/`test()` blocks during the move — mitigated by naming all four explicitly here and in the File Structure Plan rather than referring to "the test files" generically.
|
|
|
|
#### xce-build.sh Dispatch
|
|
|
|
| Field | Detail |
|
|
|-------|--------|
|
|
| Intent | Independent `harness` build entry point; `controller`'s explicit compile step no longer names xce_test (see Implementation Notes for a documented Meson test-runner exception) |
|
|
| Requirements | 2.1, 2.2, 2.3, 2.4 |
|
|
|
|
**Responsibilities & Constraints**
|
|
- `controller)` case (`xce-build.sh:292-299`) drops `controller/xce_test` from its compile target list.
|
|
- New `harness)` case follows the identical three-part template already used by `scheduler)`/`worker)`: `setup_build`, an explicit `meson compile` target list (all 6 harness targets: `xce_test`, `harness_unit_test`, and the four standalone test binaries), then a conditional `meson test --suite harness`.
|
|
- `all)` requires no code change — it already compiles with no explicit target list, so it picks up the new `subdir('harness')` automatically.
|
|
|
|
**Dependencies**
|
|
- Inbound: engineers, the four downstream verification skills.
|
|
- Outbound: Meson (P0).
|
|
|
|
**Contracts**: Batch [x]
|
|
|
|
##### Batch / Job Contract
|
|
- Trigger: `/xce-build harness [profile]`, `/xce-build controller [profile]`, `/xce-build all [profile]`.
|
|
- Input / validation: existing profile validation (`asan`/`debug`/`release`) — unchanged.
|
|
- Output / destination: component-scoped binaries under the chosen build dir.
|
|
- Idempotency & recovery: same full-wipe-vs-`--incremental` semantics that already exist for every other component pair (`build-deployment.md` bug-1565 note) — unchanged behavior, now also applicable to the harness/controller pair.
|
|
|
|
**Implementation Notes**
|
|
- Integration: no new orchestration concept — this is the same case-block template every other component already uses.
|
|
- Validation: run `/xce-build harness` alone (no prior controller build) and confirm `xce_test` is produced; run `/xce-build controller` alone and confirm its explicit compile step never targets `xce_test`; run `/xce-build all` and confirm both exist.
|
|
- Risks: none beyond the pre-existing, already-documented full-wipe behavior.
|
|
- **Known limitation, discovered and accepted in task 4.3**: under the default `asan` profile (which runs tests), `/xce-build controller`'s `meson test --suite controller` step transitively rebuilds `harness/xce_test` on disk, because Meson's test-runner rebuilds every registered test's build-order dependencies project-wide before filtering which tests actually run by `--suite` — and `harness_unit_test` (Requirement 3.2) depends on `xce_test`. Verified deterministic via build-log line analysis (the explicit `meson compile` step is clean; only the subsequent `meson test` step touches it), not a flake. Requirement 2.2 was narrowed to scope this component's guarantee to what it directly controls — its own explicit compile-target list — rather than the full transitive behavior of Meson's test infrastructure. See requirements.md's Boundary Context → "Known limitation" for the full rationale and rejected alternatives (completing every component's explicit test-target list; weakening `harness_unit_test`'s build-order dependency, which task 4.2 verified is load-bearing).
|
|
|
|
#### Downstream Verification Skills
|
|
|
|
| Field | Detail |
|
|
|-------|--------|
|
|
| Intent | Keep `/xce-test-ha`, `/xce-deploy-ha`, `/xce-test-multihost`, `/xce-test-resources` working once the old coupling is removed |
|
|
| Requirements | 5.1, 5.2, 5.3, 5.4 |
|
|
|
|
**Responsibilities & Constraints**
|
|
- Each workflow must ensure the harness is built before it is needed — either by invoking the harness build itself or by checking for the binary and failing with a clear, actionable message (matching the existing soft-skip pattern `xce-deploy-ha` already uses for worker/scheduler/xceadmin).
|
|
- Must not fail with an unrelated error (e.g. a raw "no such file" from a spawned process) when the real cause is "harness was never built."
|
|
|
|
**Dependencies**
|
|
- Outbound: harness component (P0).
|
|
|
|
**Contracts**: Batch [x]
|
|
|
|
##### Batch / Job Contract
|
|
- Trigger: `/xce-test-ha`, `/xce-deploy-ha`, `/xce-test-multihost`, `/xce-test-resources`.
|
|
- Input / validation: presence check for `<build_dir>/harness/xce_test` (or equivalent) before use.
|
|
- Output / destination: existing workflow outputs, unchanged; only the failure path for a missing binary changes (from an unrelated/confusing error to `xce-deploy-ha`'s existing clear soft-skip style).
|
|
- Idempotency & recovery: re-running after building the harness proceeds normally.
|
|
|
|
**Implementation Notes**
|
|
- Integration: `xce-deploy-ha` already has a soft-skip precedent (worker/scheduler/xceadmin) to extend to harness rather than inventing a new pattern.
|
|
- Validation: run each of the four workflows from a clean build directory (harness never built) and confirm a clear, actionable message rather than a confusing downstream failure.
|
|
- Risks: missing one of the four in the actual edit pass — mitigated by the grep-verified inventory in `research.md` (three shell scripts + four `SKILL.md` files, all named explicitly above).
|
|
|
|
## Error Handling
|
|
|
|
### Error Strategy
|
|
This feature has no runtime user-facing error surface (it is build/workflow infrastructure); "errors" here are build-time and workflow-time failure modes.
|
|
|
|
### Error Categories and Responses
|
|
- **Missing harness binary at workflow time** (Requirement 5.4): the consuming skill must report a clear, actionable message (e.g. "harness not built — run `/xce-build harness` first"), matching `xce-deploy-ha`'s existing soft-skip wording style, rather than surfacing a raw spawn/connect failure.
|
|
- **Stale harness binary vs. a fresh `xce_test` change**: prevented structurally by `harness_unit_test`'s explicit build-order dependency on the `xce_test` target (Architecture, System Flows).
|
|
- **Build-dir clobbering** (pre-existing project behavior, `build-deployment.md` bug-1565): unchanged — a harness build after a controller build in the same dir without `--incremental` wipes the controller's binaries, exactly as any other two-component pair behaves today; no new mitigation required, only awareness.
|
|
|
|
### Monitoring
|
|
No new monitoring surface — this is dev/CI-time tooling, not a running service.
|
|
|
|
## Testing Strategy
|
|
|
|
- **Unit Tests** (Requirement 3): all 17 relocated test files pass under the harness's five `suite: 'harness'` binaries (`harness_unit_test` plus the four relocated standalone binaries) with identical pass/fail semantics to their pre-move behavior; `controller_unit_test`'s remaining test count (and `controller_test.c`'s remaining extern/RUN_TEST list) is unchanged aside from the 13 entries removed for the new `harness_unit_test`, and the four standalone `executable()`/`test()` blocks are confirmed gone from `src/controller/meson.build` (no accidental loss).
|
|
- **Integration Tests** (Requirement 2): `/xce-build harness` alone (no prior controller build) produces `xce_test`; `/xce-build controller` alone does not produce `xce_test`; `/xce-build all` produces both; deliberately removing the built `xce_test` binary before running `harness_unit_test` causes `test_test_cli.c`/`test_xcetest_process_entry.c` to fail loudly (proves the build-order dependency is real, not incidental).
|
|
- **Workflow Tests** (Requirement 5): run `/xce-test-ha`, `/xce-deploy-ha`, `/xce-test-multihost`, `/xce-test-resources` starting from a clean build directory (harness never built) and confirm each either builds it or fails with a clear, actionable message — never a silent skip or an unrelated error.
|
|
- **Behavioral Parity Tests** (Requirement 4): capture `xce_test`'s command/flag/output behavior before the move (via the existing `xce-test-cli-ux`/`xce-test-e2e-primitives`/`xce-test-rc-file` coverage), rerun the same coverage against the relocated binary, and diff for any deviation.
|
|
|
|
## Supporting References
|
|
- `.kiro/specs/xce-harness-component/research.md` — full discovery log, header-dependency audit, and design-decision rationale.
|
|
- `.kiro/specs/xce-harness-component/brief.md` — original discovery brief and locked-in scope decisions.
|