feat(admin-config-tool): drop ha/data/ entirely, match real client TLS
deploy-ha.sh no longer takes a <data-dir> argument or --build-config
flag. Every deploy now generates a fresh self-signed TLS cert/key
(openssl, RSA 2048, 10y validity) directly into the deploy dir's
certs/, and builds cluster.bin directly into controller/ctrl-1/conf/
via xceadmin -- no intermediate staging directory, nothing preserved
across runs. run-ha-soak.sh (/xce-test-ha) updated to match: dropped
its own XCE_HA_DATADIR-based precondition entirely. Updated 4 callers
(run-ha-soak.sh, test-heartbeat-tolerance.sh, test-commit-durability.sh,
both xce-deploy-ha/xce-test-ha SKILL.md docs).
Separately: xcecommon.wire.connect() was loading a client-side TLS
cert, diverging from the real reference client -- netio.c's
create_ssl_context() presents no certificate at all when acting as a
client (cert_file/key_file both NULL, bare TLS_client_method()).
Fixed connect() to match, and since nothing else consumed cert_dir
(verify_mode=CERT_NONE throughout, no CA check either), removed it
end-to-end: discovery.follow_redirects(), push.push_config()/
probe_config(), and the --cert-dir CLI flag.
Server-side cert loading (mock TLS test servers) is unaffected --
moved into a shared session-scoped tls_certs fixture (new
tests/conftest.py, openssl-generated) replacing 3 duplicated
REPO_HA_DATA_DIR/skipif blocks. Net result: those tests no longer skip
under a hermetic nix build sandbox (which never had ha/data/) --
verified 165/165 passed, 0 skipped, via both meson compile and a full
nix build --option sandbox false .#xcecore_debug rebuild (previously
128 passed / 37 skipped there).
ha/data/{xce.crt,xce.key} removed from git tracking. Requirements
2.3/2.4 and 8.3/8.4/8.5 marked superseded in requirements.md with
dated correction notes.
This commit is contained in:
parent
db2b5f17f0
commit
ee8473bd99
20 changed files with 244 additions and 297 deletions
|
|
@ -83,8 +83,8 @@ Criteria 8–12 apply the **Configuration Inclusion Rules** (brief.md): a field
|
|||
|
||||
1. When the operator runs `xceadmin config push <cluster.bin>`, the system shall connect to the controller via TLS, send the binary as a capnp `Request` with `OP_upgradeConfig` opcode, `upgradeconfigrequestmsg` body (binary bytes + SHA-256 hash), framed with an 8-byte uint64 size prefix in host byte order, and display the result.
|
||||
2. When the operator specifies `--controller <host:port>`, the system shall connect to the given controller address instead of the default `localhost:9991`.
|
||||
3. The system shall use the same TLS certificate bundle the controller cluster is deployed with (`xce.crt`/`xce.key`, default path `ha/data/`) to establish trust — no separate client CA flag or cert-issuance step. This matches the trust model `xce_test` and `deploy-ha.sh` already use for controller-to-controller TLS.
|
||||
4. When the operator specifies `--cert-dir <path>`, the system shall load `xce.crt`/`xce.key` from the given directory instead of the default `ha/data/`.
|
||||
3. ~~The system shall use the same TLS certificate bundle the controller cluster is deployed with (`xce.crt`/`xce.key`, default path `ha/data/`) to establish trust — no separate client CA flag or cert-issuance step.~~ **Superseded (2026-07-22)**: verified against the real client TLS setup (`netio.c`'s `create_ssl_context()`) that the actual controller-to-controller/CLI client presents **no** certificate at all when acting as a client (`cert_file`/`key_file` both `NULL` → bare `TLS_client_method()`) — only the server side loads a cert/key pair. `xceadmin`'s client previously diverged from this (loaded a client cert too); fixed to match exactly. No CA flag or cert-issuance step either way — `verify_mode=CERT_NONE` on both the real client and `xceadmin`.
|
||||
4. ~~When the operator specifies `--cert-dir <path>`, the system shall load `xce.crt`/`xce.key` from the given directory instead of the default `ha/data/`.~~ **Superseded (2026-07-22)**: removed entirely — with no client cert to load, `--cert-dir` had no remaining purpose. `ha/data/` itself was also removed (deploy-ha.sh and the HA test harness now generate a fresh server-side cert/key per run via `openssl` instead of reading a checked-in fixture).
|
||||
5. After receiving the controller's reply, the system shall display the result to the operator: `versionTag` (the new generation number assigned by the controller), `ackedCount` (number of standbys that acknowledged), and any `error` message.
|
||||
6. When the controller indicates the config was committed (positive `ackedCount`), the system shall exit 0.
|
||||
7. If the controller reports an error (non-empty `error` field or zero `ackedCount`), the system shall display the error and exit non-zero.
|
||||
|
|
@ -153,11 +153,11 @@ Criteria 8–12 apply the **Configuration Inclusion Rules** (brief.md): a field
|
|||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. When the developer runs `/xce-build xceadmin`, the system shall make the `xceadmin` CLI available on PATH and run the full test suite, reporting pass/fail.
|
||||
2. When the developer runs `/xce-build xceadmin release`, the system shall install `xceadmin` for production use (pinned, non-editable) **and then run the full test suite against that non-editable install**, reporting pass/fail — this is the only path that exercises the packaged (`pip install .`) resolution of the bundled `controller.capnp`/JSON schemas (Requirement 5.5); an editable-install-only test run would never catch a regression there.
|
||||
3. When the developer runs `/xce-deploy-ha` with the `--build-config` flag, the system shall invoke `xceadmin config build configs/ -o ha/data/cluster.bin` as a precondition before deploying the cluster.
|
||||
4. When `cluster.bin` is missing from `ha/data/` during deploy, `/xce-deploy-ha` shall run `xceadmin config build` automatically (same as `--build-config`).
|
||||
5. When `cluster.bin` already exists and `--build-config` is not passed, `/xce-deploy-ha` shall use the existing binary as-is (backward compatible).
|
||||
1. ~~When the developer runs `/xce-build xceadmin`, the system shall make the `xceadmin` CLI available on PATH and run the full test suite, reporting pass/fail.~~ **Superseded (2026-07-22)**: `xceadmin` is now meson-built (`src/xceadmin/meson.build`) — `xce-build.sh xceadmin` delegates to `meson compile -C "$BUILD_DIR" xceadmin`, and `nix build .#xcecore[_debug]` installs a standalone `result/bin/xceadmin` (no venv at runtime). See design.md's File Structure Plan Amendment.
|
||||
2. ~~When the developer runs `/xce-build xceadmin release`, the system shall install `xceadmin` for production use (pinned, non-editable) and then run the full test suite against that non-editable install...~~ **Superseded (2026-07-22)**: the editable-vs-pinned distinction collapsed — every meson-driven build (`xce-build.sh xceadmin`, `meson compile`, `nix build`) now runs the pinned/non-editable install path unconditionally, a strictly stronger guarantee (exercises real `package_data` resolution every run, not just under a `release` profile).
|
||||
3. ~~When the developer runs `/xce-deploy-ha` with the `--build-config` flag, the system shall invoke `xceadmin config build configs/ -o ha/data/cluster.bin` as a precondition before deploying the cluster.~~ **Superseded (2026-07-22)**: `--build-config` removed entirely — every `deploy-ha.sh` run now unconditionally builds `cluster.bin` fresh from `src/configs/*.yaml` directly into the deploy dir's own `controller/ctrl-1/conf/` (no `ha/data/` staging step, no flag needed since nothing is ever preserved across runs to opt out of rebuilding).
|
||||
4. ~~When `cluster.bin` is missing from `ha/data/` during deploy, `/xce-deploy-ha` shall run `xceadmin config build` automatically (same as `--build-config`).~~ **Superseded (2026-07-22)**: `ha/data/` removed entirely (directory no longer exists). See 8.3 above — always-fresh generation directly into the deploy dir replaces the missing-file check.
|
||||
5. ~~When `cluster.bin` already exists and `--build-config` is not passed, `/xce-deploy-ha` shall use the existing binary as-is (backward compatible).~~ **Superseded (2026-07-22)**: no longer applicable — nothing persists across deploys for this to preserve; every deploy is fully fresh (cert included, generated via `openssl`, self-signed, 10y validity — TLS verification is `SSL_VERIFY_NONE` everywhere in this codebase, so a fresh pair per deploy loses no real security property versus a long-lived shared one).
|
||||
6. The nix dev shell (`nix develop ./src`) shall include all Python dependencies required by xceadmin after the nix environment is updated (see Nix Environment section in brief.md).
|
||||
|
||||
### Requirement 9: Deterministic Builds
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
{
|
||||
"feature_name": "admin-config-tool",
|
||||
"created_at": "2026-07-06T01:30:00Z",
|
||||
"updated_at": "2026-07-22T10:45:00Z",
|
||||
"updated_at": "2026-07-22T14:50:00Z",
|
||||
"language": "en",
|
||||
"phase": "implementation-complete",
|
||||
"approvals": {
|
||||
|
|
|
|||
|
|
@ -162,3 +162,7 @@
|
|||
- **5.5**: Already fully satisfied by task 2.1's `test_validation.py` — every sub-case (schema violation, reserved name, cross-type duplicate collision via `test_check_duplicates_allows_limit_name_to_collide_with_queue_name`, undefined reference) already has an explicit test with the exact same requirement list. No new tests written.
|
||||
- **5.6**: `probe_config()`'s dry-run has no reachable `accepted=False` path — `ProbeConfigTagReply` genuinely carries no rejection signal on the wire (see task 3.3's note). 5.6's bullet asking for both an accepted *and* a rejected outcome cannot be satisfied without inventing undocumented client-side rejection logic; instead added `test_probe_config_has_no_wire_level_rejection_signal_by_design` (parametrized over several `configTag` values) that locks in and documents the limitation explicitly rather than leaving it silently uncovered. Added two new end-to-end propagation tests through `push_config`'s real composition with `xceadmin.discovery` (not just discovery.py's own unit tests): a real 4-mock-server redirect chain proving `TooManyRedirectsError` surfaces correctly, and a `NoPrimaryError`-on-timeout test that monkeypatches `xcecommon.wire.connect` to raise `ConnectionTimeoutError` immediately rather than waiting out a real 30s socket timeout.
|
||||
- **Post-implementation, out-of-band (2026-07-22)**: reopened and reversed the 2026-07-21 "not wired into meson" Design Review correction — see `design.md`'s File Structure Plan Amendment for full rationale and verification. Summary: added `src/xceadmin/meson.build` (`custom_target()`, stamp-file output, `build_always_stale: true`) + `src/xceadmin/meson-build.sh` (out-of-tree install+test, `repo-mirror/` symlink staging so `src/configs/`, `src/schema/`, `ha/data/` relative-path test assumptions still resolve from a copied location); `src/meson.build` gained `subdir('xceadmin')`; `src/tools/xce-build.sh`'s `xceadmin)` case now delegates to `meson compile -C "$BUILD_DIR" xceadmin` instead of its own pip/venv logic. Verified both `meson compile` (165/165 passed against a real dev checkout with certs present) and `nix build --option sandbox false .#xcecore_debug` (128 passed / 37 skipped — cert-dependent tests correctly skip under nix's git-tracked-only hermetic source, since `ha/data/xce.crt`/`.key` are intentionally not committed). AppImage bundling (README 3.5) explicitly deferred, not part of this change. Net effect on Requirement 8: the editable-vs-pinned distinction (8.1 vs. 8.2) collapses — every `xce-build.sh xceadmin` run is now the pinned/meson-driven path, a strictly stronger guarantee than the old default.
|
||||
- **Post-implementation, out-of-band (2026-07-22, second pass)**: `meson-build.sh`/`meson-install.sh` no longer resolve `python3` via a bare PATH lookup — `meson.build` now pins it once via `import('python').find_installation('python3')` (same mechanism as `find_program('go')`/capnp/packcc elsewhere in this build, visible at `meson setup` time as `Program python3 found: YES (/nix/store/...)`) and threads the resolved path into both scripts explicitly. Verified: `meson setup` prints the pinned interpreter, rebuild stays 165/165 green, and the built venv's own `python3` resolves through to that exact store path.
|
||||
- **Post-implementation, out-of-band (2026-07-22, third pass — removes `ha/data/` entirely)**: `deploy-ha.sh` no longer takes a `<data-dir>` argument at all (`--build-config` flag also removed) — every invocation now generates a fresh self-signed TLS cert/key (`openssl req -x509 -newkey rsa:2048 -nodes -days 3650`, 10y validity) directly into the deploy dir's `certs/`, and builds `cluster.bin` directly into `controller/ctrl-1/conf/` via `xceadmin`, with no intermediate staging directory. `run-ha-soak.sh` (`/xce-test-ha`) updated to match: dropped its own `XCE_HA_DATADIR`-based precondition entirely, since `deploy-ha.sh` now handles both cert and config generation itself. Updated 4 callers (`run-ha-soak.sh`, `test-heartbeat-tolerance.sh`, `test-commit-durability.sh`, plus both `xce-deploy-ha`/`xce-test-ha` SKILL.md docs) to the new 3-arg signature. `ha/data/{xce.crt,cluster.bin}` removed from git tracking (`cluster.bin` was already gitignored from the prior pass; the certs were a real checked-in TLS private key, now gone from history going forward).
|
||||
|
||||
Investigating this surfaced a real, independent finding: `xcecommon.wire.connect()` was loading a **client-side** TLS cert (`context.load_cert_chain(...)` on the `PROTOCOL_TLS_CLIENT` context) — verified against the actual reference implementation (`netio.c`'s `create_ssl_context()`) that the real client presents **no** certificate at all when `cert_file`/`key_file` are `NULL` (bare `TLS_client_method()`, no `SSL_CTX_use_certificate_chain_file`/`SSL_CTX_use_PrivateKey_file` call). `xceadmin`'s client diverged from the reference client it's meant to mirror. Fixed: `connect()` no longer loads a client cert or takes a `cert_dir` parameter at all. Since nothing else consumed `cert_dir` (no CA verification happens either — `verify_mode=CERT_NONE` throughout), removed it end-to-end: `xceadmin.discovery.follow_redirects()`'s `cert_dir` param (already dead/`noqa`'d — signature-parity leftover), `xceadmin.push.push_config()`/`probe_config()`, and the `--cert-dir` CLI flag on `config push`. Server-side cert loading (the mock TLS servers in `test_wire.py`/`test_cli.py`/`test_config_push_mock.py`) is unaffected and still needed — moved into a shared session-scoped `tls_certs` fixture in a new `tests/conftest.py` (openssl-generated, same approach as `deploy-ha.sh`), replacing 3 duplicated module-level `REPO_HA_DATA_DIR`/`pytest.mark.skipif(not ha/data/xce.crt...)` blocks. Net result: those 3 files' TLS-dependent tests no longer skip under a hermetic `nix build` sandbox (which never had `ha/data/` in its git-tracked-only source) — verified 165/165 passed, 0 skipped, both via `meson compile` and via a full `nix build --option sandbox false .#xcecore_debug` rebuild (previously 128 passed / 37 skipped there). Requirements 2.3/2.4 and 8.3/8.4/8.5 marked superseded in `requirements.md` with dated correction notes rather than silently rewritten.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue