feat(admin-config-tool): drop ha/data/ entirely, match real client TLS

deploy-ha.sh no longer takes a <data-dir> argument or --build-config
flag. Every deploy now generates a fresh self-signed TLS cert/key
(openssl, RSA 2048, 10y validity) directly into the deploy dir's
certs/, and builds cluster.bin directly into controller/ctrl-1/conf/
via xceadmin -- no intermediate staging directory, nothing preserved
across runs. run-ha-soak.sh (/xce-test-ha) updated to match: dropped
its own XCE_HA_DATADIR-based precondition entirely. Updated 4 callers
(run-ha-soak.sh, test-heartbeat-tolerance.sh, test-commit-durability.sh,
both xce-deploy-ha/xce-test-ha SKILL.md docs).

Separately: xcecommon.wire.connect() was loading a client-side TLS
cert, diverging from the real reference client -- netio.c's
create_ssl_context() presents no certificate at all when acting as a
client (cert_file/key_file both NULL, bare TLS_client_method()).
Fixed connect() to match, and since nothing else consumed cert_dir
(verify_mode=CERT_NONE throughout, no CA check either), removed it
end-to-end: discovery.follow_redirects(), push.push_config()/
probe_config(), and the --cert-dir CLI flag.

Server-side cert loading (mock TLS test servers) is unaffected --
moved into a shared session-scoped tls_certs fixture (new
tests/conftest.py, openssl-generated) replacing 3 duplicated
REPO_HA_DATA_DIR/skipif blocks. Net result: those tests no longer skip
under a hermetic nix build sandbox (which never had ha/data/) --
verified 165/165 passed, 0 skipped, via both meson compile and a full
nix build --option sandbox false .#xcecore_debug rebuild (previously
128 passed / 37 skipped there).

ha/data/{xce.crt,xce.key} removed from git tracking. Requirements
2.3/2.4 and 8.3/8.4/8.5 marked superseded in requirements.md with
dated correction notes.
This commit is contained in:
Rongsong Shen 2026-07-22 14:50:51 +00:00
parent db2b5f17f0
commit ee8473bd99
20 changed files with 244 additions and 297 deletions

View file

@ -162,3 +162,7 @@
- **5.5**: Already fully satisfied by task 2.1's `test_validation.py` — every sub-case (schema violation, reserved name, cross-type duplicate collision via `test_check_duplicates_allows_limit_name_to_collide_with_queue_name`, undefined reference) already has an explicit test with the exact same requirement list. No new tests written.
- **5.6**: `probe_config()`'s dry-run has no reachable `accepted=False` path — `ProbeConfigTagReply` genuinely carries no rejection signal on the wire (see task 3.3's note). 5.6's bullet asking for both an accepted *and* a rejected outcome cannot be satisfied without inventing undocumented client-side rejection logic; instead added `test_probe_config_has_no_wire_level_rejection_signal_by_design` (parametrized over several `configTag` values) that locks in and documents the limitation explicitly rather than leaving it silently uncovered. Added two new end-to-end propagation tests through `push_config`'s real composition with `xceadmin.discovery` (not just discovery.py's own unit tests): a real 4-mock-server redirect chain proving `TooManyRedirectsError` surfaces correctly, and a `NoPrimaryError`-on-timeout test that monkeypatches `xcecommon.wire.connect` to raise `ConnectionTimeoutError` immediately rather than waiting out a real 30s socket timeout.
- **Post-implementation, out-of-band (2026-07-22)**: reopened and reversed the 2026-07-21 "not wired into meson" Design Review correction — see `design.md`'s File Structure Plan Amendment for full rationale and verification. Summary: added `src/xceadmin/meson.build` (`custom_target()`, stamp-file output, `build_always_stale: true`) + `src/xceadmin/meson-build.sh` (out-of-tree install+test, `repo-mirror/` symlink staging so `src/configs/`, `src/schema/`, `ha/data/` relative-path test assumptions still resolve from a copied location); `src/meson.build` gained `subdir('xceadmin')`; `src/tools/xce-build.sh`'s `xceadmin)` case now delegates to `meson compile -C "$BUILD_DIR" xceadmin` instead of its own pip/venv logic. Verified both `meson compile` (165/165 passed against a real dev checkout with certs present) and `nix build --option sandbox false .#xcecore_debug` (128 passed / 37 skipped — cert-dependent tests correctly skip under nix's git-tracked-only hermetic source, since `ha/data/xce.crt`/`.key` are intentionally not committed). AppImage bundling (README 3.5) explicitly deferred, not part of this change. Net effect on Requirement 8: the editable-vs-pinned distinction (8.1 vs. 8.2) collapses — every `xce-build.sh xceadmin` run is now the pinned/meson-driven path, a strictly stronger guarantee than the old default.
- **Post-implementation, out-of-band (2026-07-22, second pass)**: `meson-build.sh`/`meson-install.sh` no longer resolve `python3` via a bare PATH lookup — `meson.build` now pins it once via `import('python').find_installation('python3')` (same mechanism as `find_program('go')`/capnp/packcc elsewhere in this build, visible at `meson setup` time as `Program python3 found: YES (/nix/store/...)`) and threads the resolved path into both scripts explicitly. Verified: `meson setup` prints the pinned interpreter, rebuild stays 165/165 green, and the built venv's own `python3` resolves through to that exact store path.
- **Post-implementation, out-of-band (2026-07-22, third pass — removes `ha/data/` entirely)**: `deploy-ha.sh` no longer takes a `<data-dir>` argument at all (`--build-config` flag also removed) — every invocation now generates a fresh self-signed TLS cert/key (`openssl req -x509 -newkey rsa:2048 -nodes -days 3650`, 10y validity) directly into the deploy dir's `certs/`, and builds `cluster.bin` directly into `controller/ctrl-1/conf/` via `xceadmin`, with no intermediate staging directory. `run-ha-soak.sh` (`/xce-test-ha`) updated to match: dropped its own `XCE_HA_DATADIR`-based precondition entirely, since `deploy-ha.sh` now handles both cert and config generation itself. Updated 4 callers (`run-ha-soak.sh`, `test-heartbeat-tolerance.sh`, `test-commit-durability.sh`, plus both `xce-deploy-ha`/`xce-test-ha` SKILL.md docs) to the new 3-arg signature. `ha/data/{xce.crt,cluster.bin}` removed from git tracking (`cluster.bin` was already gitignored from the prior pass; the certs were a real checked-in TLS private key, now gone from history going forward).
Investigating this surfaced a real, independent finding: `xcecommon.wire.connect()` was loading a **client-side** TLS cert (`context.load_cert_chain(...)` on the `PROTOCOL_TLS_CLIENT` context) — verified against the actual reference implementation (`netio.c`'s `create_ssl_context()`) that the real client presents **no** certificate at all when `cert_file`/`key_file` are `NULL` (bare `TLS_client_method()`, no `SSL_CTX_use_certificate_chain_file`/`SSL_CTX_use_PrivateKey_file` call). `xceadmin`'s client diverged from the reference client it's meant to mirror. Fixed: `connect()` no longer loads a client cert or takes a `cert_dir` parameter at all. Since nothing else consumed `cert_dir` (no CA verification happens either — `verify_mode=CERT_NONE` throughout), removed it end-to-end: `xceadmin.discovery.follow_redirects()`'s `cert_dir` param (already dead/`noqa`'d — signature-parity leftover), `xceadmin.push.push_config()`/`probe_config()`, and the `--cert-dir` CLI flag on `config push`. Server-side cert loading (the mock TLS servers in `test_wire.py`/`test_cli.py`/`test_config_push_mock.py`) is unaffected and still needed — moved into a shared session-scoped `tls_certs` fixture in a new `tests/conftest.py` (openssl-generated, same approach as `deploy-ha.sh`), replacing 3 duplicated module-level `REPO_HA_DATA_DIR`/`pytest.mark.skipif(not ha/data/xce.crt...)` blocks. Net result: those 3 files' TLS-dependent tests no longer skip under a hermetic `nix build` sandbox (which never had `ha/data/` in its git-tracked-only source) — verified 165/165 passed, 0 skipped, both via `meson compile` and via a full `nix build --option sandbox false .#xcecore_debug` rebuild (previously 128 passed / 37 skipped there). Requirements 2.3/2.4 and 8.3/8.4/8.5 marked superseded in `requirements.md` with dated correction notes rather than silently rewritten.