feat(admin-config-tool): drop ha/data/ entirely, match real client TLS
deploy-ha.sh no longer takes a <data-dir> argument or --build-config
flag. Every deploy now generates a fresh self-signed TLS cert/key
(openssl, RSA 2048, 10y validity) directly into the deploy dir's
certs/, and builds cluster.bin directly into controller/ctrl-1/conf/
via xceadmin -- no intermediate staging directory, nothing preserved
across runs. run-ha-soak.sh (/xce-test-ha) updated to match: dropped
its own XCE_HA_DATADIR-based precondition entirely. Updated 4 callers
(run-ha-soak.sh, test-heartbeat-tolerance.sh, test-commit-durability.sh,
both xce-deploy-ha/xce-test-ha SKILL.md docs).
Separately: xcecommon.wire.connect() was loading a client-side TLS
cert, diverging from the real reference client -- netio.c's
create_ssl_context() presents no certificate at all when acting as a
client (cert_file/key_file both NULL, bare TLS_client_method()).
Fixed connect() to match, and since nothing else consumed cert_dir
(verify_mode=CERT_NONE throughout, no CA check either), removed it
end-to-end: discovery.follow_redirects(), push.push_config()/
probe_config(), and the --cert-dir CLI flag.
Server-side cert loading (mock TLS test servers) is unaffected --
moved into a shared session-scoped tls_certs fixture (new
tests/conftest.py, openssl-generated) replacing 3 duplicated
REPO_HA_DATA_DIR/skipif blocks. Net result: those tests no longer skip
under a hermetic nix build sandbox (which never had ha/data/) --
verified 165/165 passed, 0 skipped, via both meson compile and a full
nix build --option sandbox false .#xcecore_debug rebuild (previously
128 passed / 37 skipped there).
ha/data/{xce.crt,xce.key} removed from git tracking. Requirements
2.3/2.4 and 8.3/8.4/8.5 marked superseded in requirements.md with
dated correction notes.
This commit is contained in:
parent
db2b5f17f0
commit
ee8473bd99
20 changed files with 244 additions and 297 deletions
|
|
@ -32,13 +32,15 @@ disown`) before relying on subagent dispatch for unattended use.
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
- Build must be current: run `/xce-build controller` first if binary is stale
|
- Build must be current: run `/xce-build controller` first if binary is stale
|
||||||
- TLS certs must exist under `ha/data/`:
|
- Run `/xce-build xceadmin` first (or `nix build .#xcecore`) so a built
|
||||||
- `ha/data/xce.crt`
|
`xceadmin` binary exists — `deploy-ha.sh` (Step 3) uses it to build
|
||||||
- `ha/data/xce.key`
|
`cluster.bin` from `src/configs/*.yaml` fresh on every deploy
|
||||||
- `ha/data/cluster.bin` — either already present, or built automatically by
|
- `openssl` must be on `PATH` (provided by `nix develop`) — `deploy-ha.sh`
|
||||||
`deploy-ha.sh` from `src/configs/*.yaml` via `xceadmin` (see Step 3). Run
|
generates a fresh self-signed TLS cert/key for every deploy
|
||||||
`/xce-build xceadmin` first (or `nix build .#xcecore`) so a built
|
|
||||||
`xceadmin` binary exists for that auto-build path to find.
|
No data directory needed: TLS cert/key and `cluster.bin` are both generated
|
||||||
|
directly into the deploy dir (`$TOP`) by `deploy-ha.sh` itself, not read from
|
||||||
|
any pre-existing `ha/data/`-style fixture.
|
||||||
|
|
||||||
## Arguments
|
## Arguments
|
||||||
|
|
||||||
|
|
@ -55,7 +57,6 @@ If no arguments are given, default to `localhost:9991 localhost:9992 localhost:9
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
BINARY="src/xce.build/controller/xce_controller"
|
BINARY="src/xce.build/controller/xce_controller"
|
||||||
DATADIR="ha/data"
|
|
||||||
TOP="/tmp/xce-deploy"
|
TOP="/tmp/xce-deploy"
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
@ -78,27 +79,25 @@ Join peers as a comma-separated string and call the deploy script:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
PEERS="localhost:9991,localhost:9992,localhost:9993" # from args
|
PEERS="localhost:9991,localhost:9992,localhost:9993" # from args
|
||||||
bash src/tools/deploy-ha.sh "$BINARY" "$DATADIR" "$TOP" "$PEERS"
|
bash src/tools/deploy-ha.sh "$BINARY" "$TOP" "$PEERS"
|
||||||
```
|
```
|
||||||
|
|
||||||
Add `--build-config` (anywhere in the args) to force a fresh `cluster.bin`
|
Every invocation is a fresh, self-contained deploy — no flag needed to force
|
||||||
rebuild from `src/configs/*.yaml` via `xceadmin` even if `$DATADIR/cluster.bin`
|
a rebuild, nothing is ever reused across deploys:
|
||||||
already exists — useful after editing the example configs:
|
- `cluster.bin` is always rebuilt from `src/configs/*.yaml` via `xceadmin`,
|
||||||
|
resolved from the meson build dir (`xce.build/xceadmin/.venv/bin/xceadmin`
|
||||||
```bash
|
by default, also checks `build.release`/`build.dbg`), a nix-built
|
||||||
bash src/tools/deploy-ha.sh --build-config "$BINARY" "$DATADIR" "$TOP" "$PEERS"
|
`result/bin/xceadmin`, or `PATH` — in that order. If none is found, the
|
||||||
```
|
script errors and tells you to run `/xce-build xceadmin` or
|
||||||
|
`nix build .#xcecore` first.
|
||||||
Without the flag, an existing `$DATADIR/cluster.bin` is left untouched; one
|
- The TLS cert/key pair is always freshly generated via `openssl` (self-signed,
|
||||||
is auto-built via `xceadmin` only if missing. `xceadmin` is resolved from
|
RSA 2048, 10y validity — long-lived since it's per-deploy anyway; loses no
|
||||||
the meson build dir (`xce.build/xceadmin/.venv/bin/xceadmin` by default,
|
real security property since TLS verification is `SSL_VERIFY_NONE`
|
||||||
also checks `build.release`/`build.dbg`), a nix-built `result/bin/xceadmin`,
|
everywhere in this codebase). Errors if `openssl` isn't on `PATH`.
|
||||||
or `PATH` — in that order. If none is found, the script errors and tells
|
|
||||||
you to run `/xce-build xceadmin` or `nix build .#xcecore` first.
|
|
||||||
|
|
||||||
This:
|
This:
|
||||||
- Creates `$TOP/certs/`, `$TOP/controller/ctrl-{1..N}/conf`, `$TOP/controller/ctrl-{1..N}/data`, `$TOP/logs/`
|
- Creates `$TOP/certs/`, `$TOP/controller/ctrl-{1..N}/conf`, `$TOP/controller/ctrl-{1..N}/data`, `$TOP/logs/`
|
||||||
- Copies `xce.crt` and `xce.key` into the shared `$TOP/certs/` dir
|
- Generates `xce.crt`/`xce.key` directly into `$TOP/certs/`
|
||||||
- Places `cluster.bin.<tag>` + `conf/current` in `controller/ctrl-1/conf/` only
|
- Places `cluster.bin.<tag>` + `conf/current` in `controller/ctrl-1/conf/` only
|
||||||
(standbys receive config from primary via `OPCode_replConfigPush`)
|
(standbys receive config from primary via `OPCode_replConfigPush`)
|
||||||
- Generates `$TOP/scripts/start.sh` (with `--workdir $TOP/controller --certdir $TOP/certs`) and `cleanup.sh`
|
- Generates `$TOP/scripts/start.sh` (with `--workdir $TOP/controller --certdir $TOP/certs`) and `cleanup.sh`
|
||||||
|
|
|
||||||
|
|
@ -361,7 +361,6 @@ XCE_HA_ROUNDS=10 XCE_HA_TOP=/tmp/my-ha-run nix develop ./src --command bash "$SK
|
||||||
| Env var | Meaning | Default |
|
| Env var | Meaning | Default |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `XCE_HA_TOP` | deploy directory | `/home/claude/tmp/xce-ha-deploy` |
|
| `XCE_HA_TOP` | deploy directory | `/home/claude/tmp/xce-ha-deploy` |
|
||||||
| `XCE_HA_DATADIR` | config/certs dir (`xce.crt`, `xce.key`; `cluster.bin` freshly built here by `xceadmin` every run, not a fixture) | `<repo>/ha/data` |
|
|
||||||
| `XCE_HA_MODE` | `sequential`, `chaotic`, `force-snapshot`, `factstream`, `array-soak`, or `array-chaos-retry` (its `--flag` wins) | `sequential` |
|
| `XCE_HA_MODE` | `sequential`, `chaotic`, `force-snapshot`, `factstream`, `array-soak`, or `array-chaos-retry` (its `--flag` wins) | `sequential` |
|
||||||
| `XCE_HA_BIN` | `xce_controller` binary (point at real-fs release for the gate) | `<repo>/src/xce.build/controller/xce_controller` |
|
| `XCE_HA_BIN` | `xce_controller` binary (point at real-fs release for the gate) | `<repo>/src/xce.build/controller/xce_controller` |
|
||||||
| `XCE_HA_XT` | `xce_test` binary | `<repo>/src/xce.build/controller/xce_test` |
|
| `XCE_HA_XT` | `xce_test` binary | `<repo>/src/xce.build/controller/xce_test` |
|
||||||
|
|
@ -385,10 +384,18 @@ XCE_HA_ROUNDS=10 XCE_HA_TOP=/tmp/my-ha-run nix develop ./src --command bash "$SK
|
||||||
|
|
||||||
## What the driver does (maps to the requested steps)
|
## What the driver does (maps to the requested steps)
|
||||||
|
|
||||||
1. **Preconditions** — verifies `xce_controller`, `xce_test`, and certs exist; builds `cluster.bin` fresh under `XCE_HA_DATADIR` via `xceadmin` (resolved from the meson build dir, a nix-installed prefix, or `PATH`, in that order — fails with a clear message if none is found; run `/xce-build xceadmin` or `nix build .#xcecore` first).
|
1. **Preconditions** — verifies `xce_controller` and `xce_test` exist. No data
|
||||||
|
directory, no cert/config precondition: `deploy-ha.sh` (next step)
|
||||||
|
generates both fresh.
|
||||||
2. **Deploy** — `src/tools/deploy-ha.sh` into `XCE_HA_TOP` with nodes
|
2. **Deploy** — `src/tools/deploy-ha.sh` into `XCE_HA_TOP` with nodes
|
||||||
`ctrl-1=localhost:9991, ctrl-2=localhost:9992, ctrl-3=localhost:9993`,
|
`ctrl-1=localhost:9991, ctrl-2=localhost:9992, ctrl-3=localhost:9993`.
|
||||||
using config/certs from `XCE_HA_DATADIR`.
|
Generates a fresh self-signed TLS cert/key via `openssl` and builds
|
||||||
|
`cluster.bin` from `src/configs/*.yaml` via `xceadmin` (resolved from the
|
||||||
|
meson build dir, a nix-installed prefix, or `PATH`, in that order — fails
|
||||||
|
with a clear message if either tool is missing; run `/xce-build xceadmin`
|
||||||
|
or `nix build .#xcecore`, and ensure `openssl` is on `PATH` via
|
||||||
|
`nix develop`) — both written directly into the deploy dir, nothing read
|
||||||
|
from or written to any pre-existing data directory.
|
||||||
3. **Start** — `scripts/start.sh ctrl-N -vvv` for all three, 0.5s staggered.
|
3. **Start** — `scripts/start.sh ctrl-N -vvv` for all three, 0.5s staggered.
|
||||||
4. **Elect** — detects the elected primary via a **client probe** (`xce_test`
|
4. **Elect** — detects the elected primary via a **client probe** (`xce_test`
|
||||||
`wait-primary` + `ctrlstatus`), reading live controller state. Primary,
|
`wait-primary` + `ctrlstatus`), reading live controller state. Primary,
|
||||||
|
|
@ -440,7 +447,8 @@ goal is "the HA soak test passes". The driver is built for that contract:
|
||||||
decides done/continue by grepping that line / checking the exit code; no
|
decides done/continue by grepping that line / checking the exit code; no
|
||||||
human-style parsing of intermediate output is needed.
|
human-style parsing of intermediate output is needed.
|
||||||
- **Fully idempotent & self-contained.** Every run `rm -rf`s `$XCE_HA_TOP`,
|
- **Fully idempotent & self-contained.** Every run `rm -rf`s `$XCE_HA_TOP`,
|
||||||
redeploys from `ha/data`, kills any stale `xce_controller`, and cleans up on
|
redeploys with a freshly generated cert and `cluster.bin` (no data
|
||||||
|
directory involved), kills any stale `xce_controller`, and cleans up on
|
||||||
exit (trap). A loop can invoke it any number of times with no manual reset.
|
exit (trap). A loop can invoke it any number of times with no manual reset.
|
||||||
- **Deterministic-enough.** Cold-start election ~15s, each round ~20-30s; a
|
- **Deterministic-enough.** Cold-start election ~15s, each round ~20-30s; a
|
||||||
5-round run is ~3-5 min, 10-round ~6-10 min. Budget the loop's per-iteration
|
5-round run is ~3-5 min, 10-round ~6-10 min. Budget the loop's per-iteration
|
||||||
|
|
|
||||||
|
|
@ -82,7 +82,6 @@
|
||||||
#
|
#
|
||||||
# Env overrides:
|
# Env overrides:
|
||||||
# XCE_HA_TOP deploy dir (default /home/claude/tmp/xce-ha-deploy)
|
# XCE_HA_TOP deploy dir (default /home/claude/tmp/xce-ha-deploy)
|
||||||
# XCE_HA_DATADIR config/certs source (default <repo>/ha/data)
|
|
||||||
# XCE_HA_MODE sequential | chaotic | force-snapshot (default sequential)
|
# XCE_HA_MODE sequential | chaotic | force-snapshot (default sequential)
|
||||||
# XCE_HA_ROUNDS kill/restart rounds (default 6, clamp 5..50)
|
# XCE_HA_ROUNDS kill/restart rounds (default 6, clamp 5..50)
|
||||||
# XCE_HA_JOBS jobs per submit batch (default 3)
|
# XCE_HA_JOBS jobs per submit batch (default 3)
|
||||||
|
|
@ -202,7 +201,6 @@ esac
|
||||||
# Sentinel written into a deploy we own; the guard before `rm -rf` only wipes a
|
# Sentinel written into a deploy we own; the guard before `rm -rf` only wipes a
|
||||||
# non-empty dir that carries this marker (see "Step 2: deploy" below).
|
# non-empty dir that carries this marker (see "Step 2: deploy" below).
|
||||||
MARKER_NAME=".xce-ha-deploy"
|
MARKER_NAME=".xce-ha-deploy"
|
||||||
DATADIR="${XCE_HA_DATADIR:-$REPO/ha/data}"
|
|
||||||
BIN="${XCE_HA_BIN:-$REPO/src/xce.build/controller/xce_controller}"
|
BIN="${XCE_HA_BIN:-$REPO/src/xce.build/controller/xce_controller}"
|
||||||
XT="${XCE_HA_XT:-$REPO/src/xce.build/controller/xce_test}"
|
XT="${XCE_HA_XT:-$REPO/src/xce.build/controller/xce_test}"
|
||||||
DEPLOY="$REPO/src/tools/deploy-ha.sh"
|
DEPLOY="$REPO/src/tools/deploy-ha.sh"
|
||||||
|
|
@ -302,29 +300,10 @@ kill_node() {
|
||||||
# ── Preconditions ────────────────────────────────────────────────────────────
|
# ── Preconditions ────────────────────────────────────────────────────────────
|
||||||
[ -x "$BIN" ] || { echo "[xce-test-ha] FAIL: controller binary missing: $BIN (run /xce-build controller)" >&2; exit 2; }
|
[ -x "$BIN" ] || { echo "[xce-test-ha] FAIL: controller binary missing: $BIN (run /xce-build controller)" >&2; exit 2; }
|
||||||
[ -x "$XT" ] || { echo "[xce-test-ha] FAIL: xce_test missing: $XT (run /xce-build controller)" >&2; exit 2; }
|
[ -x "$XT" ] || { echo "[xce-test-ha] FAIL: xce_test missing: $XT (run /xce-build controller)" >&2; exit 2; }
|
||||||
# cluster.bin is always (re)built fresh from src/configs/*.yaml via xceadmin
|
# TLS cert/key and cluster.bin are no longer preconditions here at all --
|
||||||
# -- no pre-created/checked-in cluster.bin fixture is used or kept around
|
# deploy-ha.sh (Step 2 below) generates both fresh directly into $TOP on
|
||||||
# any more (ha/data/cluster.bin was removed from git tracking accordingly).
|
# every run (openssl self-signed cert; cluster.bin from src/configs/*.yaml
|
||||||
# Same xceadmin resolution order as deploy-ha.sh's --build-config path
|
# via xceadmin). No data directory, no pre-created/checked-in fixture.
|
||||||
# (meson build dir, then a nix-installed prefix, then PATH).
|
|
||||||
XCEADMIN=""
|
|
||||||
for candidate in \
|
|
||||||
"$REPO/src/xce.build/xceadmin/.venv/bin/xceadmin" \
|
|
||||||
"$REPO/src/build.release/xceadmin/.venv/bin/xceadmin" \
|
|
||||||
"$REPO/src/build.dbg/xceadmin/.venv/bin/xceadmin" \
|
|
||||||
"$REPO/src/result/bin/xceadmin" \
|
|
||||||
"$(command -v xceadmin 2>/dev/null || true)"
|
|
||||||
do
|
|
||||||
[ -n "$candidate" ] && [ -x "$candidate" ] && { XCEADMIN="$candidate"; break; }
|
|
||||||
done
|
|
||||||
if [ -z "$XCEADMIN" ]; then
|
|
||||||
echo "[xce-test-ha] FAIL: no xceadmin binary found to build cluster.bin (run /xce-build xceadmin or nix build .#xcecore)" >&2
|
|
||||||
exit 2
|
|
||||||
fi
|
|
||||||
echo "[xce-test-ha] building cluster.bin under $DATADIR via $XCEADMIN"
|
|
||||||
"$XCEADMIN" config build "$REPO/src/configs" -o "$DATADIR/cluster.bin" \
|
|
||||||
|| { echo "[xce-test-ha] FAIL: xceadmin config build failed" >&2; exit 2; }
|
|
||||||
[ -r "$DATADIR/xce.crt" ] && [ -r "$DATADIR/xce.key" ] || { echo "[xce-test-ha] FAIL: certs missing under $DATADIR" >&2; exit 2; }
|
|
||||||
|
|
||||||
trap cleanup EXIT INT TERM
|
trap cleanup EXIT INT TERM
|
||||||
|
|
||||||
|
|
@ -1786,7 +1765,7 @@ fi
|
||||||
rm -rf "$TOP"
|
rm -rf "$TOP"
|
||||||
mkdir -p "$TOP" || fail "could not create deploy dir $TOP"
|
mkdir -p "$TOP" || fail "could not create deploy dir $TOP"
|
||||||
touch "$TOP/$MARKER_NAME" # claim ownership so a later run may safely re-wipe it
|
touch "$TOP/$MARKER_NAME" # claim ownership so a later run may safely re-wipe it
|
||||||
bash "$DEPLOY" "$BIN" "$DATADIR" "$TOP" "localhost:9991,localhost:9992,localhost:9993" \
|
bash "$DEPLOY" "$BIN" "$TOP" "localhost:9991,localhost:9992,localhost:9993" \
|
||||||
>"$TOP/deploy.log" 2>&1 || fail "deploy-ha.sh failed (see $TOP/deploy.log)"
|
>"$TOP/deploy.log" 2>&1 || fail "deploy-ha.sh failed (see $TOP/deploy.log)"
|
||||||
|
|
||||||
# ── Step 3: start all nodes ──────────────────────────────────────────────────
|
# ── Step 3: start all nodes ──────────────────────────────────────────────────
|
||||||
|
|
|
||||||
|
|
@ -83,8 +83,8 @@ Criteria 8–12 apply the **Configuration Inclusion Rules** (brief.md): a field
|
||||||
|
|
||||||
1. When the operator runs `xceadmin config push <cluster.bin>`, the system shall connect to the controller via TLS, send the binary as a capnp `Request` with `OP_upgradeConfig` opcode, `upgradeconfigrequestmsg` body (binary bytes + SHA-256 hash), framed with an 8-byte uint64 size prefix in host byte order, and display the result.
|
1. When the operator runs `xceadmin config push <cluster.bin>`, the system shall connect to the controller via TLS, send the binary as a capnp `Request` with `OP_upgradeConfig` opcode, `upgradeconfigrequestmsg` body (binary bytes + SHA-256 hash), framed with an 8-byte uint64 size prefix in host byte order, and display the result.
|
||||||
2. When the operator specifies `--controller <host:port>`, the system shall connect to the given controller address instead of the default `localhost:9991`.
|
2. When the operator specifies `--controller <host:port>`, the system shall connect to the given controller address instead of the default `localhost:9991`.
|
||||||
3. The system shall use the same TLS certificate bundle the controller cluster is deployed with (`xce.crt`/`xce.key`, default path `ha/data/`) to establish trust — no separate client CA flag or cert-issuance step. This matches the trust model `xce_test` and `deploy-ha.sh` already use for controller-to-controller TLS.
|
3. ~~The system shall use the same TLS certificate bundle the controller cluster is deployed with (`xce.crt`/`xce.key`, default path `ha/data/`) to establish trust — no separate client CA flag or cert-issuance step.~~ **Superseded (2026-07-22)**: verified against the real client TLS setup (`netio.c`'s `create_ssl_context()`) that the actual controller-to-controller/CLI client presents **no** certificate at all when acting as a client (`cert_file`/`key_file` both `NULL` → bare `TLS_client_method()`) — only the server side loads a cert/key pair. `xceadmin`'s client previously diverged from this (loaded a client cert too); fixed to match exactly. No CA flag or cert-issuance step either way — `verify_mode=CERT_NONE` on both the real client and `xceadmin`.
|
||||||
4. When the operator specifies `--cert-dir <path>`, the system shall load `xce.crt`/`xce.key` from the given directory instead of the default `ha/data/`.
|
4. ~~When the operator specifies `--cert-dir <path>`, the system shall load `xce.crt`/`xce.key` from the given directory instead of the default `ha/data/`.~~ **Superseded (2026-07-22)**: removed entirely — with no client cert to load, `--cert-dir` had no remaining purpose. `ha/data/` itself was also removed (deploy-ha.sh and the HA test harness now generate a fresh server-side cert/key per run via `openssl` instead of reading a checked-in fixture).
|
||||||
5. After receiving the controller's reply, the system shall display the result to the operator: `versionTag` (the new generation number assigned by the controller), `ackedCount` (number of standbys that acknowledged), and any `error` message.
|
5. After receiving the controller's reply, the system shall display the result to the operator: `versionTag` (the new generation number assigned by the controller), `ackedCount` (number of standbys that acknowledged), and any `error` message.
|
||||||
6. When the controller indicates the config was committed (positive `ackedCount`), the system shall exit 0.
|
6. When the controller indicates the config was committed (positive `ackedCount`), the system shall exit 0.
|
||||||
7. If the controller reports an error (non-empty `error` field or zero `ackedCount`), the system shall display the error and exit non-zero.
|
7. If the controller reports an error (non-empty `error` field or zero `ackedCount`), the system shall display the error and exit non-zero.
|
||||||
|
|
@ -153,11 +153,11 @@ Criteria 8–12 apply the **Configuration Inclusion Rules** (brief.md): a field
|
||||||
|
|
||||||
#### Acceptance Criteria
|
#### Acceptance Criteria
|
||||||
|
|
||||||
1. When the developer runs `/xce-build xceadmin`, the system shall make the `xceadmin` CLI available on PATH and run the full test suite, reporting pass/fail.
|
1. ~~When the developer runs `/xce-build xceadmin`, the system shall make the `xceadmin` CLI available on PATH and run the full test suite, reporting pass/fail.~~ **Superseded (2026-07-22)**: `xceadmin` is now meson-built (`src/xceadmin/meson.build`) — `xce-build.sh xceadmin` delegates to `meson compile -C "$BUILD_DIR" xceadmin`, and `nix build .#xcecore[_debug]` installs a standalone `result/bin/xceadmin` (no venv at runtime). See design.md's File Structure Plan Amendment.
|
||||||
2. When the developer runs `/xce-build xceadmin release`, the system shall install `xceadmin` for production use (pinned, non-editable) **and then run the full test suite against that non-editable install**, reporting pass/fail — this is the only path that exercises the packaged (`pip install .`) resolution of the bundled `controller.capnp`/JSON schemas (Requirement 5.5); an editable-install-only test run would never catch a regression there.
|
2. ~~When the developer runs `/xce-build xceadmin release`, the system shall install `xceadmin` for production use (pinned, non-editable) and then run the full test suite against that non-editable install...~~ **Superseded (2026-07-22)**: the editable-vs-pinned distinction collapsed — every meson-driven build (`xce-build.sh xceadmin`, `meson compile`, `nix build`) now runs the pinned/non-editable install path unconditionally, a strictly stronger guarantee (exercises real `package_data` resolution every run, not just under a `release` profile).
|
||||||
3. When the developer runs `/xce-deploy-ha` with the `--build-config` flag, the system shall invoke `xceadmin config build configs/ -o ha/data/cluster.bin` as a precondition before deploying the cluster.
|
3. ~~When the developer runs `/xce-deploy-ha` with the `--build-config` flag, the system shall invoke `xceadmin config build configs/ -o ha/data/cluster.bin` as a precondition before deploying the cluster.~~ **Superseded (2026-07-22)**: `--build-config` removed entirely — every `deploy-ha.sh` run now unconditionally builds `cluster.bin` fresh from `src/configs/*.yaml` directly into the deploy dir's own `controller/ctrl-1/conf/` (no `ha/data/` staging step, no flag needed since nothing is ever preserved across runs to opt out of rebuilding).
|
||||||
4. When `cluster.bin` is missing from `ha/data/` during deploy, `/xce-deploy-ha` shall run `xceadmin config build` automatically (same as `--build-config`).
|
4. ~~When `cluster.bin` is missing from `ha/data/` during deploy, `/xce-deploy-ha` shall run `xceadmin config build` automatically (same as `--build-config`).~~ **Superseded (2026-07-22)**: `ha/data/` removed entirely (directory no longer exists). See 8.3 above — always-fresh generation directly into the deploy dir replaces the missing-file check.
|
||||||
5. When `cluster.bin` already exists and `--build-config` is not passed, `/xce-deploy-ha` shall use the existing binary as-is (backward compatible).
|
5. ~~When `cluster.bin` already exists and `--build-config` is not passed, `/xce-deploy-ha` shall use the existing binary as-is (backward compatible).~~ **Superseded (2026-07-22)**: no longer applicable — nothing persists across deploys for this to preserve; every deploy is fully fresh (cert included, generated via `openssl`, self-signed, 10y validity — TLS verification is `SSL_VERIFY_NONE` everywhere in this codebase, so a fresh pair per deploy loses no real security property versus a long-lived shared one).
|
||||||
6. The nix dev shell (`nix develop ./src`) shall include all Python dependencies required by xceadmin after the nix environment is updated (see Nix Environment section in brief.md).
|
6. The nix dev shell (`nix develop ./src`) shall include all Python dependencies required by xceadmin after the nix environment is updated (see Nix Environment section in brief.md).
|
||||||
|
|
||||||
### Requirement 9: Deterministic Builds
|
### Requirement 9: Deterministic Builds
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
{
|
{
|
||||||
"feature_name": "admin-config-tool",
|
"feature_name": "admin-config-tool",
|
||||||
"created_at": "2026-07-06T01:30:00Z",
|
"created_at": "2026-07-06T01:30:00Z",
|
||||||
"updated_at": "2026-07-22T10:45:00Z",
|
"updated_at": "2026-07-22T14:50:00Z",
|
||||||
"language": "en",
|
"language": "en",
|
||||||
"phase": "implementation-complete",
|
"phase": "implementation-complete",
|
||||||
"approvals": {
|
"approvals": {
|
||||||
|
|
|
||||||
|
|
@ -162,3 +162,7 @@
|
||||||
- **5.5**: Already fully satisfied by task 2.1's `test_validation.py` — every sub-case (schema violation, reserved name, cross-type duplicate collision via `test_check_duplicates_allows_limit_name_to_collide_with_queue_name`, undefined reference) already has an explicit test with the exact same requirement list. No new tests written.
|
- **5.5**: Already fully satisfied by task 2.1's `test_validation.py` — every sub-case (schema violation, reserved name, cross-type duplicate collision via `test_check_duplicates_allows_limit_name_to_collide_with_queue_name`, undefined reference) already has an explicit test with the exact same requirement list. No new tests written.
|
||||||
- **5.6**: `probe_config()`'s dry-run has no reachable `accepted=False` path — `ProbeConfigTagReply` genuinely carries no rejection signal on the wire (see task 3.3's note). 5.6's bullet asking for both an accepted *and* a rejected outcome cannot be satisfied without inventing undocumented client-side rejection logic; instead added `test_probe_config_has_no_wire_level_rejection_signal_by_design` (parametrized over several `configTag` values) that locks in and documents the limitation explicitly rather than leaving it silently uncovered. Added two new end-to-end propagation tests through `push_config`'s real composition with `xceadmin.discovery` (not just discovery.py's own unit tests): a real 4-mock-server redirect chain proving `TooManyRedirectsError` surfaces correctly, and a `NoPrimaryError`-on-timeout test that monkeypatches `xcecommon.wire.connect` to raise `ConnectionTimeoutError` immediately rather than waiting out a real 30s socket timeout.
|
- **5.6**: `probe_config()`'s dry-run has no reachable `accepted=False` path — `ProbeConfigTagReply` genuinely carries no rejection signal on the wire (see task 3.3's note). 5.6's bullet asking for both an accepted *and* a rejected outcome cannot be satisfied without inventing undocumented client-side rejection logic; instead added `test_probe_config_has_no_wire_level_rejection_signal_by_design` (parametrized over several `configTag` values) that locks in and documents the limitation explicitly rather than leaving it silently uncovered. Added two new end-to-end propagation tests through `push_config`'s real composition with `xceadmin.discovery` (not just discovery.py's own unit tests): a real 4-mock-server redirect chain proving `TooManyRedirectsError` surfaces correctly, and a `NoPrimaryError`-on-timeout test that monkeypatches `xcecommon.wire.connect` to raise `ConnectionTimeoutError` immediately rather than waiting out a real 30s socket timeout.
|
||||||
- **Post-implementation, out-of-band (2026-07-22)**: reopened and reversed the 2026-07-21 "not wired into meson" Design Review correction — see `design.md`'s File Structure Plan Amendment for full rationale and verification. Summary: added `src/xceadmin/meson.build` (`custom_target()`, stamp-file output, `build_always_stale: true`) + `src/xceadmin/meson-build.sh` (out-of-tree install+test, `repo-mirror/` symlink staging so `src/configs/`, `src/schema/`, `ha/data/` relative-path test assumptions still resolve from a copied location); `src/meson.build` gained `subdir('xceadmin')`; `src/tools/xce-build.sh`'s `xceadmin)` case now delegates to `meson compile -C "$BUILD_DIR" xceadmin` instead of its own pip/venv logic. Verified both `meson compile` (165/165 passed against a real dev checkout with certs present) and `nix build --option sandbox false .#xcecore_debug` (128 passed / 37 skipped — cert-dependent tests correctly skip under nix's git-tracked-only hermetic source, since `ha/data/xce.crt`/`.key` are intentionally not committed). AppImage bundling (README 3.5) explicitly deferred, not part of this change. Net effect on Requirement 8: the editable-vs-pinned distinction (8.1 vs. 8.2) collapses — every `xce-build.sh xceadmin` run is now the pinned/meson-driven path, a strictly stronger guarantee than the old default.
|
- **Post-implementation, out-of-band (2026-07-22)**: reopened and reversed the 2026-07-21 "not wired into meson" Design Review correction — see `design.md`'s File Structure Plan Amendment for full rationale and verification. Summary: added `src/xceadmin/meson.build` (`custom_target()`, stamp-file output, `build_always_stale: true`) + `src/xceadmin/meson-build.sh` (out-of-tree install+test, `repo-mirror/` symlink staging so `src/configs/`, `src/schema/`, `ha/data/` relative-path test assumptions still resolve from a copied location); `src/meson.build` gained `subdir('xceadmin')`; `src/tools/xce-build.sh`'s `xceadmin)` case now delegates to `meson compile -C "$BUILD_DIR" xceadmin` instead of its own pip/venv logic. Verified both `meson compile` (165/165 passed against a real dev checkout with certs present) and `nix build --option sandbox false .#xcecore_debug` (128 passed / 37 skipped — cert-dependent tests correctly skip under nix's git-tracked-only hermetic source, since `ha/data/xce.crt`/`.key` are intentionally not committed). AppImage bundling (README 3.5) explicitly deferred, not part of this change. Net effect on Requirement 8: the editable-vs-pinned distinction (8.1 vs. 8.2) collapses — every `xce-build.sh xceadmin` run is now the pinned/meson-driven path, a strictly stronger guarantee than the old default.
|
||||||
|
- **Post-implementation, out-of-band (2026-07-22, second pass)**: `meson-build.sh`/`meson-install.sh` no longer resolve `python3` via a bare PATH lookup — `meson.build` now pins it once via `import('python').find_installation('python3')` (same mechanism as `find_program('go')`/capnp/packcc elsewhere in this build, visible at `meson setup` time as `Program python3 found: YES (/nix/store/...)`) and threads the resolved path into both scripts explicitly. Verified: `meson setup` prints the pinned interpreter, rebuild stays 165/165 green, and the built venv's own `python3` resolves through to that exact store path.
|
||||||
|
- **Post-implementation, out-of-band (2026-07-22, third pass — removes `ha/data/` entirely)**: `deploy-ha.sh` no longer takes a `<data-dir>` argument at all (`--build-config` flag also removed) — every invocation now generates a fresh self-signed TLS cert/key (`openssl req -x509 -newkey rsa:2048 -nodes -days 3650`, 10y validity) directly into the deploy dir's `certs/`, and builds `cluster.bin` directly into `controller/ctrl-1/conf/` via `xceadmin`, with no intermediate staging directory. `run-ha-soak.sh` (`/xce-test-ha`) updated to match: dropped its own `XCE_HA_DATADIR`-based precondition entirely, since `deploy-ha.sh` now handles both cert and config generation itself. Updated 4 callers (`run-ha-soak.sh`, `test-heartbeat-tolerance.sh`, `test-commit-durability.sh`, plus both `xce-deploy-ha`/`xce-test-ha` SKILL.md docs) to the new 3-arg signature. `ha/data/{xce.crt,cluster.bin}` removed from git tracking (`cluster.bin` was already gitignored from the prior pass; the certs were a real checked-in TLS private key, now gone from history going forward).
|
||||||
|
|
||||||
|
Investigating this surfaced a real, independent finding: `xcecommon.wire.connect()` was loading a **client-side** TLS cert (`context.load_cert_chain(...)` on the `PROTOCOL_TLS_CLIENT` context) — verified against the actual reference implementation (`netio.c`'s `create_ssl_context()`) that the real client presents **no** certificate at all when `cert_file`/`key_file` are `NULL` (bare `TLS_client_method()`, no `SSL_CTX_use_certificate_chain_file`/`SSL_CTX_use_PrivateKey_file` call). `xceadmin`'s client diverged from the reference client it's meant to mirror. Fixed: `connect()` no longer loads a client cert or takes a `cert_dir` parameter at all. Since nothing else consumed `cert_dir` (no CA verification happens either — `verify_mode=CERT_NONE` throughout), removed it end-to-end: `xceadmin.discovery.follow_redirects()`'s `cert_dir` param (already dead/`noqa`'d — signature-parity leftover), `xceadmin.push.push_config()`/`probe_config()`, and the `--cert-dir` CLI flag on `config push`. Server-side cert loading (the mock TLS servers in `test_wire.py`/`test_cli.py`/`test_config_push_mock.py`) is unaffected and still needed — moved into a shared session-scoped `tls_certs` fixture in a new `tests/conftest.py` (openssl-generated, same approach as `deploy-ha.sh`), replacing 3 duplicated module-level `REPO_HA_DATA_DIR`/`pytest.mark.skipif(not ha/data/xce.crt...)` blocks. Net result: those 3 files' TLS-dependent tests no longer skip under a hermetic `nix build` sandbox (which never had `ha/data/` in its git-tracked-only source) — verified 165/165 passed, 0 skipped, both via `meson compile` and via a full `nix build --option sandbox false .#xcecore_debug` rebuild (previously 128 passed / 37 skipped there). Requirements 2.3/2.4 and 8.3/8.4/8.5 marked superseded in `requirements.md` with dated correction notes rather than silently rewritten.
|
||||||
|
|
|
||||||
|
|
@ -1,24 +0,0 @@
|
||||||
-----BEGIN CERTIFICATE-----
|
|
||||||
MIID8TCCAtmgAwIBAgIUZq70Bs4uGMdEfikQHB/eSvyukH0wDQYJKoZIhvcNAQEL
|
|
||||||
BQAwgYcxCzAJBgNVBAYTAkNOMRAwDgYDVQQIDAdCZWlqaW5nMRAwDgYDVQQHDAdC
|
|
||||||
ZWlqaW5nMQ0wCwYDVQQKDARORVhBMQ0wCwYDVQQLDAR0ZWNoMRYwFAYDVQQDDA1S
|
|
||||||
b25nc29uZyBTaGVuMR4wHAYJKoZIhvcNAQkBFg9yc2hlbkBzaGVucnMuZXUwHhcN
|
|
||||||
MjUxMjEwMDMzOTUwWhcNMzUxMjA4MDMzOTUwWjCBhzELMAkGA1UEBhMCQ04xEDAO
|
|
||||||
BgNVBAgMB0JlaWppbmcxEDAOBgNVBAcMB0JlaWppbmcxDTALBgNVBAoMBE5FWEEx
|
|
||||||
DTALBgNVBAsMBHRlY2gxFjAUBgNVBAMMDVJvbmdzb25nIFNoZW4xHjAcBgkqhkiG
|
|
||||||
9w0BCQEWD3JzaGVuQHNoZW5ycy5ldTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC
|
|
||||||
AQoCggEBAOgLF21maOzpGxy6YoO1pKJnOb3ybinzLsln2aOSejSTVYhZYlV9/I1a
|
|
||||||
JWiHRV0D8yc3YnXiuui1ZNhrh9z71DdiXkhNA4cUJ2WEcAA7/rhx104rorSgfWIN
|
|
||||||
PsJtpr7Z3GWTLD4emBuhzPwkX25CLkY3nIWO/9yb3yHOi+mteXWrhTIWd/U1Gm34
|
|
||||||
9a8dSP0Dcg3DGE4sXnaBrs15yHpx2kw5uYU8z3/fAd6ghiXJattP7uP9E3iJk3OC
|
|
||||||
fUFhLEBsy9O6FtKjoFW6U5/b/3XK36+7n6a+V0NvRA2yVTAfIIIAIl+UF63qbJig
|
|
||||||
QJkYR+A/IblyRNn4YlxHxbm1ztJ3bDsCAwEAAaNTMFEwHQYDVR0OBBYEFANJkzql
|
|
||||||
6kaBCwFaKOC8n7Nsu1R9MB8GA1UdIwQYMBaAFANJkzql6kaBCwFaKOC8n7Nsu1R9
|
|
||||||
MA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAKFQgG6L8zcrVzx/
|
|
||||||
8H9ID9b0d7wGSkGDg5Of86MWOr9qVcGUO1CiBCYQSIdrspXGztASMnHm1KpnePH/
|
|
||||||
aQPlnQ71ozMxJhWVmRZ7jNjG5fPUM5JWSXDlhKvwgPeF2qkggukJ8M00yQPp/ena
|
|
||||||
BuRkPi2pbI/Nq9YISRF/LzQXsDwoY/0MXFjsauQfXTclD8lwvw/3zPihamPlU8DR
|
|
||||||
cD6deRQtGbh9QA/nEdZHwJHi6A2+kuADezw2qUE7Ddp93BJr6PWT29VuGFT7ZIvU
|
|
||||||
TTrIrxlO68DxjqC05mf1QK6iXVmu6Pbt6KLZ1dBy4I3FU3fg0bkfmImSURrklrf6
|
|
||||||
HhH9RKM=
|
|
||||||
-----END CERTIFICATE-----
|
|
||||||
|
|
@ -1,28 +0,0 @@
|
||||||
-----BEGIN PRIVATE KEY-----
|
|
||||||
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDoCxdtZmjs6Rsc
|
|
||||||
umKDtaSiZzm98m4p8y7JZ9mjkno0k1WIWWJVffyNWiVoh0VdA/MnN2J14rrotWTY
|
|
||||||
a4fc+9Q3Yl5ITQOHFCdlhHAAO/64cddOK6K0oH1iDT7Cbaa+2dxlkyw+Hpgbocz8
|
|
||||||
JF9uQi5GN5yFjv/cm98hzovprXl1q4UyFnf1NRpt+PWvHUj9A3INwxhOLF52ga7N
|
|
||||||
ech6cdpMObmFPM9/3wHeoIYlyWrbT+7j/RN4iZNzgn1BYSxAbMvTuhbSo6BVulOf
|
|
||||||
2/91yt+vu5+mvldDb0QNslUwHyCCACJflBet6myYoECZGEfgPyG5ckTZ+GJcR8W5
|
|
||||||
tc7Sd2w7AgMBAAECggEACI3gHzhXReSSC00TtKdNmm8TwdzeEs3UO7iIXW9Y8fc8
|
|
||||||
f/pLlbs/RDa+ikvyUquCmz9/cbkaCtolDZVpPMxL1n7Z2vB74M7tZOCtAFU13rbT
|
|
||||||
qR4SsXX5KVQzrBc+vmpppZT3XJLkKKCeXe05V2c5LX1/V60d4ZQddbGXltW+VIgc
|
|
||||||
ODifQz3UywgsswvrgtfKQW51v+/mZeqMh68aLcPpK+tYsJUx0TJstqgnTxwpY58w
|
|
||||||
FoxbLbM/ligxWEZ3nPqGT5YKGhnQYF8Dc+l/MgD66LCkNffwRHgZWuhNvSv3rY3d
|
|
||||||
o59aWnTm2oH4ot9Kl5E4x1l4tabIL+zVkIlaSr3KAQKBgQD2vFYWOvo7cEKkzlDp
|
|
||||||
9qCjXV212gJOrd5N8zVTKO80uLWsOynE/gRScXhM6+IJZYlsUJfmybQXIs2MjAO5
|
|
||||||
3SLDMwYr2sIXajUb3fa+uz1sk+gusPQP594tZ0ixlgY0ZtFT7P/0cTHor7F+wWTV
|
|
||||||
X+kKHSyy7EzSZHepM4nQpuscawKBgQDwwYekSPLQfS+QHOsKY0f9Lp/2HswlNQqJ
|
|
||||||
TgPWSTxm3aIUgIzaXBtUnByST5NoQ7N3y7gQfdXF6ikgIQpiw7LMN/oYum7V09mU
|
|
||||||
sUPdLdGXYc6u/PrIlgSakeR8B0R85goEiqHqQDC/T2ovhy3IPPcde9JHbOBZAXP6
|
|
||||||
5ToajrfjcQKBgQCDSnpgf+eRY04DFknWBUc7Rr5LuwMJTNmDDwxsANcwba02RZEX
|
|
||||||
Yrd4Cyshe08fAumaOlRqGSB8QyevRv3gk/wgdHEvOsuW3AwAlR7sJQQtV5Lv7Xch
|
|
||||||
U5dU/6Wd4Z/RmHiAuRcFwon1MDA5q7z1EYEWwal8V+MU/SLzZRpLbdajmwKBgEFC
|
|
||||||
5Kbm01MqrArvm9LsCKabzeakgIgRcbK4gBfFMGdd6sKN+iBy2UG9M42Bnp6jw3FU
|
|
||||||
uqdVxO3qRI2eBzewfrYW8mdjmgnCbHd++btp4Sec7q7GiqB28faIV5w7SBhnLw3+
|
|
||||||
GlfqYIxrVYhep01hIUdPujd8TvAxkaPQIUjzqOsxAoGANB2tCcafv9E+noyrtP7B
|
|
||||||
VB+azsvzRSUUc8udN0InPyQhfRiOZNxrDbRdv/wzOp3zqT59KQKDNr9Zvn0wXQGx
|
|
||||||
N6bwtG6iLZV761B5bG70SzwU0eMVwYuzizpY5RXr8+pTAOcd01Zwx9nR4a9SmH0Z
|
|
||||||
gAkwNNUqj3im4/Tf9izgvgM=
|
|
||||||
-----END PRIVATE KEY-----
|
|
||||||
|
|
@ -3,57 +3,46 @@ set -e
|
||||||
|
|
||||||
SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
||||||
|
|
||||||
# --build-config may appear anywhere among the positional args.
|
BINARY=$1
|
||||||
BUILD_CONFIG=0
|
TOP=$2
|
||||||
ARGS=()
|
PEERS=$3
|
||||||
for arg in "$@"; do
|
|
||||||
if [ "$arg" = "--build-config" ]; then
|
|
||||||
BUILD_CONFIG=1
|
|
||||||
else
|
|
||||||
ARGS+=("$arg")
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
|
|
||||||
BINARY=${ARGS[0]}
|
if [ -z "$BINARY" ] || [ -z "$TOP" ] || [ -z "$PEERS" ]; then
|
||||||
DATADIR=${ARGS[1]}
|
echo "Usage: $0 <binary> <top-dir> <peers>"
|
||||||
TOP=${ARGS[2]}
|
|
||||||
PEERS=${ARGS[3]}
|
|
||||||
|
|
||||||
if [ -z "$BINARY" ] || [ -z "$DATADIR" ] || [ -z "$TOP" ] || [ -z "$PEERS" ]; then
|
|
||||||
echo "Usage: $0 [--build-config] <binary> <data-dir> <top-dir> <peers>"
|
|
||||||
echo " --build-config: rebuild <data-dir>/cluster.bin from configs/ before deploying,"
|
|
||||||
echo " even if one already exists"
|
|
||||||
echo " peers: comma-separated host:port list (e.g. host1:port1,host2:port2)"
|
echo " peers: comma-separated host:port list (e.g. host1:port1,host2:port2)"
|
||||||
|
echo " TLS cert/key and cluster.bin are generated fresh into <top-dir> every"
|
||||||
|
echo " run (openssl self-signed cert; cluster.bin via xceadmin from"
|
||||||
|
echo " src/configs/*.yaml) -- no pre-existing data directory needed."
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
BINARY=$(realpath "$BINARY")
|
BINARY=$(realpath "$BINARY")
|
||||||
|
|
||||||
# Requirements 8.3/8.4/8.5: rebuild cluster.bin when --build-config is
|
# xceadmin is meson-built now (src/xceadmin/meson.build), not a plain
|
||||||
# passed or none exists yet; otherwise leave an existing one untouched.
|
# `pip install -e .` venv checked out under src/xceadmin/ -- its build
|
||||||
if [ "$BUILD_CONFIG" = "1" ] || [ ! -f "$DATADIR/cluster.bin" ]; then
|
# output lives under whichever meson build dir xce-build.sh's profile
|
||||||
# xceadmin is meson-built now (src/xceadmin/meson.build), not a plain
|
# picked, or under a nix-installed prefix's bin/. Try each in turn.
|
||||||
# `pip install -e .` venv checked out under src/xceadmin/ -- its build
|
XCEADMIN=""
|
||||||
# output lives under whichever meson build dir xce-build.sh's profile
|
for candidate in \
|
||||||
# picked, or under a nix-installed prefix's bin/. Try each in turn.
|
"$SCRIPT_DIR/../xce.build/xceadmin/.venv/bin/xceadmin" \
|
||||||
XCEADMIN=""
|
"$SCRIPT_DIR/../build.release/xceadmin/.venv/bin/xceadmin" \
|
||||||
for candidate in \
|
"$SCRIPT_DIR/../build.dbg/xceadmin/.venv/bin/xceadmin" \
|
||||||
"$SCRIPT_DIR/../xce.build/xceadmin/.venv/bin/xceadmin" \
|
"$SCRIPT_DIR/../result/bin/xceadmin" \
|
||||||
"$SCRIPT_DIR/../build.release/xceadmin/.venv/bin/xceadmin" \
|
"$(command -v xceadmin 2>/dev/null || true)"
|
||||||
"$SCRIPT_DIR/../build.dbg/xceadmin/.venv/bin/xceadmin" \
|
do
|
||||||
"$SCRIPT_DIR/../result/bin/xceadmin" \
|
if [ -n "$candidate" ] && [ -x "$candidate" ]; then
|
||||||
"$(command -v xceadmin 2>/dev/null || true)"
|
XCEADMIN="$candidate"
|
||||||
do
|
break
|
||||||
if [ -n "$candidate" ] && [ -x "$candidate" ]; then
|
|
||||||
XCEADMIN="$candidate"
|
|
||||||
break
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
if [ -z "$XCEADMIN" ]; then
|
|
||||||
echo "Error: xceadmin binary not found -- run 'xce-build.sh xceadmin' or 'nix build .#xcecore' first"
|
|
||||||
exit 1
|
|
||||||
fi
|
fi
|
||||||
"$XCEADMIN" config build "$SCRIPT_DIR/../configs" -o "$DATADIR/cluster.bin"
|
done
|
||||||
|
if [ -z "$XCEADMIN" ]; then
|
||||||
|
echo "Error: xceadmin binary not found -- run 'xce-build.sh xceadmin' or 'nix build .#xcecore' first"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ! command -v openssl >/dev/null 2>&1; then
|
||||||
|
echo "Error: openssl not found -- run under 'nix develop' (provides openssl on PATH)"
|
||||||
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
IFS=',' read -ra PEER_ARRAY <<< "$PEERS"
|
IFS=',' read -ra PEER_ARRAY <<< "$PEERS"
|
||||||
|
|
@ -68,13 +57,22 @@ for i in $(seq 1 $N); do
|
||||||
mkdir -p "$TOP/controller/ctrl-$i/data"
|
mkdir -p "$TOP/controller/ctrl-$i/data"
|
||||||
done
|
done
|
||||||
|
|
||||||
# Copy certificates
|
# Self-signed dev/test TLS cert+key, generated fresh directly into the
|
||||||
cp "$DATADIR/xce.crt" "$TOP/certs/"
|
# deployment dir every run -- nothing checked into git, nothing shared
|
||||||
cp "$DATADIR/xce.key" "$TOP/certs/"
|
# across deploys. Long-lived (10y) since it's regenerated per deploy
|
||||||
|
# anyway; TLS verification is SSL_VERIFY_NONE everywhere in this codebase
|
||||||
|
# (no hostname/identity check), so a fresh self-signed pair per run loses
|
||||||
|
# no real security property versus a long-lived shared one.
|
||||||
|
openssl req -x509 -newkey rsa:2048 -nodes -days 3650 \
|
||||||
|
-keyout "$TOP/certs/xce.key" -out "$TOP/certs/xce.crt" \
|
||||||
|
-subj "/CN=xce-dev-test" 2>/dev/null
|
||||||
|
|
||||||
# Tag and copy cluster.bin
|
# Build cluster.bin from src/configs/*.yaml via xceadmin, straight into
|
||||||
TAG=$(sha256sum "$DATADIR/cluster.bin" | cut -c1-12)
|
# this deploy's own ctrl-1/conf/ (no intermediate data-dir copy).
|
||||||
cp "$DATADIR/cluster.bin" "$TOP/controller/ctrl-1/conf/cluster.bin.$TAG"
|
CLUSTER_BIN_TMP="$TOP/.cluster.bin.tmp"
|
||||||
|
"$XCEADMIN" config build "$SCRIPT_DIR/../configs" -o "$CLUSTER_BIN_TMP"
|
||||||
|
TAG=$(sha256sum "$CLUSTER_BIN_TMP" | cut -c1-12)
|
||||||
|
mv "$CLUSTER_BIN_TMP" "$TOP/controller/ctrl-1/conf/cluster.bin.$TAG"
|
||||||
echo "cluster.bin.$TAG" > "$TOP/controller/ctrl-1/conf/current"
|
echo "cluster.bin.$TAG" > "$TOP/controller/ctrl-1/conf/current"
|
||||||
|
|
||||||
# Build peers string: ctrl-1=host1:port1,ctrl-2=host2:port2,...
|
# Build peers string: ctrl-1=host1:port1,ctrl-2=host2:port2,...
|
||||||
|
|
|
||||||
|
|
@ -95,7 +95,6 @@ fi
|
||||||
WORKTREE=$(cd "$(dirname "$0")/../.." && pwd)
|
WORKTREE=$(cd "$(dirname "$0")/../.." && pwd)
|
||||||
BINARY="$WORKTREE/src/xce.build/controller/xce_controller"
|
BINARY="$WORKTREE/src/xce.build/controller/xce_controller"
|
||||||
XCE_TEST="$WORKTREE/src/xce.build/controller/xce_test"
|
XCE_TEST="$WORKTREE/src/xce.build/controller/xce_test"
|
||||||
DATADIR="$WORKTREE/ha/data"
|
|
||||||
DEPLOY_DIR="${1:-/tmp/xce-ha-deploy}"
|
DEPLOY_DIR="${1:-/tmp/xce-ha-deploy}"
|
||||||
ROUNDS="${2:-8}"
|
ROUNDS="${2:-8}"
|
||||||
PEERS="localhost:9991,localhost:9992,localhost:9993"
|
PEERS="localhost:9991,localhost:9992,localhost:9993"
|
||||||
|
|
@ -151,7 +150,7 @@ deploy_cluster() {
|
||||||
exit 2
|
exit 2
|
||||||
fi
|
fi
|
||||||
yellow "Deploying 3-node cluster to $DEPLOY_DIR ..."
|
yellow "Deploying 3-node cluster to $DEPLOY_DIR ..."
|
||||||
bash "$WORKTREE/src/tools/deploy-ha.sh" "$BINARY" "$DATADIR" "$DEPLOY_DIR" "$PEERS"
|
bash "$WORKTREE/src/tools/deploy-ha.sh" "$BINARY" "$DEPLOY_DIR" "$PEERS"
|
||||||
}
|
}
|
||||||
|
|
||||||
start_node() {
|
start_node() {
|
||||||
|
|
|
||||||
|
|
@ -17,7 +17,6 @@ set -e
|
||||||
|
|
||||||
WORKTREE=$(cd "$(dirname "$0")/../.." && pwd)
|
WORKTREE=$(cd "$(dirname "$0")/../.." && pwd)
|
||||||
BINARY="$WORKTREE/src/xce.build/controller/xce_controller"
|
BINARY="$WORKTREE/src/xce.build/controller/xce_controller"
|
||||||
DATADIR="$WORKTREE/ha/data"
|
|
||||||
DEPLOY_DIR="${1:-/tmp/xce-deploy}"
|
DEPLOY_DIR="${1:-/tmp/xce-deploy}"
|
||||||
PEERS="localhost:9991,localhost:9992,localhost:9993"
|
PEERS="localhost:9991,localhost:9992,localhost:9993"
|
||||||
LOGDIR="$DEPLOY_DIR/logs"
|
LOGDIR="$DEPLOY_DIR/logs"
|
||||||
|
|
@ -48,7 +47,7 @@ deploy_cluster() {
|
||||||
fi
|
fi
|
||||||
yellow "Deploying 3-node cluster to $DEPLOY_DIR ..."
|
yellow "Deploying 3-node cluster to $DEPLOY_DIR ..."
|
||||||
bash "$WORKTREE/src/tools/deploy-ha.sh" \
|
bash "$WORKTREE/src/tools/deploy-ha.sh" \
|
||||||
"$BINARY" "$DATADIR" "$DEPLOY_DIR" "$PEERS"
|
"$BINARY" "$DEPLOY_DIR" "$PEERS"
|
||||||
}
|
}
|
||||||
|
|
||||||
start_cluster() {
|
start_cluster() {
|
||||||
|
|
|
||||||
|
|
@ -28,7 +28,6 @@ from xceadmin.push import probe_config, push_config
|
||||||
|
|
||||||
_DEFAULT_OUTPUT = "cluster.bin"
|
_DEFAULT_OUTPUT = "cluster.bin"
|
||||||
_DEFAULT_CONTROLLER = "localhost:9991"
|
_DEFAULT_CONTROLLER = "localhost:9991"
|
||||||
_DEFAULT_CERT_DIR = "ha/data"
|
|
||||||
|
|
||||||
|
|
||||||
@click.group()
|
@click.group()
|
||||||
|
|
@ -59,15 +58,12 @@ def config_build(configs_dir: str, output: str) -> None:
|
||||||
@click.option(
|
@click.option(
|
||||||
"--controller", default=_DEFAULT_CONTROLLER, help="host:port of any controller node (2.2)."
|
"--controller", default=_DEFAULT_CONTROLLER, help="host:port of any controller node (2.2)."
|
||||||
)
|
)
|
||||||
@click.option(
|
|
||||||
"--cert-dir", default=_DEFAULT_CERT_DIR, help="Directory containing xce.crt/xce.key (2.4)."
|
|
||||||
)
|
|
||||||
@click.option("--dry-run", is_flag=True, help="Probe acceptance without committing (Requirement 6).")
|
@click.option("--dry-run", is_flag=True, help="Probe acceptance without committing (Requirement 6).")
|
||||||
def config_push(cluster_bin_path: str, controller: str, cert_dir: str, dry_run: bool) -> None:
|
def config_push(cluster_bin_path: str, controller: str, dry_run: bool) -> None:
|
||||||
"""Push CLUSTER_BIN_PATH to the controller cluster."""
|
"""Push CLUSTER_BIN_PATH to the controller cluster."""
|
||||||
try:
|
try:
|
||||||
if dry_run:
|
if dry_run:
|
||||||
result = probe_config(cluster_bin_path, controller, cert_dir)
|
result = probe_config(cluster_bin_path, controller)
|
||||||
if not result.accepted:
|
if not result.accepted:
|
||||||
raise click.ClickException(result.reason) # 6.3
|
raise click.ClickException(result.reason) # 6.3
|
||||||
click.echo(
|
click.echo(
|
||||||
|
|
@ -75,7 +71,7 @@ def config_push(cluster_bin_path: str, controller: str, cert_dir: str, dry_run:
|
||||||
f"(generation {result.current_generation}→{result.would_be_generation})"
|
f"(generation {result.current_generation}→{result.would_be_generation})"
|
||||||
) # 6.2
|
) # 6.2
|
||||||
else:
|
else:
|
||||||
result = push_config(cluster_bin_path, controller, cert_dir)
|
result = push_config(cluster_bin_path, controller)
|
||||||
click.echo(f"versionTag={result.version_tag} ackedCount={result.acked_count}") # 2.5
|
click.echo(f"versionTag={result.version_tag} ackedCount={result.acked_count}") # 2.5
|
||||||
if result.error or result.acked_count <= 0:
|
if result.error or result.acked_count <= 0:
|
||||||
raise click.ClickException(result.error or "push rejected (ackedCount == 0)") # 2.7
|
raise click.ClickException(result.error or "push rejected (ackedCount == 0)") # 2.7
|
||||||
|
|
|
||||||
|
|
@ -42,8 +42,6 @@ def _redirect_target(reply_bytes: bytes) -> tuple[str, int] | None:
|
||||||
def follow_redirects(
|
def follow_redirects(
|
||||||
initial_host: str,
|
initial_host: str,
|
||||||
initial_port: int,
|
initial_port: int,
|
||||||
cert_dir: str, # noqa: ARG001 (signature parity with design.md; all
|
|
||||||
# I/O, including TLS setup, is delegated to send_fn's own closure)
|
|
||||||
send_fn,
|
send_fn,
|
||||||
) -> tuple[str, int, bytes]:
|
) -> tuple[str, int, bytes]:
|
||||||
"""Send via ``send_fn(host, port) -> reply_bytes``, following up to 3
|
"""Send via ``send_fn(host, port) -> reply_bytes``, following up to 3
|
||||||
|
|
|
||||||
|
|
@ -79,9 +79,9 @@ def _split_controller(controller: str) -> tuple[str, int]:
|
||||||
return controller, _DEFAULT_PORT
|
return controller, _DEFAULT_PORT
|
||||||
|
|
||||||
|
|
||||||
def _send_via(request_bytes: bytes, cert_dir: str):
|
def _send_via(request_bytes: bytes):
|
||||||
def _send(host: str, port: int) -> bytes:
|
def _send(host: str, port: int) -> bytes:
|
||||||
sock = wire.connect(host, port, cert_dir)
|
sock = wire.connect(host, port)
|
||||||
try:
|
try:
|
||||||
return wire.send_request(sock, request_bytes)
|
return wire.send_request(sock, request_bytes)
|
||||||
finally:
|
finally:
|
||||||
|
|
@ -90,7 +90,7 @@ def _send_via(request_bytes: bytes, cert_dir: str):
|
||||||
return _send
|
return _send
|
||||||
|
|
||||||
|
|
||||||
def push_config(cluster_bin_path: str, controller: str, cert_dir: str) -> PushResult:
|
def push_config(cluster_bin_path: str, controller: str) -> PushResult:
|
||||||
"""Push ``cluster_bin_path`` to the controller cluster (2.1). Follows
|
"""Push ``cluster_bin_path`` to the controller cluster (2.1). Follows
|
||||||
redirects to the primary (3.1-3.4) before sending.
|
redirects to the primary (3.1-3.4) before sending.
|
||||||
|
|
||||||
|
|
@ -111,7 +111,7 @@ def push_config(cluster_bin_path: str, controller: str, cert_dir: str) -> PushRe
|
||||||
request_bytes = request.to_bytes()
|
request_bytes = request.to_bytes()
|
||||||
|
|
||||||
host, port = _split_controller(controller)
|
host, port = _split_controller(controller)
|
||||||
_, _, reply_bytes = follow_redirects(host, port, cert_dir, _send_via(request_bytes, cert_dir))
|
_, _, reply_bytes = follow_redirects(host, port, _send_via(request_bytes))
|
||||||
|
|
||||||
with mod.RequestReply.from_bytes(reply_bytes) as reply:
|
with mod.RequestReply.from_bytes(reply_bytes) as reply:
|
||||||
if reply.status != 0:
|
if reply.status != 0:
|
||||||
|
|
@ -129,7 +129,7 @@ def push_config(cluster_bin_path: str, controller: str, cert_dir: str) -> PushRe
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def probe_config(cluster_bin_path: str, controller: str, cert_dir: str) -> DryRunResult:
|
def probe_config(cluster_bin_path: str, controller: str) -> DryRunResult:
|
||||||
"""Probe the controller's current config generation without pushing
|
"""Probe the controller's current config generation without pushing
|
||||||
anything (6.1). ``cluster_bin_path`` is read only to confirm it
|
anything (6.1). ``cluster_bin_path`` is read only to confirm it
|
||||||
exists and is readable before spending a network round-trip -- its
|
exists and is readable before spending a network round-trip -- its
|
||||||
|
|
@ -149,7 +149,7 @@ def probe_config(cluster_bin_path: str, controller: str, cert_dir: str) -> DryRu
|
||||||
request_bytes = request.to_bytes()
|
request_bytes = request.to_bytes()
|
||||||
|
|
||||||
host, port = _split_controller(controller)
|
host, port = _split_controller(controller)
|
||||||
reply_bytes = _send_via(request_bytes, cert_dir)(host, port)
|
reply_bytes = _send_via(request_bytes)(host, port)
|
||||||
|
|
||||||
with mod.ProbeConfigTagReply.from_bytes(reply_bytes) as reply:
|
with mod.ProbeConfigTagReply.from_bytes(reply_bytes) as reply:
|
||||||
current_generation = reply.configTag
|
current_generation = reply.configTag
|
||||||
|
|
|
||||||
|
|
@ -12,9 +12,14 @@ Wire compatibility (verified against the real C implementation, not
|
||||||
assumed):
|
assumed):
|
||||||
|
|
||||||
- TLS posture mirrors the controller's own outbound client context exactly
|
- TLS posture mirrors the controller's own outbound client context exactly
|
||||||
(``myapp.c:103-109``): ``SSL_CTX_new(TLS_client_method())`` +
|
(``netio.c``'s ``create_ssl_context()``: when acting as a client, cert/key
|
||||||
``SSL_CTX_set_verify(..., SSL_VERIFY_NONE, NULL)`` -- no CN/peer-cert
|
are both ``NULL`` -- ``TLS_client_method()`` with no
|
||||||
validation anywhere in this trust model to diverge from.
|
``SSL_CTX_use_certificate_chain_file``/``SSL_CTX_use_PrivateKey_file``
|
||||||
|
call at all, i.e. the client presents **no** certificate) plus
|
||||||
|
``SSL_VERIFY_NONE``/``check_hostname=False`` -- no CN/peer-cert
|
||||||
|
validation anywhere in this trust model to diverge from, and no client
|
||||||
|
cert to load. Only the server side (controller, and this module's own
|
||||||
|
test mock servers) loads a cert/key pair.
|
||||||
- Framing matches ``request.c``'s ``xce_replconfig_req_ssl``
|
- Framing matches ``request.c``'s ``xce_replconfig_req_ssl``
|
||||||
(``src/lib/request.c:925-934``): an 8-byte ``uint64_t`` length header
|
(``src/lib/request.c:925-934``): an 8-byte ``uint64_t`` length header
|
||||||
written via a raw ``SSL_write(ssl, &hdr, sizeof(hdr))`` -- host/native
|
written via a raw ``SSL_write(ssl, &hdr, sizeof(hdr))`` -- host/native
|
||||||
|
|
@ -46,9 +51,6 @@ _HEADER = struct.Struct("=Q")
|
||||||
#: claiming to be larger than this is corrupt/hostile, not merely large.
|
#: claiming to be larger than this is corrupt/hostile, not merely large.
|
||||||
_MAX_FRAME_BYTES = 64 * 1024 * 1024
|
_MAX_FRAME_BYTES = 64 * 1024 * 1024
|
||||||
|
|
||||||
_CERT_FILENAME = "xce.crt"
|
|
||||||
_KEY_FILENAME = "xce.key"
|
|
||||||
|
|
||||||
#: Timeout values below this are almost certainly a test deliberately
|
#: Timeout values below this are almost certainly a test deliberately
|
||||||
#: forcing a fast timeout, not a production caller -- kept as a constant
|
#: forcing a fast timeout, not a production caller -- kept as a constant
|
||||||
#: only so ``connect``'s default is named, not to gate behavior.
|
#: only so ``connect``'s default is named, not to gate behavior.
|
||||||
|
|
@ -63,16 +65,14 @@ def _peername_or(sock: socket.socket, default: tuple[str, int]) -> tuple[str, in
|
||||||
return default
|
return default
|
||||||
|
|
||||||
|
|
||||||
def connect(
|
def connect(host: str, port: int, timeout_s: float = _DEFAULT_TIMEOUT_S) -> ssl.SSLSocket:
|
||||||
host: str, port: int, cert_dir: str, timeout_s: float = _DEFAULT_TIMEOUT_S
|
|
||||||
) -> ssl.SSLSocket:
|
|
||||||
"""Open a TLS connection to a controller node.
|
"""Open a TLS connection to a controller node.
|
||||||
|
|
||||||
Loads ``xce.crt``/``xce.key`` from ``cert_dir`` (2.3, 2.4, 7.1, 7.6) and
|
Establishes trust exactly as the controller's own outbound TLS does --
|
||||||
establishes trust exactly as the controller's own outbound TLS does --
|
no client certificate (matching ``netio.c``'s ``create_ssl_context()``
|
||||||
``check_hostname=False``, ``verify_mode=ssl.CERT_NONE`` (no separate
|
with ``cert_file``/``key_file`` both ``NULL``), ``check_hostname=False``,
|
||||||
client CA flag or cert-issuance step, matching ``xce_test`` and
|
``verify_mode=ssl.CERT_NONE`` (no CA flag or cert-issuance step,
|
||||||
``deploy-ha.sh``'s existing trust model).
|
matching ``xce_test`` and ``deploy-ha.sh``'s existing trust model).
|
||||||
|
|
||||||
Raises :class:`ConnectionTimeoutError` if the TCP connect or the TLS
|
Raises :class:`ConnectionTimeoutError` if the TCP connect or the TLS
|
||||||
handshake does not complete within ``timeout_s``. A refused connection
|
handshake does not complete within ``timeout_s``. A refused connection
|
||||||
|
|
@ -83,9 +83,6 @@ def connect(
|
||||||
context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
|
context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
|
||||||
context.check_hostname = False
|
context.check_hostname = False
|
||||||
context.verify_mode = ssl.CERT_NONE
|
context.verify_mode = ssl.CERT_NONE
|
||||||
context.load_cert_chain(
|
|
||||||
certfile=f"{cert_dir}/{_CERT_FILENAME}", keyfile=f"{cert_dir}/{_KEY_FILENAME}"
|
|
||||||
)
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
raw_sock = socket.create_connection((host, port), timeout=timeout_s)
|
raw_sock = socket.create_connection((host, port), timeout=timeout_s)
|
||||||
|
|
|
||||||
44
src/xceadmin/tests/conftest.py
Normal file
44
src/xceadmin/tests/conftest.py
Normal file
|
|
@ -0,0 +1,44 @@
|
||||||
|
"""Shared pytest fixtures for xceadmin's test suite."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class TlsCertPair:
|
||||||
|
cert_file: Path
|
||||||
|
key_file: Path
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(scope="session")
|
||||||
|
def tls_certs(tmp_path_factory) -> TlsCertPair:
|
||||||
|
"""A fresh self-signed TLS cert/key pair, generated once per test
|
||||||
|
session via openssl -- not read from any checked-in fixture (this
|
||||||
|
repo carries none; ``ha/data/`` was removed). Matches deploy-ha.sh's
|
||||||
|
own cert generation: TLS verification is ``SSL_VERIFY_NONE``
|
||||||
|
everywhere in this codebase (no hostname/identity check anywhere),
|
||||||
|
so a fresh self-signed pair is exactly as valid a test double as a
|
||||||
|
long-lived shared one.
|
||||||
|
"""
|
||||||
|
if shutil.which("openssl") is None:
|
||||||
|
pytest.skip("openssl not found on PATH -- required to generate a test TLS cert")
|
||||||
|
cert_dir = tmp_path_factory.mktemp("xceadmin-tls")
|
||||||
|
cert_file = cert_dir / "xce.crt"
|
||||||
|
key_file = cert_dir / "xce.key"
|
||||||
|
subprocess.run(
|
||||||
|
[
|
||||||
|
"openssl", "req", "-x509", "-newkey", "rsa:2048", "-nodes",
|
||||||
|
"-days", "3650",
|
||||||
|
"-keyout", str(key_file), "-out", str(cert_file),
|
||||||
|
"-subj", "/CN=xceadmin-test",
|
||||||
|
],
|
||||||
|
check=True,
|
||||||
|
capture_output=True,
|
||||||
|
)
|
||||||
|
return TlsCertPair(cert_file=cert_file, key_file=key_file)
|
||||||
|
|
@ -19,11 +19,6 @@ from click.testing import CliRunner
|
||||||
from xcecommon.schema import load_schema
|
from xcecommon.schema import load_schema
|
||||||
from xceadmin.cli import cli
|
from xceadmin.cli import cli
|
||||||
|
|
||||||
REPO_HA_DATA_DIR = Path(__file__).resolve().parents[3] / "ha" / "data"
|
|
||||||
CERT_FILE = REPO_HA_DATA_DIR / "xce.crt"
|
|
||||||
KEY_FILE = REPO_HA_DATA_DIR / "xce.key"
|
|
||||||
CERT_DIR = str(REPO_HA_DATA_DIR)
|
|
||||||
|
|
||||||
|
|
||||||
def test_cli_help_exits_zero() -> None:
|
def test_cli_help_exits_zero() -> None:
|
||||||
"""`xceadmin --help` must run cleanly and print usage text."""
|
"""`xceadmin --help` must run cleanly and print usage text."""
|
||||||
|
|
@ -138,15 +133,17 @@ def test_config_build_missing_configs_dir_does_not_raw_traceback(tmp_path):
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
def _server_context() -> ssl.SSLContext:
|
def _server_context(cert_file: Path, key_file: Path) -> ssl.SSLContext:
|
||||||
context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
|
context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
|
||||||
context.load_cert_chain(certfile=str(CERT_FILE), keyfile=str(KEY_FILE))
|
context.load_cert_chain(certfile=str(cert_file), keyfile=str(key_file))
|
||||||
return context
|
return context
|
||||||
|
|
||||||
|
|
||||||
class _MockController:
|
class _MockController:
|
||||||
def __init__(self, handler):
|
def __init__(self, handler, cert_file: Path, key_file: Path):
|
||||||
self._handler = handler
|
self._handler = handler
|
||||||
|
self._cert_file = cert_file
|
||||||
|
self._key_file = key_file
|
||||||
self._listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
self._listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||||
self._listener.bind(("127.0.0.1", 0))
|
self._listener.bind(("127.0.0.1", 0))
|
||||||
self._listener.listen(1)
|
self._listener.listen(1)
|
||||||
|
|
@ -160,7 +157,9 @@ class _MockController:
|
||||||
except OSError:
|
except OSError:
|
||||||
return
|
return
|
||||||
try:
|
try:
|
||||||
tls_conn = _server_context().wrap_socket(raw_conn, server_side=True)
|
tls_conn = _server_context(self._cert_file, self._key_file).wrap_socket(
|
||||||
|
raw_conn, server_side=True
|
||||||
|
)
|
||||||
except (ssl.SSLError, OSError):
|
except (ssl.SSLError, OSError):
|
||||||
raw_conn.close()
|
raw_conn.close()
|
||||||
return
|
return
|
||||||
|
|
@ -176,18 +175,12 @@ class _MockController:
|
||||||
self._thread.join(timeout=5)
|
self._thread.join(timeout=5)
|
||||||
|
|
||||||
|
|
||||||
pytestmark = pytest.mark.skipif(
|
|
||||||
not (CERT_FILE.is_file() and KEY_FILE.is_file()),
|
|
||||||
reason="ha/data/xce.crt or ha/data/xce.key not present",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def mock_controller():
|
def mock_controller(tls_certs):
|
||||||
servers: list[_MockController] = []
|
servers: list[_MockController] = []
|
||||||
|
|
||||||
def _make(handler) -> _MockController:
|
def _make(handler) -> _MockController:
|
||||||
srv = _MockController(handler)
|
srv = _MockController(handler, tls_certs.cert_file, tls_certs.key_file)
|
||||||
servers.append(srv)
|
servers.append(srv)
|
||||||
return srv
|
return srv
|
||||||
|
|
||||||
|
|
@ -223,7 +216,7 @@ def test_config_push_success_exits_zero(mock_controller, cluster_bin):
|
||||||
runner = CliRunner()
|
runner = CliRunner()
|
||||||
result = runner.invoke(
|
result = runner.invoke(
|
||||||
cli,
|
cli,
|
||||||
["config", "push", cluster_bin, "--controller", f"127.0.0.1:{srv.port}", "--cert-dir", CERT_DIR],
|
["config", "push", cluster_bin, "--controller", f"127.0.0.1:{srv.port}"],
|
||||||
)
|
)
|
||||||
|
|
||||||
assert result.exit_code == 0, result.output
|
assert result.exit_code == 0, result.output
|
||||||
|
|
@ -251,7 +244,7 @@ def test_config_push_zero_acked_count_exits_nonzero(mock_controller, cluster_bin
|
||||||
runner = CliRunner()
|
runner = CliRunner()
|
||||||
result = runner.invoke(
|
result = runner.invoke(
|
||||||
cli,
|
cli,
|
||||||
["config", "push", cluster_bin, "--controller", f"127.0.0.1:{srv.port}", "--cert-dir", CERT_DIR],
|
["config", "push", cluster_bin, "--controller", f"127.0.0.1:{srv.port}"],
|
||||||
)
|
)
|
||||||
|
|
||||||
assert result.exit_code != 0
|
assert result.exit_code != 0
|
||||||
|
|
@ -267,7 +260,7 @@ def test_config_push_connection_refused_exits_nonzero_not_raw_traceback(cluster_
|
||||||
runner = CliRunner()
|
runner = CliRunner()
|
||||||
result = runner.invoke(
|
result = runner.invoke(
|
||||||
cli,
|
cli,
|
||||||
["config", "push", cluster_bin, "--controller", f"127.0.0.1:{port}", "--cert-dir", CERT_DIR],
|
["config", "push", cluster_bin, "--controller", f"127.0.0.1:{port}"],
|
||||||
)
|
)
|
||||||
|
|
||||||
assert result.exit_code != 0
|
assert result.exit_code != 0
|
||||||
|
|
@ -297,8 +290,6 @@ def test_config_push_dry_run_reports_generations(mock_controller, cluster_bin):
|
||||||
cluster_bin,
|
cluster_bin,
|
||||||
"--controller",
|
"--controller",
|
||||||
f"127.0.0.1:{srv.port}",
|
f"127.0.0.1:{srv.port}",
|
||||||
"--cert-dir",
|
|
||||||
CERT_DIR,
|
|
||||||
],
|
],
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -26,20 +26,10 @@ from xcecommon.errors import ConnectionTimeoutError, NoPrimaryError, ReplyDecode
|
||||||
from xcecommon.schema import load_schema
|
from xcecommon.schema import load_schema
|
||||||
from xceadmin import push
|
from xceadmin import push
|
||||||
|
|
||||||
REPO_HA_DATA_DIR = Path(__file__).resolve().parents[3] / "ha" / "data"
|
|
||||||
CERT_FILE = REPO_HA_DATA_DIR / "xce.crt"
|
|
||||||
KEY_FILE = REPO_HA_DATA_DIR / "xce.key"
|
|
||||||
CERT_DIR = str(REPO_HA_DATA_DIR)
|
|
||||||
|
|
||||||
pytestmark = pytest.mark.skipif(
|
def _server_context(cert_file: Path, key_file: Path) -> ssl.SSLContext:
|
||||||
not (CERT_FILE.is_file() and KEY_FILE.is_file()),
|
|
||||||
reason="ha/data/xce.crt or ha/data/xce.key not present",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _server_context() -> ssl.SSLContext:
|
|
||||||
context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
|
context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
|
||||||
context.load_cert_chain(certfile=str(CERT_FILE), keyfile=str(KEY_FILE))
|
context.load_cert_chain(certfile=str(cert_file), keyfile=str(key_file))
|
||||||
return context
|
return context
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -48,8 +38,10 @@ class _MockController:
|
||||||
accepted TLS socket to ``handler``, then closes.
|
accepted TLS socket to ``handler``, then closes.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
def __init__(self, handler):
|
def __init__(self, handler, cert_file: Path, key_file: Path):
|
||||||
self._handler = handler
|
self._handler = handler
|
||||||
|
self._cert_file = cert_file
|
||||||
|
self._key_file = key_file
|
||||||
self._listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
self._listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||||
self._listener.bind(("127.0.0.1", 0))
|
self._listener.bind(("127.0.0.1", 0))
|
||||||
self._listener.listen(1)
|
self._listener.listen(1)
|
||||||
|
|
@ -63,7 +55,9 @@ class _MockController:
|
||||||
except OSError:
|
except OSError:
|
||||||
return
|
return
|
||||||
try:
|
try:
|
||||||
tls_conn = _server_context().wrap_socket(raw_conn, server_side=True)
|
tls_conn = _server_context(self._cert_file, self._key_file).wrap_socket(
|
||||||
|
raw_conn, server_side=True
|
||||||
|
)
|
||||||
except (ssl.SSLError, OSError):
|
except (ssl.SSLError, OSError):
|
||||||
raw_conn.close()
|
raw_conn.close()
|
||||||
return
|
return
|
||||||
|
|
@ -80,11 +74,11 @@ class _MockController:
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def mock_controller():
|
def mock_controller(tls_certs):
|
||||||
servers: list[_MockController] = []
|
servers: list[_MockController] = []
|
||||||
|
|
||||||
def _make(handler) -> _MockController:
|
def _make(handler) -> _MockController:
|
||||||
srv = _MockController(handler)
|
srv = _MockController(handler, tls_certs.cert_file, tls_certs.key_file)
|
||||||
servers.append(srv)
|
servers.append(srv)
|
||||||
return srv
|
return srv
|
||||||
|
|
||||||
|
|
@ -127,7 +121,7 @@ def test_push_config_successful_commit(mock_controller, cluster_bin):
|
||||||
|
|
||||||
srv = mock_controller(_handler)
|
srv = mock_controller(_handler)
|
||||||
|
|
||||||
result = push.push_config(cluster_bin, f"127.0.0.1:{srv.port}", CERT_DIR)
|
result = push.push_config(cluster_bin, f"127.0.0.1:{srv.port}")
|
||||||
|
|
||||||
assert result.version_tag == 7
|
assert result.version_tag == 7
|
||||||
assert result.acked_count == 3
|
assert result.acked_count == 3
|
||||||
|
|
@ -166,7 +160,7 @@ def test_push_config_follows_one_redirect_to_primary(mock_controller, cluster_bi
|
||||||
_standby_handler.primary_port = primary_srv.port
|
_standby_handler.primary_port = primary_srv.port
|
||||||
standby_srv = mock_controller(_standby_handler)
|
standby_srv = mock_controller(_standby_handler)
|
||||||
|
|
||||||
result = push.push_config(cluster_bin, f"127.0.0.1:{standby_srv.port}", CERT_DIR)
|
result = push.push_config(cluster_bin, f"127.0.0.1:{standby_srv.port}")
|
||||||
|
|
||||||
assert result.version_tag == 9
|
assert result.version_tag == 9
|
||||||
assert result.acked_count == 2
|
assert result.acked_count == 2
|
||||||
|
|
@ -184,7 +178,7 @@ def test_push_config_non_zero_status_reports_error_not_exception(mock_controller
|
||||||
|
|
||||||
srv = mock_controller(_handler)
|
srv = mock_controller(_handler)
|
||||||
|
|
||||||
result = push.push_config(cluster_bin, f"127.0.0.1:{srv.port}", CERT_DIR)
|
result = push.push_config(cluster_bin, f"127.0.0.1:{srv.port}")
|
||||||
|
|
||||||
assert result.acked_count == 0
|
assert result.acked_count == 0
|
||||||
assert "status 1" in result.error
|
assert "status 1" in result.error
|
||||||
|
|
@ -203,7 +197,7 @@ def test_push_config_wrong_reply_variant_raises_reply_decode_error(mock_controll
|
||||||
srv = mock_controller(_handler)
|
srv = mock_controller(_handler)
|
||||||
|
|
||||||
with pytest.raises(ReplyDecodeError):
|
with pytest.raises(ReplyDecodeError):
|
||||||
push.push_config(cluster_bin, f"127.0.0.1:{srv.port}", CERT_DIR)
|
push.push_config(cluster_bin, f"127.0.0.1:{srv.port}")
|
||||||
|
|
||||||
|
|
||||||
def test_push_config_default_port_used_when_omitted():
|
def test_push_config_default_port_used_when_omitted():
|
||||||
|
|
@ -239,7 +233,7 @@ def test_probe_config_reports_current_and_would_be_generation(mock_controller, c
|
||||||
|
|
||||||
srv = mock_controller(_handler)
|
srv = mock_controller(_handler)
|
||||||
|
|
||||||
result = push.probe_config(cluster_bin, f"127.0.0.1:{srv.port}", CERT_DIR)
|
result = push.probe_config(cluster_bin, f"127.0.0.1:{srv.port}")
|
||||||
|
|
||||||
assert result.current_generation == 41
|
assert result.current_generation == 41
|
||||||
assert result.would_be_generation == 42
|
assert result.would_be_generation == 42
|
||||||
|
|
@ -263,7 +257,7 @@ def test_probe_config_zero_generation_still_accepted(mock_controller, cluster_bi
|
||||||
|
|
||||||
srv = mock_controller(_handler)
|
srv = mock_controller(_handler)
|
||||||
|
|
||||||
result = push.probe_config(cluster_bin, f"127.0.0.1:{srv.port}", CERT_DIR)
|
result = push.probe_config(cluster_bin, f"127.0.0.1:{srv.port}")
|
||||||
|
|
||||||
assert result.current_generation == 0
|
assert result.current_generation == 0
|
||||||
assert result.would_be_generation == 1
|
assert result.would_be_generation == 1
|
||||||
|
|
@ -296,7 +290,7 @@ def test_probe_config_has_no_wire_level_rejection_signal_by_design(
|
||||||
|
|
||||||
srv = mock_controller(_handler)
|
srv = mock_controller(_handler)
|
||||||
|
|
||||||
result = push.probe_config(cluster_bin, f"127.0.0.1:{srv.port}", CERT_DIR)
|
result = push.probe_config(cluster_bin, f"127.0.0.1:{srv.port}")
|
||||||
|
|
||||||
assert result.accepted is True
|
assert result.accepted is True
|
||||||
assert result.reason == ""
|
assert result.reason == ""
|
||||||
|
|
@ -337,7 +331,7 @@ def test_push_config_propagates_too_many_redirects_end_to_end(mock_controller, c
|
||||||
servers["e"] = mock_controller(_make_redirect_handler(lambda: 1))
|
servers["e"] = mock_controller(_make_redirect_handler(lambda: 1))
|
||||||
|
|
||||||
with pytest.raises(TooManyRedirectsError):
|
with pytest.raises(TooManyRedirectsError):
|
||||||
push.push_config(cluster_bin, f"127.0.0.1:{servers['a'].port}", CERT_DIR)
|
push.push_config(cluster_bin, f"127.0.0.1:{servers['a'].port}")
|
||||||
|
|
||||||
|
|
||||||
def test_push_config_propagates_no_primary_error_on_connection_timeout(cluster_bin, monkeypatch):
|
def test_push_config_propagates_no_primary_error_on_connection_timeout(cluster_bin, monkeypatch):
|
||||||
|
|
@ -349,10 +343,10 @@ def test_push_config_propagates_no_primary_error_on_connection_timeout(cluster_b
|
||||||
eventually raise on a genuine timeout.
|
eventually raise on a genuine timeout.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
def _fake_connect(host, port, cert_dir, timeout_s=30.0):
|
def _fake_connect(host, port, timeout_s=30.0):
|
||||||
raise ConnectionTimeoutError(f"timeout connecting to {host}:{port}", host, port)
|
raise ConnectionTimeoutError(f"timeout connecting to {host}:{port}", host, port)
|
||||||
|
|
||||||
monkeypatch.setattr(wire, "connect", _fake_connect)
|
monkeypatch.setattr(wire, "connect", _fake_connect)
|
||||||
|
|
||||||
with pytest.raises(NoPrimaryError):
|
with pytest.raises(NoPrimaryError):
|
||||||
push.push_config(cluster_bin, "127.0.0.1:9991", CERT_DIR)
|
push.push_config(cluster_bin, "127.0.0.1:9991")
|
||||||
|
|
|
||||||
|
|
@ -30,7 +30,7 @@ def test_follow_redirects_no_redirect_returns_first_reply_unchanged():
|
||||||
return reply
|
return reply
|
||||||
|
|
||||||
final_host, final_port, reply_bytes = discovery.follow_redirects(
|
final_host, final_port, reply_bytes = discovery.follow_redirects(
|
||||||
"node1", 9991, "ha/data", send_fn
|
"node1", 9991, send_fn
|
||||||
)
|
)
|
||||||
|
|
||||||
assert (final_host, final_port) == ("node1", 9991)
|
assert (final_host, final_port) == ("node1", 9991)
|
||||||
|
|
@ -49,7 +49,7 @@ def test_follow_redirects_follows_one_redirect(capsys):
|
||||||
return responses[(host, port)]
|
return responses[(host, port)]
|
||||||
|
|
||||||
final_host, final_port, reply_bytes = discovery.follow_redirects(
|
final_host, final_port, reply_bytes = discovery.follow_redirects(
|
||||||
"node1", 9991, "ha/data", send_fn
|
"node1", 9991, send_fn
|
||||||
)
|
)
|
||||||
|
|
||||||
assert (final_host, final_port) == ("node2", 9991)
|
assert (final_host, final_port) == ("node2", 9991)
|
||||||
|
|
@ -73,7 +73,7 @@ def test_follow_redirects_follows_exactly_three_hops(capsys):
|
||||||
calls.append((host, port))
|
calls.append((host, port))
|
||||||
return replies[(host, port)]
|
return replies[(host, port)]
|
||||||
|
|
||||||
final_host, final_port, _ = discovery.follow_redirects("a", 1, "ha/data", send_fn)
|
final_host, final_port, _ = discovery.follow_redirects("a", 1, send_fn)
|
||||||
|
|
||||||
assert (final_host, final_port) == ("d", 4)
|
assert (final_host, final_port) == ("d", 4)
|
||||||
assert calls == [("a", 1), ("b", 2), ("c", 3), ("d", 4)]
|
assert calls == [("a", 1), ("b", 2), ("c", 3), ("d", 4)]
|
||||||
|
|
@ -92,7 +92,7 @@ def test_follow_redirects_raises_too_many_redirects_on_fourth_hop():
|
||||||
return replies[(host, port)]
|
return replies[(host, port)]
|
||||||
|
|
||||||
with pytest.raises(TooManyRedirectsError) as exc_info:
|
with pytest.raises(TooManyRedirectsError) as exc_info:
|
||||||
discovery.follow_redirects("a", 1, "ha/data", send_fn)
|
discovery.follow_redirects("a", 1, send_fn)
|
||||||
|
|
||||||
assert exc_info.value.hops == 4
|
assert exc_info.value.hops == 4
|
||||||
|
|
||||||
|
|
@ -102,7 +102,7 @@ def test_follow_redirects_raises_no_primary_error_on_connection_timeout():
|
||||||
raise ConnectionTimeoutError(f"timeout connecting to {host}:{port}", host, port)
|
raise ConnectionTimeoutError(f"timeout connecting to {host}:{port}", host, port)
|
||||||
|
|
||||||
with pytest.raises(NoPrimaryError) as exc_info:
|
with pytest.raises(NoPrimaryError) as exc_info:
|
||||||
discovery.follow_redirects("node1", 9991, "ha/data", send_fn)
|
discovery.follow_redirects("node1", 9991, send_fn)
|
||||||
|
|
||||||
assert exc_info.value.host == "node1"
|
assert exc_info.value.host == "node1"
|
||||||
assert exc_info.value.port == 9991
|
assert exc_info.value.port == 9991
|
||||||
|
|
@ -117,7 +117,7 @@ def test_follow_redirects_no_primary_error_names_last_hop_not_initial_hop():
|
||||||
return replies[(host, port)]
|
return replies[(host, port)]
|
||||||
|
|
||||||
with pytest.raises(NoPrimaryError) as exc_info:
|
with pytest.raises(NoPrimaryError) as exc_info:
|
||||||
discovery.follow_redirects("a", 1, "ha/data", send_fn)
|
discovery.follow_redirects("a", 1, send_fn)
|
||||||
|
|
||||||
assert exc_info.value.host == "b"
|
assert exc_info.value.host == "b"
|
||||||
assert exc_info.value.port == 2
|
assert exc_info.value.port == 2
|
||||||
|
|
@ -133,7 +133,7 @@ def test_follow_redirects_zero_port_is_not_treated_as_a_redirect():
|
||||||
return reply
|
return reply
|
||||||
|
|
||||||
final_host, final_port, reply_bytes = discovery.follow_redirects(
|
final_host, final_port, reply_bytes = discovery.follow_redirects(
|
||||||
"node1", 9991, "ha/data", send_fn
|
"node1", 9991, send_fn
|
||||||
)
|
)
|
||||||
|
|
||||||
assert (final_host, final_port) == ("node1", 9991)
|
assert (final_host, final_port) == ("node1", 9991)
|
||||||
|
|
|
||||||
|
|
@ -1,11 +1,12 @@
|
||||||
"""Unit tests for ``xcecommon.wire`` (design.md, ``xcecommon.wire``;
|
"""Unit tests for ``xcecommon.wire`` (design.md, ``xcecommon.wire``;
|
||||||
Requirements 2.1, 2.3, 2.4, 5.2, 6.1, 7.1, 7.3, 7.6).
|
Requirements 2.1, 2.3, 2.4, 5.2, 6.1, 7.1, 7.3, 7.6).
|
||||||
|
|
||||||
Uses a real local TLS loopback server (threading) wrapped with this
|
Uses a real local TLS loopback server (threading) wrapped with a
|
||||||
repo's existing ``ha/data/xce.crt``/``xce.key`` -- the same cert pair
|
self-signed cert/key generated fresh per test session (see conftest.py's
|
||||||
``deploy-ha.sh`` already copies for controller-to-controller TLS -- so
|
``tls_certs`` fixture, same generation approach ``deploy-ha.sh`` uses) --
|
||||||
these tests exercise the real trust model (no CA, ``SSL_VERIFY_NONE``),
|
so these tests exercise the real trust model (no CA, ``SSL_VERIFY_NONE``,
|
||||||
not a synthetic substitute.
|
no client certificate -- matching ``netio.c``'s client-side
|
||||||
|
``create_ssl_context(NULL, NULL)``), not a synthetic substitute.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
@ -21,19 +22,10 @@ import pytest
|
||||||
from xcecommon import wire
|
from xcecommon import wire
|
||||||
from xcecommon.errors import ConnectionTimeoutError, ReplyDecodeError
|
from xcecommon.errors import ConnectionTimeoutError, ReplyDecodeError
|
||||||
|
|
||||||
REPO_HA_DATA_DIR = Path(__file__).resolve().parents[3] / "ha" / "data"
|
|
||||||
CERT_FILE = REPO_HA_DATA_DIR / "xce.crt"
|
|
||||||
KEY_FILE = REPO_HA_DATA_DIR / "xce.key"
|
|
||||||
|
|
||||||
pytestmark = pytest.mark.skipif(
|
def _server_context(cert_file: Path, key_file: Path) -> ssl.SSLContext:
|
||||||
not (CERT_FILE.is_file() and KEY_FILE.is_file()),
|
|
||||||
reason="ha/data/xce.crt or ha/data/xce.key not present",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _server_context() -> ssl.SSLContext:
|
|
||||||
context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
|
context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
|
||||||
context.load_cert_chain(certfile=str(CERT_FILE), keyfile=str(KEY_FILE))
|
context.load_cert_chain(certfile=str(cert_file), keyfile=str(key_file))
|
||||||
return context
|
return context
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -43,8 +35,10 @@ class _TlsEchoServer:
|
||||||
receives the wrapped :class:`ssl.SSLSocket`.
|
receives the wrapped :class:`ssl.SSLSocket`.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
def __init__(self, handler):
|
def __init__(self, handler, cert_file: Path, key_file: Path):
|
||||||
self._handler = handler
|
self._handler = handler
|
||||||
|
self._cert_file = cert_file
|
||||||
|
self._key_file = key_file
|
||||||
self._listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
self._listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||||
self._listener.bind(("127.0.0.1", 0))
|
self._listener.bind(("127.0.0.1", 0))
|
||||||
self._listener.listen(1)
|
self._listener.listen(1)
|
||||||
|
|
@ -58,7 +52,9 @@ class _TlsEchoServer:
|
||||||
except OSError:
|
except OSError:
|
||||||
return
|
return
|
||||||
try:
|
try:
|
||||||
tls_conn = _server_context().wrap_socket(raw_conn, server_side=True)
|
tls_conn = _server_context(self._cert_file, self._key_file).wrap_socket(
|
||||||
|
raw_conn, server_side=True
|
||||||
|
)
|
||||||
except (ssl.SSLError, OSError):
|
except (ssl.SSLError, OSError):
|
||||||
raw_conn.close()
|
raw_conn.close()
|
||||||
return
|
return
|
||||||
|
|
@ -75,7 +71,7 @@ class _TlsEchoServer:
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def echo_server():
|
def echo_server(tls_certs):
|
||||||
"""A server that reads exactly one frame and writes it straight back."""
|
"""A server that reads exactly one frame and writes it straight back."""
|
||||||
servers: list[_TlsEchoServer] = []
|
servers: list[_TlsEchoServer] = []
|
||||||
|
|
||||||
|
|
@ -91,7 +87,7 @@ def echo_server():
|
||||||
wire.send_frame(conn, payload)
|
wire.send_frame(conn, payload)
|
||||||
|
|
||||||
def _make() -> _TlsEchoServer:
|
def _make() -> _TlsEchoServer:
|
||||||
srv = _TlsEchoServer(_handler)
|
srv = _TlsEchoServer(_handler, tls_certs.cert_file, tls_certs.key_file)
|
||||||
servers.append(srv)
|
servers.append(srv)
|
||||||
return srv
|
return srv
|
||||||
|
|
||||||
|
|
@ -100,9 +96,6 @@ def echo_server():
|
||||||
srv.close()
|
srv.close()
|
||||||
|
|
||||||
|
|
||||||
CERT_DIR = str(REPO_HA_DATA_DIR)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# connect()
|
# connect()
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
@ -110,7 +103,7 @@ CERT_DIR = str(REPO_HA_DATA_DIR)
|
||||||
|
|
||||||
def test_connect_establishes_real_tls_session(echo_server):
|
def test_connect_establishes_real_tls_session(echo_server):
|
||||||
srv = echo_server()
|
srv = echo_server()
|
||||||
sock = wire.connect("127.0.0.1", srv.port, CERT_DIR, timeout_s=5.0)
|
sock = wire.connect("127.0.0.1", srv.port, timeout_s=5.0)
|
||||||
try:
|
try:
|
||||||
assert isinstance(sock, ssl.SSLSocket)
|
assert isinstance(sock, ssl.SSLSocket)
|
||||||
assert sock.getpeername()[1] == srv.port
|
assert sock.getpeername()[1] == srv.port
|
||||||
|
|
@ -125,7 +118,7 @@ def test_connect_refused_propagates_unwrapped():
|
||||||
listener.close() # nothing listens on this port now
|
listener.close() # nothing listens on this port now
|
||||||
|
|
||||||
with pytest.raises(ConnectionRefusedError):
|
with pytest.raises(ConnectionRefusedError):
|
||||||
wire.connect("127.0.0.1", port, CERT_DIR, timeout_s=5.0)
|
wire.connect("127.0.0.1", port, timeout_s=5.0)
|
||||||
|
|
||||||
|
|
||||||
def test_connect_handshake_timeout_raises_connection_timeout_error_and_closes_cleanly():
|
def test_connect_handshake_timeout_raises_connection_timeout_error_and_closes_cleanly():
|
||||||
|
|
@ -151,7 +144,7 @@ def test_connect_handshake_timeout_raises_connection_timeout_error_and_closes_cl
|
||||||
thread.start()
|
thread.start()
|
||||||
try:
|
try:
|
||||||
with pytest.raises(ConnectionTimeoutError) as exc_info:
|
with pytest.raises(ConnectionTimeoutError) as exc_info:
|
||||||
wire.connect("127.0.0.1", port, CERT_DIR, timeout_s=0.3)
|
wire.connect("127.0.0.1", port, timeout_s=0.3)
|
||||||
assert exc_info.value.host == "127.0.0.1"
|
assert exc_info.value.host == "127.0.0.1"
|
||||||
assert exc_info.value.port == port
|
assert exc_info.value.port == port
|
||||||
|
|
||||||
|
|
@ -175,7 +168,7 @@ def test_connect_handshake_timeout_raises_connection_timeout_error_and_closes_cl
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
def test_send_frame_writes_exact_known_good_byte_sequence(echo_server):
|
def test_send_frame_writes_exact_known_good_byte_sequence(tls_certs):
|
||||||
"""Requirement: framing matches the controller's own byte-for-byte,
|
"""Requirement: framing matches the controller's own byte-for-byte,
|
||||||
verified against a known-good byte sequence (not just a round-trip).
|
verified against a known-good byte sequence (not just a round-trip).
|
||||||
"""
|
"""
|
||||||
|
|
@ -191,9 +184,9 @@ def test_send_frame_writes_exact_known_good_byte_sequence(echo_server):
|
||||||
break
|
break
|
||||||
captured.extend(chunk)
|
captured.extend(chunk)
|
||||||
|
|
||||||
srv = _TlsEchoServer(_capture_handler)
|
srv = _TlsEchoServer(_capture_handler, tls_certs.cert_file, tls_certs.key_file)
|
||||||
try:
|
try:
|
||||||
sock = wire.connect("127.0.0.1", srv.port, CERT_DIR, timeout_s=5.0)
|
sock = wire.connect("127.0.0.1", srv.port, timeout_s=5.0)
|
||||||
try:
|
try:
|
||||||
wire.send_frame(sock, b"xce!")
|
wire.send_frame(sock, b"xce!")
|
||||||
finally:
|
finally:
|
||||||
|
|
@ -209,7 +202,7 @@ def test_send_frame_writes_exact_known_good_byte_sequence(echo_server):
|
||||||
|
|
||||||
def test_recv_frame_round_trips_arbitrary_payload(echo_server):
|
def test_recv_frame_round_trips_arbitrary_payload(echo_server):
|
||||||
srv = echo_server()
|
srv = echo_server()
|
||||||
sock = wire.connect("127.0.0.1", srv.port, CERT_DIR, timeout_s=5.0)
|
sock = wire.connect("127.0.0.1", srv.port, timeout_s=5.0)
|
||||||
try:
|
try:
|
||||||
payload = b"\x00\x01\xffcapnp-shaped-bytes" * 100
|
payload = b"\x00\x01\xffcapnp-shaped-bytes" * 100
|
||||||
wire.send_frame(sock, payload)
|
wire.send_frame(sock, payload)
|
||||||
|
|
@ -220,7 +213,7 @@ def test_recv_frame_round_trips_arbitrary_payload(echo_server):
|
||||||
|
|
||||||
def test_recv_frame_round_trips_empty_payload(echo_server):
|
def test_recv_frame_round_trips_empty_payload(echo_server):
|
||||||
srv = echo_server()
|
srv = echo_server()
|
||||||
sock = wire.connect("127.0.0.1", srv.port, CERT_DIR, timeout_s=5.0)
|
sock = wire.connect("127.0.0.1", srv.port, timeout_s=5.0)
|
||||||
try:
|
try:
|
||||||
wire.send_frame(sock, b"")
|
wire.send_frame(sock, b"")
|
||||||
assert wire.recv_frame(sock) == b""
|
assert wire.recv_frame(sock) == b""
|
||||||
|
|
@ -228,13 +221,13 @@ def test_recv_frame_round_trips_empty_payload(echo_server):
|
||||||
sock.close()
|
sock.close()
|
||||||
|
|
||||||
|
|
||||||
def test_recv_frame_raises_reply_decode_error_on_truncated_header():
|
def test_recv_frame_raises_reply_decode_error_on_truncated_header(tls_certs):
|
||||||
def _close_after_partial_header(conn: ssl.SSLSocket) -> None:
|
def _close_after_partial_header(conn: ssl.SSLSocket) -> None:
|
||||||
conn.sendall(b"\x01\x02\x03") # 3 of 8 header bytes, then hang up
|
conn.sendall(b"\x01\x02\x03") # 3 of 8 header bytes, then hang up
|
||||||
|
|
||||||
srv = _TlsEchoServer(_close_after_partial_header)
|
srv = _TlsEchoServer(_close_after_partial_header, tls_certs.cert_file, tls_certs.key_file)
|
||||||
try:
|
try:
|
||||||
sock = wire.connect("127.0.0.1", srv.port, CERT_DIR, timeout_s=5.0)
|
sock = wire.connect("127.0.0.1", srv.port, timeout_s=5.0)
|
||||||
try:
|
try:
|
||||||
with pytest.raises(ReplyDecodeError):
|
with pytest.raises(ReplyDecodeError):
|
||||||
wire.recv_frame(sock)
|
wire.recv_frame(sock)
|
||||||
|
|
@ -244,14 +237,14 @@ def test_recv_frame_raises_reply_decode_error_on_truncated_header():
|
||||||
srv.close()
|
srv.close()
|
||||||
|
|
||||||
|
|
||||||
def test_recv_frame_raises_reply_decode_error_on_truncated_payload():
|
def test_recv_frame_raises_reply_decode_error_on_truncated_payload(tls_certs):
|
||||||
def _promise_more_than_sent(conn: ssl.SSLSocket) -> None:
|
def _promise_more_than_sent(conn: ssl.SSLSocket) -> None:
|
||||||
conn.sendall(struct.pack("=Q", 100))
|
conn.sendall(struct.pack("=Q", 100))
|
||||||
conn.sendall(b"only ten!") # 9 bytes, not 100 -- then hang up
|
conn.sendall(b"only ten!") # 9 bytes, not 100 -- then hang up
|
||||||
|
|
||||||
srv = _TlsEchoServer(_promise_more_than_sent)
|
srv = _TlsEchoServer(_promise_more_than_sent, tls_certs.cert_file, tls_certs.key_file)
|
||||||
try:
|
try:
|
||||||
sock = wire.connect("127.0.0.1", srv.port, CERT_DIR, timeout_s=5.0)
|
sock = wire.connect("127.0.0.1", srv.port, timeout_s=5.0)
|
||||||
try:
|
try:
|
||||||
with pytest.raises(ReplyDecodeError):
|
with pytest.raises(ReplyDecodeError):
|
||||||
wire.recv_frame(sock)
|
wire.recv_frame(sock)
|
||||||
|
|
@ -261,13 +254,13 @@ def test_recv_frame_raises_reply_decode_error_on_truncated_payload():
|
||||||
srv.close()
|
srv.close()
|
||||||
|
|
||||||
|
|
||||||
def test_recv_frame_raises_reply_decode_error_on_implausible_length():
|
def test_recv_frame_raises_reply_decode_error_on_implausible_length(tls_certs):
|
||||||
def _claim_huge_length(conn: ssl.SSLSocket) -> None:
|
def _claim_huge_length(conn: ssl.SSLSocket) -> None:
|
||||||
conn.sendall(struct.pack("=Q", wire._MAX_FRAME_BYTES + 1))
|
conn.sendall(struct.pack("=Q", wire._MAX_FRAME_BYTES + 1))
|
||||||
|
|
||||||
srv = _TlsEchoServer(_claim_huge_length)
|
srv = _TlsEchoServer(_claim_huge_length, tls_certs.cert_file, tls_certs.key_file)
|
||||||
try:
|
try:
|
||||||
sock = wire.connect("127.0.0.1", srv.port, CERT_DIR, timeout_s=5.0)
|
sock = wire.connect("127.0.0.1", srv.port, timeout_s=5.0)
|
||||||
try:
|
try:
|
||||||
with pytest.raises(ReplyDecodeError):
|
with pytest.raises(ReplyDecodeError):
|
||||||
wire.recv_frame(sock)
|
wire.recv_frame(sock)
|
||||||
|
|
@ -284,7 +277,7 @@ def test_recv_frame_raises_reply_decode_error_on_implausible_length():
|
||||||
|
|
||||||
def test_send_request_returns_exact_reply_bytes(echo_server):
|
def test_send_request_returns_exact_reply_bytes(echo_server):
|
||||||
srv = echo_server()
|
srv = echo_server()
|
||||||
sock = wire.connect("127.0.0.1", srv.port, CERT_DIR, timeout_s=5.0)
|
sock = wire.connect("127.0.0.1", srv.port, timeout_s=5.0)
|
||||||
try:
|
try:
|
||||||
reply = wire.send_request(sock, b"pretend-capnp-request-bytes")
|
reply = wire.send_request(sock, b"pretend-capnp-request-bytes")
|
||||||
assert reply == b"pretend-capnp-request-bytes"
|
assert reply == b"pretend-capnp-request-bytes"
|
||||||
|
|
@ -298,7 +291,7 @@ def test_send_request_is_opcode_agnostic_works_for_any_byte_shape(echo_server):
|
||||||
identically, since both are just bytes to this module.
|
identically, since both are just bytes to this module.
|
||||||
"""
|
"""
|
||||||
srv = echo_server()
|
srv = echo_server()
|
||||||
sock = wire.connect("127.0.0.1", srv.port, CERT_DIR, timeout_s=5.0)
|
sock = wire.connect("127.0.0.1", srv.port, timeout_s=5.0)
|
||||||
try:
|
try:
|
||||||
for payload in (b"", b"\x00", b"short", b"x" * 10_000):
|
for payload in (b"", b"\x00", b"short", b"x" * 10_000):
|
||||||
reply = wire.send_request(sock, payload)
|
reply = wire.send_request(sock, payload)
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue