forked from lei.chen/myxce
feat(admin-config-tool): drop ha/data/ entirely, match real client TLS
deploy-ha.sh no longer takes a <data-dir> argument or --build-config
flag. Every deploy now generates a fresh self-signed TLS cert/key
(openssl, RSA 2048, 10y validity) directly into the deploy dir's
certs/, and builds cluster.bin directly into controller/ctrl-1/conf/
via xceadmin -- no intermediate staging directory, nothing preserved
across runs. run-ha-soak.sh (/xce-test-ha) updated to match: dropped
its own XCE_HA_DATADIR-based precondition entirely. Updated 4 callers
(run-ha-soak.sh, test-heartbeat-tolerance.sh, test-commit-durability.sh,
both xce-deploy-ha/xce-test-ha SKILL.md docs).
Separately: xcecommon.wire.connect() was loading a client-side TLS
cert, diverging from the real reference client -- netio.c's
create_ssl_context() presents no certificate at all when acting as a
client (cert_file/key_file both NULL, bare TLS_client_method()).
Fixed connect() to match, and since nothing else consumed cert_dir
(verify_mode=CERT_NONE throughout, no CA check either), removed it
end-to-end: discovery.follow_redirects(), push.push_config()/
probe_config(), and the --cert-dir CLI flag.
Server-side cert loading (mock TLS test servers) is unaffected --
moved into a shared session-scoped tls_certs fixture (new
tests/conftest.py, openssl-generated) replacing 3 duplicated
REPO_HA_DATA_DIR/skipif blocks. Net result: those tests no longer skip
under a hermetic nix build sandbox (which never had ha/data/) --
verified 165/165 passed, 0 skipped, via both meson compile and a full
nix build --option sandbox false .#xcecore_debug rebuild (previously
128 passed / 37 skipped there).
ha/data/{xce.crt,xce.key} removed from git tracking. Requirements
2.3/2.4 and 8.3/8.4/8.5 marked superseded in requirements.md with
dated correction notes.
This commit is contained in:
parent
db2b5f17f0
commit
ee8473bd99
20 changed files with 244 additions and 297 deletions
|
|
@ -32,13 +32,15 @@ disown`) before relying on subagent dispatch for unattended use.
|
|||
## Prerequisites
|
||||
|
||||
- Build must be current: run `/xce-build controller` first if binary is stale
|
||||
- TLS certs must exist under `ha/data/`:
|
||||
- `ha/data/xce.crt`
|
||||
- `ha/data/xce.key`
|
||||
- `ha/data/cluster.bin` — either already present, or built automatically by
|
||||
`deploy-ha.sh` from `src/configs/*.yaml` via `xceadmin` (see Step 3). Run
|
||||
`/xce-build xceadmin` first (or `nix build .#xcecore`) so a built
|
||||
`xceadmin` binary exists for that auto-build path to find.
|
||||
- Run `/xce-build xceadmin` first (or `nix build .#xcecore`) so a built
|
||||
`xceadmin` binary exists — `deploy-ha.sh` (Step 3) uses it to build
|
||||
`cluster.bin` from `src/configs/*.yaml` fresh on every deploy
|
||||
- `openssl` must be on `PATH` (provided by `nix develop`) — `deploy-ha.sh`
|
||||
generates a fresh self-signed TLS cert/key for every deploy
|
||||
|
||||
No data directory needed: TLS cert/key and `cluster.bin` are both generated
|
||||
directly into the deploy dir (`$TOP`) by `deploy-ha.sh` itself, not read from
|
||||
any pre-existing `ha/data/`-style fixture.
|
||||
|
||||
## Arguments
|
||||
|
||||
|
|
@ -55,7 +57,6 @@ If no arguments are given, default to `localhost:9991 localhost:9992 localhost:9
|
|||
|
||||
```bash
|
||||
BINARY="src/xce.build/controller/xce_controller"
|
||||
DATADIR="ha/data"
|
||||
TOP="/tmp/xce-deploy"
|
||||
```
|
||||
|
||||
|
|
@ -78,27 +79,25 @@ Join peers as a comma-separated string and call the deploy script:
|
|||
|
||||
```bash
|
||||
PEERS="localhost:9991,localhost:9992,localhost:9993" # from args
|
||||
bash src/tools/deploy-ha.sh "$BINARY" "$DATADIR" "$TOP" "$PEERS"
|
||||
bash src/tools/deploy-ha.sh "$BINARY" "$TOP" "$PEERS"
|
||||
```
|
||||
|
||||
Add `--build-config` (anywhere in the args) to force a fresh `cluster.bin`
|
||||
rebuild from `src/configs/*.yaml` via `xceadmin` even if `$DATADIR/cluster.bin`
|
||||
already exists — useful after editing the example configs:
|
||||
|
||||
```bash
|
||||
bash src/tools/deploy-ha.sh --build-config "$BINARY" "$DATADIR" "$TOP" "$PEERS"
|
||||
```
|
||||
|
||||
Without the flag, an existing `$DATADIR/cluster.bin` is left untouched; one
|
||||
is auto-built via `xceadmin` only if missing. `xceadmin` is resolved from
|
||||
the meson build dir (`xce.build/xceadmin/.venv/bin/xceadmin` by default,
|
||||
also checks `build.release`/`build.dbg`), a nix-built `result/bin/xceadmin`,
|
||||
or `PATH` — in that order. If none is found, the script errors and tells
|
||||
you to run `/xce-build xceadmin` or `nix build .#xcecore` first.
|
||||
Every invocation is a fresh, self-contained deploy — no flag needed to force
|
||||
a rebuild, nothing is ever reused across deploys:
|
||||
- `cluster.bin` is always rebuilt from `src/configs/*.yaml` via `xceadmin`,
|
||||
resolved from the meson build dir (`xce.build/xceadmin/.venv/bin/xceadmin`
|
||||
by default, also checks `build.release`/`build.dbg`), a nix-built
|
||||
`result/bin/xceadmin`, or `PATH` — in that order. If none is found, the
|
||||
script errors and tells you to run `/xce-build xceadmin` or
|
||||
`nix build .#xcecore` first.
|
||||
- The TLS cert/key pair is always freshly generated via `openssl` (self-signed,
|
||||
RSA 2048, 10y validity — long-lived since it's per-deploy anyway; loses no
|
||||
real security property since TLS verification is `SSL_VERIFY_NONE`
|
||||
everywhere in this codebase). Errors if `openssl` isn't on `PATH`.
|
||||
|
||||
This:
|
||||
- Creates `$TOP/certs/`, `$TOP/controller/ctrl-{1..N}/conf`, `$TOP/controller/ctrl-{1..N}/data`, `$TOP/logs/`
|
||||
- Copies `xce.crt` and `xce.key` into the shared `$TOP/certs/` dir
|
||||
- Generates `xce.crt`/`xce.key` directly into `$TOP/certs/`
|
||||
- Places `cluster.bin.<tag>` + `conf/current` in `controller/ctrl-1/conf/` only
|
||||
(standbys receive config from primary via `OPCode_replConfigPush`)
|
||||
- Generates `$TOP/scripts/start.sh` (with `--workdir $TOP/controller --certdir $TOP/certs`) and `cleanup.sh`
|
||||
|
|
|
|||
|
|
@ -361,7 +361,6 @@ XCE_HA_ROUNDS=10 XCE_HA_TOP=/tmp/my-ha-run nix develop ./src --command bash "$SK
|
|||
| Env var | Meaning | Default |
|
||||
|---|---|---|
|
||||
| `XCE_HA_TOP` | deploy directory | `/home/claude/tmp/xce-ha-deploy` |
|
||||
| `XCE_HA_DATADIR` | config/certs dir (`xce.crt`, `xce.key`; `cluster.bin` freshly built here by `xceadmin` every run, not a fixture) | `<repo>/ha/data` |
|
||||
| `XCE_HA_MODE` | `sequential`, `chaotic`, `force-snapshot`, `factstream`, `array-soak`, or `array-chaos-retry` (its `--flag` wins) | `sequential` |
|
||||
| `XCE_HA_BIN` | `xce_controller` binary (point at real-fs release for the gate) | `<repo>/src/xce.build/controller/xce_controller` |
|
||||
| `XCE_HA_XT` | `xce_test` binary | `<repo>/src/xce.build/controller/xce_test` |
|
||||
|
|
@ -385,10 +384,18 @@ XCE_HA_ROUNDS=10 XCE_HA_TOP=/tmp/my-ha-run nix develop ./src --command bash "$SK
|
|||
|
||||
## What the driver does (maps to the requested steps)
|
||||
|
||||
1. **Preconditions** — verifies `xce_controller`, `xce_test`, and certs exist; builds `cluster.bin` fresh under `XCE_HA_DATADIR` via `xceadmin` (resolved from the meson build dir, a nix-installed prefix, or `PATH`, in that order — fails with a clear message if none is found; run `/xce-build xceadmin` or `nix build .#xcecore` first).
|
||||
1. **Preconditions** — verifies `xce_controller` and `xce_test` exist. No data
|
||||
directory, no cert/config precondition: `deploy-ha.sh` (next step)
|
||||
generates both fresh.
|
||||
2. **Deploy** — `src/tools/deploy-ha.sh` into `XCE_HA_TOP` with nodes
|
||||
`ctrl-1=localhost:9991, ctrl-2=localhost:9992, ctrl-3=localhost:9993`,
|
||||
using config/certs from `XCE_HA_DATADIR`.
|
||||
`ctrl-1=localhost:9991, ctrl-2=localhost:9992, ctrl-3=localhost:9993`.
|
||||
Generates a fresh self-signed TLS cert/key via `openssl` and builds
|
||||
`cluster.bin` from `src/configs/*.yaml` via `xceadmin` (resolved from the
|
||||
meson build dir, a nix-installed prefix, or `PATH`, in that order — fails
|
||||
with a clear message if either tool is missing; run `/xce-build xceadmin`
|
||||
or `nix build .#xcecore`, and ensure `openssl` is on `PATH` via
|
||||
`nix develop`) — both written directly into the deploy dir, nothing read
|
||||
from or written to any pre-existing data directory.
|
||||
3. **Start** — `scripts/start.sh ctrl-N -vvv` for all three, 0.5s staggered.
|
||||
4. **Elect** — detects the elected primary via a **client probe** (`xce_test`
|
||||
`wait-primary` + `ctrlstatus`), reading live controller state. Primary,
|
||||
|
|
@ -440,7 +447,8 @@ goal is "the HA soak test passes". The driver is built for that contract:
|
|||
decides done/continue by grepping that line / checking the exit code; no
|
||||
human-style parsing of intermediate output is needed.
|
||||
- **Fully idempotent & self-contained.** Every run `rm -rf`s `$XCE_HA_TOP`,
|
||||
redeploys from `ha/data`, kills any stale `xce_controller`, and cleans up on
|
||||
redeploys with a freshly generated cert and `cluster.bin` (no data
|
||||
directory involved), kills any stale `xce_controller`, and cleans up on
|
||||
exit (trap). A loop can invoke it any number of times with no manual reset.
|
||||
- **Deterministic-enough.** Cold-start election ~15s, each round ~20-30s; a
|
||||
5-round run is ~3-5 min, 10-round ~6-10 min. Budget the loop's per-iteration
|
||||
|
|
|
|||
|
|
@ -82,7 +82,6 @@
|
|||
#
|
||||
# Env overrides:
|
||||
# XCE_HA_TOP deploy dir (default /home/claude/tmp/xce-ha-deploy)
|
||||
# XCE_HA_DATADIR config/certs source (default <repo>/ha/data)
|
||||
# XCE_HA_MODE sequential | chaotic | force-snapshot (default sequential)
|
||||
# XCE_HA_ROUNDS kill/restart rounds (default 6, clamp 5..50)
|
||||
# XCE_HA_JOBS jobs per submit batch (default 3)
|
||||
|
|
@ -202,7 +201,6 @@ esac
|
|||
# Sentinel written into a deploy we own; the guard before `rm -rf` only wipes a
|
||||
# non-empty dir that carries this marker (see "Step 2: deploy" below).
|
||||
MARKER_NAME=".xce-ha-deploy"
|
||||
DATADIR="${XCE_HA_DATADIR:-$REPO/ha/data}"
|
||||
BIN="${XCE_HA_BIN:-$REPO/src/xce.build/controller/xce_controller}"
|
||||
XT="${XCE_HA_XT:-$REPO/src/xce.build/controller/xce_test}"
|
||||
DEPLOY="$REPO/src/tools/deploy-ha.sh"
|
||||
|
|
@ -302,29 +300,10 @@ kill_node() {
|
|||
# ── Preconditions ────────────────────────────────────────────────────────────
|
||||
[ -x "$BIN" ] || { echo "[xce-test-ha] FAIL: controller binary missing: $BIN (run /xce-build controller)" >&2; exit 2; }
|
||||
[ -x "$XT" ] || { echo "[xce-test-ha] FAIL: xce_test missing: $XT (run /xce-build controller)" >&2; exit 2; }
|
||||
# cluster.bin is always (re)built fresh from src/configs/*.yaml via xceadmin
|
||||
# -- no pre-created/checked-in cluster.bin fixture is used or kept around
|
||||
# any more (ha/data/cluster.bin was removed from git tracking accordingly).
|
||||
# Same xceadmin resolution order as deploy-ha.sh's --build-config path
|
||||
# (meson build dir, then a nix-installed prefix, then PATH).
|
||||
XCEADMIN=""
|
||||
for candidate in \
|
||||
"$REPO/src/xce.build/xceadmin/.venv/bin/xceadmin" \
|
||||
"$REPO/src/build.release/xceadmin/.venv/bin/xceadmin" \
|
||||
"$REPO/src/build.dbg/xceadmin/.venv/bin/xceadmin" \
|
||||
"$REPO/src/result/bin/xceadmin" \
|
||||
"$(command -v xceadmin 2>/dev/null || true)"
|
||||
do
|
||||
[ -n "$candidate" ] && [ -x "$candidate" ] && { XCEADMIN="$candidate"; break; }
|
||||
done
|
||||
if [ -z "$XCEADMIN" ]; then
|
||||
echo "[xce-test-ha] FAIL: no xceadmin binary found to build cluster.bin (run /xce-build xceadmin or nix build .#xcecore)" >&2
|
||||
exit 2
|
||||
fi
|
||||
echo "[xce-test-ha] building cluster.bin under $DATADIR via $XCEADMIN"
|
||||
"$XCEADMIN" config build "$REPO/src/configs" -o "$DATADIR/cluster.bin" \
|
||||
|| { echo "[xce-test-ha] FAIL: xceadmin config build failed" >&2; exit 2; }
|
||||
[ -r "$DATADIR/xce.crt" ] && [ -r "$DATADIR/xce.key" ] || { echo "[xce-test-ha] FAIL: certs missing under $DATADIR" >&2; exit 2; }
|
||||
# TLS cert/key and cluster.bin are no longer preconditions here at all --
|
||||
# deploy-ha.sh (Step 2 below) generates both fresh directly into $TOP on
|
||||
# every run (openssl self-signed cert; cluster.bin from src/configs/*.yaml
|
||||
# via xceadmin). No data directory, no pre-created/checked-in fixture.
|
||||
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
|
|
@ -1786,7 +1765,7 @@ fi
|
|||
rm -rf "$TOP"
|
||||
mkdir -p "$TOP" || fail "could not create deploy dir $TOP"
|
||||
touch "$TOP/$MARKER_NAME" # claim ownership so a later run may safely re-wipe it
|
||||
bash "$DEPLOY" "$BIN" "$DATADIR" "$TOP" "localhost:9991,localhost:9992,localhost:9993" \
|
||||
bash "$DEPLOY" "$BIN" "$TOP" "localhost:9991,localhost:9992,localhost:9993" \
|
||||
>"$TOP/deploy.log" 2>&1 || fail "deploy-ha.sh failed (see $TOP/deploy.log)"
|
||||
|
||||
# ── Step 3: start all nodes ──────────────────────────────────────────────────
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue