Reopens the 2026-07-21 design decision to keep xceadmin outside meson.
Standard onboarding consistency (README 3.3 `nix build .#xcecore` and
3.4 `meson compile` working for xceadmin like every other component)
outweighs the earlier concern about meson's single-artifact model not
fitting a multi-file pip install.
- src/xceadmin/meson.build: custom_target() wrapping install+test,
stamp-file output, build_always_stale (matches this repo's existing
clean-rebuild-every-invocation convention).
- src/xceadmin/meson-build.sh: out-of-tree pinned install + pytest;
stages a repo-mirror/ symlink tree so tests' fixed-relative-path
assumptions (src/configs/, src/schema/, ha/data/) still resolve from
a copied location, without touching their deliberate fail-loud
behavior.
- src/meson.build: subdir('xceadmin') under component != 'xcecli'.
- src/tools/xce-build.sh: xceadmin case now delegates to
`meson compile -C "$BUILD_DIR" xceadmin` instead of its own
pip/venv implementation, like every other component.
Verified both build paths: meson compile (165/165 passed) and
nix build --option sandbox false .#xcecore_debug (128 passed / 37
skipped — cert-dependent tests correctly skip under nix's
git-tracked-only hermetic source). AppImage bundling (README 3.5)
is explicitly deferred to future work, not part of this change.
53 KiB
Technical Design: admin-config-tool
Overview
xceadmin is a standalone Python 3.10+ CLI that compiles operator-authored YAML into the ClusterConfig Cap'n Proto binary the XCE controller expects (config build), and delivers that binary to a running controller cluster over the same TLS+Cap'n Proto wire protocol controllers already speak to each other (config push). It is the first Python component in an otherwise C/Go codebase and the first external (non-daemon) client of the controller's raw wire protocol.
Users: XCE cluster operators managing cluster topology, queues, resources, users, limits, autoscaler providers, and cluster-identity settings through git-managed YAML rather than hand-crafted binaries or manual scp.
Impact: Today cluster.bin is a hand-built, provenance-free artifact and OP_upgradeConfig has no external caller. This spec adds the missing YAML→binary compiler and the missing binary→controller uploader, closing both gaps client-side.
Goals
- Deterministic YAML →
cluster.bincompilation covering all 13ClusterConfigfields (9.1–9.3, 1.1–1.12). - TLS+capnp delivery of
cluster.binto a controller, following redirects to the primary (2.1–2.7, 3.1–3.4). - Fail-fast validation with actionable errors before anything reaches a running cluster (4.1–4.5, 7.1–7.6).
- A reusable
xcecommonpackage so a future Pythonxceclidoes not re-derive capnp/wire-protocol code (5.1–5.4). - Toolchain integration:
xce-build,xce-deploy-ha, nix dev shell (8.1–8.6).
Non-Goals
- Controller-side replication, snapshot, restart, rollback, version history (owned by
config-upgrade-snapshot). - Fixing the controller-side
OP_upgradeConfigdispatch defect discovered during this design's discovery phase (see Risks) — that is a controller bug, not an xceadmin gap. config diff— descoped to roadmap follow-up during requirements review (2026-07-21).- Peer/candidate membership management, per-instance perf tuning (
port, hash-table sizes,xstrings) — physically absent fromClusterConfigafterclusterconfig-schema-split.
Boundary Commitments
This Spec Owns
- The
xceadminCLI (config build,config push,config push --dry-run). - The
xcecommonpackage: capnp schema loading, YAML→capnp field mapping, JSON Schema + business-rule validation, TLS+framing wire client. Both schema types (controller.capnpand the 7 JSON schemas) are packaged as bundled copies (package_data), synced fromsrc/schema/andsrc/configs/schema/at build time — not read from an operator-supplied path at runtime (2026-07-21 revision; superseded Requirement 1.6, see Requirement 5.5). - Build/toolchain integration:
src/xceadmin/meson.build(meson compile/nix build .#xcecore[_debug]),xce-build.shdelegating dispatch, nix dev-shell Python packages,xce-deploy-ha--build-configflag — see File Structure Plan Design Review correction / Amendment. - Test fixtures and golden files under
src/xceadmin/tests/.
Out of Boundary
- Controller-side
OP_upgradeConfigdispatch —ha_dispatch_gate()excludesOPCode_upgradeConfigfrom the standby allow-list (ha.c:3926-3951), andcluster.c:667-679callshandle_upgrade_config()only whenha_is_primary()is false. Combined, no controller role reachesdo_upgrade_config_with_bytes()via the network today — it is exercised only by unit tests calling it directly. This is a controller defect, not an xceadmin defect; xceadmin implements the client side of the documented contract (config_repl.h:347-349— "handle an OP_upgradeConfig request from an admin tool") and cannot itself fix the gate. Flagged as a blocking risk below. - Redirect wiring for
OP_upgradeConfig—fill_redirect_reply()(join.c:27-31) is only invoked from theOPCode_joinpath (join.c:277).RequestReply.redirectHost/redirectPortare never populated for anupgradeConfigreply today. xceadmin implements generic, opcode-agnostic redirect-following per theRequestReplycontract (forward-compatible), but it will not fire forconfig pushuntil the controller is extended. ProbeConfigTagserver-side handling —xce_probe_config_tag()(config_repl.c:1829) has no server-side dispatch case anywhere incluster.cand no production caller; it is dormant plumbing. xceadmin's dry-run client implements the wire contract but has nothing live to call.- Entity-bearing snapshots, config rollback, version history, config activation status (owned by
config-upgrade-snapshot, blocked upstream onentity-bearing-snapshots). - TLS mutual authentication / hostname verification — none exists anywhere in the codebase (
SSL_VERIFY_NONEis set explicitly for the controller's own outbound client context,myapp.c:109; noX509_check_hostcall exists). xceadmin matches this existing posture; it neither introduces nor removes an authentication guarantee.
Allowed Dependencies
src/schema/controller.capnpandsrc/schema/c.capnp— schema source of truth, read-only.xceadminpackages a synced copy (see File Structure Plan /xcecommon.schema) rather than referencing this path directly at runtime —src/schema/itself stays owned by the C build, not by this spec.src/configs/*.yaml— the 7 example configs already authored (this session) as operator-facing documentation; xceadmin's test fixtures are separate, purpose-built golden inputs undersrc/xceadmin/tests/fixtures/, not these files.src/configs/schema/*.schema.jsonis different in kind from the YAML examples: it's the source of truthxcecommonpackages a synced copy of (2026-07-21 revision — seexcecommon.validation), the same relationshipsrc/schema/controller.capnpalready has.xce.crt/xce.keyunderha/data/(or--cert-dir) — same cert bundle the controller cluster already uses.- Nix
python3Packages:pycapnp,pyyaml,jsonschema,click,pytest,ruff(net-new additions tosrc/nixSupport/xce.nix). src/xceadmin/meson.build'scustom_target()as the build-integration point, delegated to bysrc/tools/xce-build.sh's case-statement dispatch pattern — see Design Review correction / Amendment below.
Revalidation Triggers
controller.capnpClusterConfigfield renumbering/addition/removal (already happened once this session viaclusterconfig-schema-split) — invalidates the field-mapping table in Data Models.- A fix to the
cluster.c/ha.cdispatch-gate contradiction — flipsconfig pushE2E testing from "mock controller only" to "real cluster required," per Testing Strategy. fill_redirect_reply()extended toOPCode_upgradeConfig— unlocks live auto-primary-discovery testing for config push (currently only testable againstOPCode_join-shaped mocks).- A server-side handler added for
OPCode_probeConfigTag— unlocksconfig push --dry-runE2E testing.
Architecture
Existing Architecture Analysis
XCE's daemon-to-daemon wire protocol is hand-rolled: iomgr/vtaskmgr (epoll loop + thread-pool dispatch) drive a raw-SSL-over-TCP channel framed as uint64_t size prefix (native/host byte order, not network byte order — confirmed at netio.c:190-199, request.c:863-938, config_repl.c:1925-1968, no byte-swap anywhere in the path) followed by that many bytes of a Cap'n Proto Request/Reply message. There is no Cap'n Proto RPC layer (no vat network) — every exchange is one framed message out, one framed message back. src/lib/request.c already contains a minimal one-shot raw-SSL helper pattern (xce_probe_config_tag-adjacent code, request.c:863-938) that this design mirrors from the Python side.
No precedent exists for a non-daemon client of this protocol: src/tools/ has no socket/capnp client, xce_test is a WAL/snapshot file-inspection CLI with no network code, and xcecli (Go) speaks HTTP/JSON to apiserver, never this raw protocol. xceadmin is the first.
Architecture Pattern & Boundary Map
Selected pattern: layered CLI — thin click entry point over a reusable library (xcecommon), matching the project's existing controllers → services → store/backend inward-dependency convention (structure.md) adapted for a one-shot tool (no persistent store; "runtime" is a single TLS exchange or a single file write).
graph TB
CLI[xceadmin cli] --> Build[xceadmin build]
CLI --> Push[xceadmin push]
Build --> Validation[xcecommon validation]
Build --> Mapping[xcecommon mapping]
Push --> Wire[xcecommon wire]
Push --> Discovery[xceadmin discovery]
Discovery --> Wire
Mapping --> Schema[xcecommon schema]
Wire --> Schema
Validation --> YamlFiles[YAML configs]
Mapping --> ClusterBin[cluster bin file]
Wire --> Controller[XCE controller]
Architecture Integration:
- Selected pattern: layered CLI over a shared library — the library (
xcecommon) has zero CLI/clickdependency so a future Pythonxcecli(brief.md Future Roadmap) can import it directly. - Domain boundaries:
xcecommonowns protocol/schema/mapping concerns;xceadminowns orchestration and CLI ergonomics (exit codes, stderr/stdout conventions, flag parsing). Neither owns controller-side behavior. - Existing patterns preserved: framing format, TLS cert bundle reuse, opcode/union shapes are consumed exactly as the C side defines them — no protocol reinterpretation.
- New components rationale:
xcecommondid not exist (first Python in the repo); every module below is net-new. - Steering compliance: Cap'n Proto for wire format (
structure.md— "Cap'n Proto for wire formats"), component lives undersrc/<component>/(structure.md— "Organization Philosophy"), fail-fast input validation (security.md— "Validate at the API boundary").
Dependency Direction: xcecommon.schema → {xcecommon.mapping, xcecommon.wire} → xcecommon.validation (independent, no inbound dependency from schema/mapping/wire) → {xceadmin.build, xceadmin.discovery} → xceadmin.push → xceadmin.cli. Each layer imports only from layers to its left; xcecommon.* never imports anything from xceadmin.* — this is what keeps xcecommon importable by a future Python xcecli without dragging in click or CLI-specific exit-code handling.
Parallel-safe implementation boundaries: within xcecommon, validation.py has no dependency on schema.py/mapping.py/wire.py and can be implemented independently of them; mapping.py and wire.py both depend only on schema.py, not on each other, so they can be implemented in parallel once schema.py is stable. Within xceadmin, build.py (depends on xcecommon.{validation,mapping}) and discovery.py+push.py (depend on xcecommon.wire) touch disjoint xcecommon modules and can be implemented in parallel once the underlying xcecommon layer is stable.
Technology Stack
| Layer | Choice / Version | Role in Feature | Notes |
|---|---|---|---|
| CLI | click (nixpkgs python3Packages.click) |
config build/config push/config push --dry-run subcommand groups |
Grouped-command idiom: @click.group() → @cli.group() → @config.command() |
| Capnp | pycapnp ~2.2.x (nixpkgs) |
Load controller.capnp, build/encode ClusterConfig, decode Request/Reply |
Dynamic-reflection (DynamicStruct) API; $C.mapname(...) annotations are declared-and-inert, not a parse hazard (verified against capnp.pyx source) |
| YAML | pyyaml (nixpkgs) |
Parse the 7 config files | — |
| Schema validation | jsonschema (nixpkgs), Draft 2020-12 |
Validate each YAML against xceadmin's bundled JSON schemas (package_data, same mechanism as xcecommon.schema's controller.capnp) |
Source of truth is src/configs/schema/*.schema.json; synced into the package at build time, never read from configs_dir at runtime (2026-07-21 revision — see xcecommon.validation) |
| Transport | Python stdlib ssl + socket |
TLS-wrapped TCP to controller | ssl.SSLContext with check_hostname=False, verify_mode=ssl.CERT_NONE — matches the controller's own outbound posture (myapp.c:109), not a weakening |
| Test framework | pytest (nixpkgs) |
Golden-file + mock-controller tests | Plain byte-equality fixture comparison (see Testing Strategy — pytest-regressions considered, not adopted, to avoid an unverified new nix package) |
| Packaging | pyproject.toml + pip install -e . (dev) / pip install . (release) |
Editable vs. pinned install | Runs inside nix dev shell so pycapnp's C-extension build links nix-provided capnproto |
File Structure Plan
Directory Structure
src/xceadmin/
├── pyproject.toml # package metadata; entry point xceadmin = xceadmin.cli:main
│ # Wired into meson via src/xceadmin/meson.build (2026-07-22
│ # amendment — see Design Review correction below)
├── src/
│ ├── xcecommon/ # shared library — no click/CLI dependency, reusable by future Python xcecli
│ │ ├── __init__.py
│ │ ├── errors.py # XceAdminError base + all subclasses (single CLI dispatch surface)
│ │ ├── schema_files/ # packaged copies, synced from src/ at build (not hand-edited)
│ │ │ ├── controller.capnp # <- src/schema/controller.capnp
│ │ │ ├── c.capnp # <- src/schema/c.capnp
│ │ │ └── json/ # <- src/configs/schema/*.schema.json (7 files)
│ │ ├── schema.py # capnp.load(controller.capnp) wrapper + typed accessors; resolves schema_files/ via importlib.resources
│ │ ├── mapping.py # declarative YAML-section -> capnp-field table + 2 mapper shapes (list, scalar)
│ │ ├── validation.py # JSON Schema validation (bundled schema_files/json/), reserved-name / dup / undefined-ref checks
│ │ └── wire.py # TLS connect, 8-byte native-order framing, generic send_request()
│ └── xceadmin/
│ ├── __init__.py
│ ├── cli.py # click group: config build / config push / config push --dry-run
│ ├── build.py # config build orchestration: scan dir -> validate -> map -> serialize -> sha256
│ ├── push.py # config push orchestration: connect -> discover primary -> send -> display
│ └── discovery.py # generic redirect-following (Req 3) — opcode-agnostic
└── tests/
├── fixtures/
│ ├── minimal/configs/{hosts,queues}.yaml + expected.bin # no schema/ — validated against xceadmin's bundled schemas
│ ├── full/configs/{hosts,queues,resources,users,limits,xce,scaler}.yaml + expected.bin
│ └── invalid/configs/... # one fixture dir per validation-error case (4.1, 4.2, 4.3, 4.4, 4.5)
├── test_schema_bundling.py # default_schema_path()/default_schema_dir() resolve under the active install (5.5) — only meaningful under `xce-build xceadmin release`, see Testing Strategy
├── test_mapping.py # pure unit tests: map_list_section/map_scalar_section against in-memory dicts, no file I/O (1.1, 1.7, 1.8, 1.9, 1.10, 1.11, 1.12, 5.3, 9.1, 9.2, 9.3) — gap closed during task generation, was in Testing Strategy's "Unit — mapping" bullet but missing from this file list
├── test_config_build.py # golden-file build tests (1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 1.10, 1.11, 1.12, 9.1, 9.2, 9.3)
├── test_schema_validation.py # per-error-case validation tests (4.1, 4.2, 4.3, 4.4, 4.5)
├── test_wire_framing.py # framing pack/unpack against known byte sequences (2.1)
└── test_config_push_mock.py # mock TLS server, opcode + redirect + error-reply cases (2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 3.1, 3.2, 3.3, 3.4, 7.1, 7.2, 7.3)
xcecommon and xceadmin are separate top-level packages under one src/xceadmin/src/ tree, installed together via one pyproject.toml for now (two console-ready packages, one distributable unit) — splitting into two distributions is deferred until a second consumer (the future Python xcecli) actually exists (Simplification: no speculative packaging split).
Modified Files
src/tools/xce-build.sh— newxceadmin)case: syncsrc/schema/{controller,c}.capnp→src/xceadmin/src/xcecommon/schema_files/andsrc/configs/schema/*.schema.json→src/xceadmin/src/xcecommon/schema_files/json/(cp, source of truth stayssrc/schema/andsrc/configs/schema/respectively — 2026-07-21 revision extends the sync to JSON schemas, Requirement 5.5), thenpip install -e ./pip install .by profile, thenpytest(8.1, 8.2). The sync step runs on every build so apip install .(non-editable, Req 8.2) always packages the current schemas aspackage_data— without it,load_schema()/xcecommon.validation's default resolution would silently package stale or missing copies.pytestruns after whichever install just happened, not before — this ordering is what makes thereleaseprofile's test run (8.2) the only one that actually exercisesdefault_schema_path()/default_schema_dir()resolving under a real non-editablesite-packageslayout; the default (dev/editable) profile's test run (8.1) never would, since editable installs keep resolving into the checked-outschema_files/copy either way. Design Review correction (original, 2026-07-21):xceadminis dispatched directly by this case statement and is deliberately not wired intosrc/meson.build/src/meson.options/asrc/xceadmin/meson.build—xcecli'scustom_target()pattern fits a single static-binary output;pip install -e .mutates a venv/site-packages tree with no single trackable output artifact, so forcing it through meson's build graph would be a dead end.xceadminsits orthogonal to the meson component option entirely, matching whatbrief.md's harness sketch already specified before this design pass over-engineered a meson integration.
Amendment (2026-07-22, supersedes the correction above): the user reopened this decision — standard-onboarding consistency (README 3.3 nix build .#xcecore[_debug] and 3.4 meson compile working for xceadmin exactly like every other component) outweighs the multi-file-output concern. src/xceadmin/meson.build wraps the install+test cycle in a custom_target() whose output is a stamp file (xceadmin-build.stamp), not a tracked multi-file tree — the real work (out-of-tree pip install . into a build-dir-local venv, then pytest) runs in src/xceadmin/meson-build.sh, invoked with build_always_stale: true so it reruns unconditionally rather than relying on ninja's file-mtime cache (matching this project's existing "clean rebuild every invocation" convention, same as every xce-build.sh component's rm -rf "$BUILD_DIR"). src/meson.build gains subdir('xceadmin') inside the component != 'xcecli' block, so it builds under component=all/component=xcecore (where xcecore's mesonFlags already sets -Dcomponent=xcecore) but not under a pure xcecli build. src/tools/xce-build.sh's xceadmin) case now just delegates (setup_build; meson compile -C "$BUILD_DIR" xceadmin) instead of carrying its own separate pip/venv implementation — the one meaningful behavior change is that the editable-vs-pinned distinction (Req 8.1 vs. 8.2) collapses: every xce-build.sh xceadmin invocation now runs the pinned/non-editable install path, a strictly stronger test (it exercises real package_data resolution on every run, not just under a release profile). A purity constraint the original design's rejection didn't have to solve: several tests reach sibling directories via a fixed relative path (src/configs/, src/schema/, ha/data/xce.crt/.key) that assume the checked-out repo's exact directory nesting; meson-build.sh resolves this by staging a repo-mirror/ tree in the build dir with symlinks recreating that exact layout, rather than touching the tests' deliberate fail-loud (not skip-on-missing-layout) assertions. Verified both paths end-to-end: meson compile -C xce.build xceadmin (165/165 passed, real ha/data/xce.crt symlinked from the live checkout) and nix build --option sandbox false .#xcecore_debug (128 passed / 37 skipped — the skips are the existing skipif on ha/data/xce.crt not present, correctly firing since nix's git-tracked-only flake source has no committed cert files; 0 failed). AppImage bundling (README 3.5) is explicitly deferred — out of scope for this amendment, tracked as roadmap future work, not implemented here.
src/nixSupport/xce.nix— addpython3,python3Packages.{pycapnp,pyyaml,jsonschema,click,pytest,ruff}tonativeBuildInputs(8.6)..claude/skills/xce-build/SKILL.md— document thexceadmincomponent and its pip-based (non-meson-profile) build path (8.1, 8.2).src/tools/deploy-ha.sh— (gap closed during task generation — the actual runtime behavior for 8.3-8.5 belongs here, not just in documentation) add a--build-configflag; before starting the cluster, if--build-configis passed orha/data/cluster.binis missing, runxceadmin config build configs/ -o ha/data/cluster.bin(8.3, 8.4); ifcluster.binalready exists and--build-configis not passed, use it as-is, unchanged (8.5)..claude/skills/xce-deploy-ha/SKILL.md— document the--build-configflag and the new precondition step added todeploy-ha.shabove (8.3, 8.4, 8.5)..claude/settings.local.json/.claude/settings.json— permission allowlist and ruff format/check hooks for*.py(full JSON already drafted inbrief.mdHarness Enhancement section; applied verbatim here, not re-derived).
System Flows
Config Build
flowchart TD
Start[xceadmin config build dir] --> ScanFiles[scan 7 known yaml filenames]
ScanFiles --> PerFile{file present}
PerFile -- no --> EmptySection[treat section as empty]
PerFile -- yes --> Validate[validate against bundled jsonschema]
Validate -- fail --> ErrValidation[report file field constraint exit nonzero]
Validate -- pass --> BusinessRules[reserved name dup check undefined ref warn]
BusinessRules --> Map[map to ClusterConfig fields]
EmptySection --> Map
Map --> Encode[pycapnp encode]
Encode --> Write[write cluster bin and sha256 to stderr]
Gating conditions: a missing individual YAML file is not an error (1.5, empty section) — there is no "missing schema" error mode at all (1.6, superseded 2026-07-21: schemas are bundled with xceadmin, not read from configs_dir). Schema validation failures are hard errors (4.1); business-rule warnings (undefined references) do not stop the build (4.4).
Config Push (including auto-discovery)
sequenceDiagram
participant Op as Operator
participant X as xceadmin push
participant C1 as Controller node1
participant C2 as Controller node primary
Op->>X: config push cluster.bin
X->>C1: TLS connect, Request OP_upgradeConfig
C1-->>X: Reply
alt redirectHost set
X->>C2: TLS connect, resend Request
C2-->>X: Reply UpgradeConfigReply
end
X->>Op: versionTag ackedCount error, exit code
At most 3 redirect hops (3.2); each hop is a fresh TLS connection with the same 30s timeout (3.4, 7.6). Because fill_redirect_reply() does not currently fire for OPCode_upgradeConfig (see Boundary Commitments), the alt branch above is exercised only in mock-controller tests until the controller gap is closed.
Requirements Traceability
| Requirement | Summary | Components | Interfaces | Flows |
|---|---|---|---|---|
| 1 | Config Build | xceadmin.build, xcecommon.{validation,mapping,schema} |
build_cluster_config(), map_section() |
Config Build |
| 2 | Config Push | xceadmin.push, xcecommon.wire |
send_request(), push_config() |
Config Push |
| 3 | Auto Primary Discovery | xceadmin.discovery |
follow_redirects() |
Config Push |
| 4 | Validation | xcecommon.validation |
validate_yaml(), check_business_rules() |
Config Build |
| 5 | xcecommon Shared Package | xcecommon.{schema,wire,mapping,validation} |
package-level exports, default_schema_path(), default_schema_dir() |
— |
| 6 | Dry-Run Push (Nice-to-Have) | xceadmin.push, xcecommon.wire |
probe_config() |
Config Push (dry-run variant, not diagrammed — mirrors push minus commit) |
| 7 | Error Handling & Resilience | xceadmin.cli, xcecommon.wire |
click.ClickException mapping, socket timeout handling |
Config Push |
| 8 | Skill & Harness Integration | build/nix/skill files (File Structure Plan — Modified Files) | — | — |
| 9 | Deterministic Builds | xcecommon.mapping, xceadmin.build |
build_cluster_config() (no wall-clock/host/user inputs) |
Config Build |
Components and Interfaces
| Component | Domain/Layer | Intent | Req Coverage | Key Dependencies (P0/P1) | Contracts |
|---|---|---|---|---|---|
xcecommon.schema |
Library | Load controller.capnp, expose typed roots |
5.1, 5.5, 7.4 | controller.capnp (P0) |
Service |
xcecommon.mapping |
Library | YAML section dict → capnp field values | 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 1.10, 1.11, 1.12, 5.3, 9.1, 9.2, 9.3 | xcecommon.schema (P0) |
Service |
xcecommon.validation |
Library | JSON Schema + business-rule checks | 4.1, 4.2, 4.3, 4.4, 4.5, 5.5 | xcecommon/schema_files/json/*.schema.json (P0, bundled) |
Service |
xcecommon.wire |
Library | TLS connect, framing, generic request/reply | 2.1, 2.3, 2.4, 5.2, 6.1, 7.1, 7.3, 7.6 | xce.crt/xce.key (P0), xcecommon.schema (P0) |
Service |
xceadmin.build |
CLI orchestration | config build command |
1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 1.10, 1.11, 1.12, 9.1, 9.2, 9.3 | xcecommon.{validation,mapping} (P0) |
Service |
xceadmin.push |
CLI orchestration | config push / --dry-run |
2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 6.1, 6.2, 6.3, 7.1, 7.2, 7.3, 7.4, 7.5, 7.6 | xcecommon.wire (P0), xceadmin.discovery (P0) |
Service |
xceadmin.discovery |
CLI orchestration | Redirect-following | 3.1, 3.2, 3.3, 3.4 | xcecommon.wire (P0) |
Service |
xceadmin.cli |
CLI entry | Argument parsing, exit codes | 1.2, 1.3, 2.2, 2.4, 7.1, 7.2, 7.3, 7.4, 7.5, 7.6, 8.1, 8.2 | all xceadmin.* (P0) |
Service |
Only components introducing a new boundary get a full block below; xceadmin.cli is presentation-only (click wiring) and is covered by the summary row plus its Implementation Note.
Library
xcecommon.schema
| Field | Detail |
|---|---|
| Intent | Load and cache the compiled controller.capnp schema; expose the ClusterConfig, Request, Reply, UpgradeConfigRequest/Reply, ProbeConfigTagRequest/Reply types. |
| Requirements | 5.1, 5.5, 7.4 |
Responsibilities & Constraints
- Single load point —
capnp.load(path, imports=[schema_dir])called once per process; other modules import the returned module object, never re-load. - Resolves
c.capnp's sibling-directory import via theimportsargument, pointed at whereverdefault_schema_path()resolved the copy to. - Read-only: never mutates or re-serializes the schema itself.
- Default path resolution (Design Review correction, 2026-07-21 — a hosted-discussion follow-up caught that the first draft never specified this, which meant
pip install ./ Req 8.2 was unimplementable as designed):default_schema_path()usesimportlib.resources.files("xcecommon") / "schema_files" / "controller.capnp"— this resolves correctly for bothpip install -e .(Req 8.1, still points into the checked-out repo'sschema_files/copy) andpip install .(Req 8.2, resolves into whateversite-packagescopypackage_datainstalled) identically, unlike a__file__-relative../../../schema/walk, which only survives editable installs.--schema-path <path>(CLI) /XCEADMIN_SCHEMA_PATH(env var) override the default for troubleshooting, per 7.4's "suggest checking the installation."
Dependencies
- External:
pycapnp(P0) — schema compilation andDynamicStructmessage API. - External:
xcecommon/schema_files/{controller,c}.capnp(P0) — packaged copy, synced fromsrc/schema/at build time (File Structure Plan → Modified Files); failure path is 7.4.
Contracts: Service [x] / API [ ] / Event [ ] / Batch [ ] / State [ ]
Service Interface
def default_schema_path() -> str:
"""importlib.resources.files('xcecommon') / 'schema_files' / 'controller.capnp',
overridable via --schema-path / XCEADMIN_SCHEMA_PATH."""
def load_schema(schema_path: str | None = None) -> "capnp.Capnp":
"""Load controller.capnp (default_schema_path() if schema_path is None);
raise SchemaNotFoundError if the resolved path or its c.capnp import is missing."""
class SchemaNotFoundError(XceAdminError):
path: str
- Preconditions:
schema_path(explicit or resolved) points atcontroller.capnp;c.capnpsits alongside it. - Postconditions: returned module exposes
.ClusterConfig,.Request,.Reply,.UpgradeConfigRequest,.UpgradeConfigReply,.ProbeConfigTagRequest,.ProbeConfigTagReply. - Invariants: no network or filesystem writes.
xcecommon.mapping
| Field | Detail |
|---|---|
| Intent | Convert validated YAML section dicts into ClusterConfig capnp field values, and back (field metadata reused by any future config diff). |
| Requirements | 1.1, 1.7, 1.8, 1.9, 1.10, 1.11, 1.12, 5.3, 9.1, 9.2, 9.3 |
Responsibilities & Constraints
- One declarative table (
SECTION_MAP) drives all 7 sections — each entry states: YAML filename, top-level YAML key, target capnp field name (@N), and mapper kind (listorscalar). Two mapper functions (map_list_section,map_scalar_section) implement every section — no per-file bespoke mapper (Generalization, seeresearch.md). - Two mapper shapes only: list sections (
resources,queues,ugroups,hgroups,hostConfigs,hproviders,limits— YAML top key is an array, mapped toList(Struct)) and scalar section (xce.yaml— flat key/value, mapped to 5 top-levelClusterConfigscalar fields). - Enforces 9.1, 9.2, 9.3: no timestamp/hostname/username is ever read into a mapped value; list-section ordering follows YAML source order exactly, not sorted or hashed (order stability is required for byte-identical output).
configGeneration(@12) is never touched by this module (1.9) — capnp leaves it at its default (0) unconditionally.candidates,port, hash-table-size fields,xstringsdo not exist in the schema (postclusterconfig-schema-split) — there is nothing for this module to exclude; 1.8/1.10 are satisfied structurally, not by runtime logic.
Dependencies
- Inbound:
xceadmin.build(P0) — sole caller. - Outbound:
xcecommon.schema(P0) — target field types.
Contracts: Service [x]
Service Interface
from typing import Literal, TypedDict
class SectionSpec(TypedDict):
yaml_file: str # e.g. "queues.yaml"
yaml_key: str # e.g. "queues"
capnp_field: str # e.g. "queues"
kind: Literal["list", "scalar"]
SECTION_MAP: list[SectionSpec] # 7 entries, one per YAML file
def map_list_section(spec: SectionSpec, items: list[dict], target: "capnp._DynamicStructBuilder") -> None:
"""Populate target.<capnp_field> as a capnp List sized to len(items)."""
def map_scalar_section(spec: SectionSpec, values: dict, target: "capnp._DynamicStructBuilder") -> None:
"""Set each scalar capnp field on target directly from values."""
- Preconditions:
items/valuesalready passedxcecommon.validation. - Postconditions:
target(aClusterConfigbuilder) has exactly the fields for the given section set; no other field is touched. - Invariants: idempotent given the same input dict — calling twice with identical input produces byte-identical serialized output (9.1).
Full field-mapping table (Data Models has the authoritative field-by-field version — see Data Contracts & Integration).
xcecommon.validation
| Field | Detail |
|---|---|
| Intent | JSON Schema validation plus business rules (reserved names, duplicates, undefined references) not expressible in JSON Schema alone. |
| Requirements | 4.1, 4.2, 4.3, 4.4, 4.5, 5.5 |
Responsibilities & Constraints
- Bundled schemas (2026-07-21 revision, Requirement 5.5, supersedes Requirement 1.6):
default_schema_dir()usesimportlib.resources.files("xcecommon") / "schema_files" / "json"— sameimportlib.resourcesmechanism asxcecommon.schema.default_schema_path(), correct for bothpip install -e .andpip install ..configs_diris never consulted for aschema/subdirectory; there is no "schema not found in configs_dir" error path anymore. No CLI override exists for this one (unlike--schema-pathfor the capnp schema) — Simplification: no requirement or use case calls for operator-swappable validation schemas, only for pointing at a differentcontroller.capnpduring cross-version troubleshooting. validate_yaml(file, schema_name)—schema_nameis the YAML file's base name with.yamlstripped (e.g.queues.yaml→"queues"→default_schema_dir() / "queues.schema.json"), matchingSECTION_MAP'syaml_fileentries and the naming convention the 7 files undersrc/configs/schema/already follow.jsonschema.Draft202012Validator; on failure raises with file name, JSON-pointer field path, and violated constraint (4.1).check_reserved_names(data)— rejects"lost_and_found"for any queue/host name (4.2), independent of schema validity.check_duplicates(all_sections)— per-file/per-section uniqueness only; a limit name may collide with a queue name (4.3, scoped intentionally). Raises likecheck_reserved_names, not return-and-let-caller-decide — 4.3, like 4.2, is an unconditional hard-fail per requirements.md, so the two share the same raise-on-violation contract (Design Review correction: an earlier draft hadcheck_duplicatesreturn messages for the caller to judge, which was an unjustified control-flow inconsistency between two checks with identical fail-semantics).check_undefined_refs(all_sections)— cross-file warnings only (queue→host-group, limit→group/resname); never blocks the build (4.4).resources.yamlis treated as the exhaustive resource catalog — everyresnameelsewhere, including the built-in/dynamic names (slots,mem, the 8add_hostattrs_resvals()attributes), must resolve against it or this check warns (4.5).
Dependencies
- Inbound:
xceadmin.build(P0). - External:
jsonschema(P0),xcecommon/schema_files/json/*.schema.json(P0, bundled — synced fromsrc/configs/schema/at build time, File Structure Plan → Modified Files).
Contracts: Service [x]
Service Interface
class ValidationError(XceAdminError):
file: str
json_pointer: str
constraint: str
class ReservedNameError(XceAdminError): ...
class DuplicateEntryError(XceAdminError): ...
def default_schema_dir() -> str:
"""importlib.resources.files('xcecommon') / 'schema_files' / 'json'."""
def validate_yaml(file_path: str, schema_name: str) -> dict: ... # raises ValidationError; schema_name resolved under default_schema_dir()
def check_reserved_names(sections: dict) -> None: ... # raises ReservedNameError
def check_duplicates(sections: dict) -> None: ... # raises DuplicateEntryError
def check_undefined_refs(sections: dict) -> list[str]: # returns warning messages, never raises — the one intentionally non-fatal check (4.4)
All raising exceptions here subclass XceAdminError (defined in xcecommon.errors, see xceadmin.cli Implementation Note) so the CLI boundary has one dispatch surface instead of catching unrelated types per module.
xcecommon.wire
| Field | Detail |
|---|---|
| Intent | TLS connection, the 8-byte native-order framing, and a generic single-request/single-reply exchange — opcode-agnostic so it also serves ProbeConfigTag's direct-struct variant. |
| Requirements | 2.1, 2.3, 2.4, 5.2, 6.1, 7.1, 7.3, 7.6 |
Responsibilities & Constraints
connect(host, port, cert_dir, timeout=30)—socket.create_connectionwrapped byssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)withcheck_hostname=False,verify_mode=ssl.CERT_NONE, loadingxce.crt/xce.keyfromcert_dir(2.3, 2.4, 7.1, 7.6). This mirrors the controller's own outbound TLS posture exactly (myapp.c:103-109) — confirmed no CN/peer-cert check exists anywhere in the codebase to diverge from.send_frame(sock, payload: bytes)/recv_frame(sock) -> bytes—struct.pack("=Q", len(payload))/struct.unpack("=Q", ...), native byte order, matchingnetio.c's raw-memory-copy framing exactly (verified: nohtole64/htobe64anywhere in the C framing path).send_request(sock, request: "Request") -> "Reply"— generic: encode via.to_bytes(), frame, send, receive frame, decode viaReply.from_bytes(). Used forOP_upgradeConfig; the same primitives (not the same function, sinceProbeConfigTagRequest/Replyare direct structs, notRequest/Reply-union-wrapped) serveconfig push --dry-run'sProbeConfigTagclient.- Never passes an
SSLSocketor itsfileno()into a pycapnp read/write call — decode/encode happens in-memory (from_bytes/to_bytes) against plaintext buffers assembled byrecv_frame/send_frame, so TLS is never bypassed (verified gotcha from discovery: pycapnp's.write()callsfileno()and writes to the raw fd, which would skip encryption entirely on a wrapped socket).
Dependencies
- Inbound:
xceadmin.push,xceadmin.discovery(P0). - External: Python stdlib
ssl,socket,struct(P0);xcecommon.schema(P0).
Contracts: Service [x]
Service Interface
import socket, ssl
def connect(host: str, port: int, cert_dir: str, timeout_s: float = 30.0) -> ssl.SSLSocket: ...
def send_frame(sock: ssl.SSLSocket, payload: bytes) -> None: ...
def recv_frame(sock: ssl.SSLSocket) -> bytes: ...
def send_request(sock: ssl.SSLSocket, request_bytes: bytes) -> bytes:
"""Frame+send request_bytes, receive+unframe one reply. Raises
ConnectionTimeoutError / ConnectionRefusedError on failure."""
class ConnectionTimeoutError(XceAdminError): host: str; port: int
class ReplyDecodeError(XceAdminError): reason: str # corrupt/truncated capnp bytes, 7.3
- Preconditions:
cert_dircontainsxce.crt+xce.key. - Postconditions: exactly one request sent, exactly one reply returned, per call — no retry, no connection pooling (one-shot CLI, not a daemon; Simplification).
- Invariants: socket closed on every exit path, including exceptions and
SIGINT(7.5).
CLI Orchestration
xceadmin.build
| Field | Detail |
|---|---|
| Intent | config build <dir> command: scan → validate → map → serialize → write → print SHA-256. |
| Requirements | 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.9, 8.3, 8.4 |
Responsibilities & Constraints
- Scans for the 7 known filenames only (
SECTION_MAPkeys) — not an open-ended directory walk (1.1). -o <path>/-o -(stdout) output targets (1.2, 1.3); SHA-256 always to stderr, regardless of-otarget (1.4).- No
schema/subdirectory check — schemas are bundled (1.6, superseded); missing individual YAML file → empty section, continue (1.5). - Unrecognized-but-schema-valid fields → warning, not an error (1.7).
Dependencies
- Outbound:
xcecommon.validation(P0),xcecommon.mapping(P0),xcecommon.schema(P0, transitively).
Contracts: Service [x]
Service Interface
def build_cluster_config(configs_dir: str) -> bytes:
"""Full pipeline: validate all present sections, map to ClusterConfig,
serialize. Raises ValidationError / ReservedNameError / DuplicateEntryError
on hard failures (all subclass XceAdminError)."""
def write_output(data: bytes, output_path: str) -> str:
"""Writes data to output_path ('-' => stdout). Returns hex SHA-256."""
xceadmin.push
| Field | Detail |
|---|---|
| Intent | config push <file> (commit) and config push --dry-run <file> (probe): connect, discover primary, send, display result. Two distinct functions, not one boolean-flagged function — the two wire messages (Request{upgradeConfig} vs. direct-struct ProbeConfigTagRequest) and their replies (UpgradeConfigReply{versionTag, ackedCount, error} vs. ProbeConfigTagReply{configTag}) share no fields, so a shared return type would misrepresent one of the two (Design Review correction — an earlier draft's single push_config(..., dry_run: bool) -> PushResult implied acked_count exists on a dry-run probe reply, which it does not). |
| Requirements | 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 6.1, 6.2, 6.3, 7.1, 7.2, 7.3, 7.4, 7.5, 7.6 |
Responsibilities & Constraints
- Builds the
Request{opcode: upgradeConfig, upgradeconfigrequestmsg: {sha256, bytes}}message exactly percontroller.capnp:628-654,768-771(2.1). - Delegates connection + redirect-following to
xceadmin.discoverybefore sending (3.1, 3.2, 3.3, 3.4). --dry-runsendsProbeConfigTagRequestinstead (direct struct, noRequestwrapper — seexcecommon.wire), reports acceptance without committing (6.1, 6.2, 6.3). Per Boundary Commitments, this path has no live server counterpart today; it is implemented against the documented contract and exercised only by the mock-controller test suite until the controller gains a handler.- Maps every failure mode to a specific message + non-zero exit per 7.1, 7.2, 7.3, 7.5, 7.6 (connection failure, non-zero
Reply.status, corrupt binary, SIGINT, timeout).
Dependencies
- Outbound:
xcecommon.wire(P0),xceadmin.discovery(P0).
Contracts: Service [x]
Service Interface
class PushResult:
version_tag: int
acked_count: int
error: str
class DryRunResult:
current_generation: int # from ProbeConfigTagReply.configTag
would_be_generation: int # current_generation + 1, computed client-side for display only (6.2)
accepted: bool
reason: str # populated when accepted is False (6.3)
def push_config(cluster_bin_path: str, controller: str, cert_dir: str) -> PushResult: ...
def probe_config(cluster_bin_path: str, controller: str, cert_dir: str) -> DryRunResult: ...
- Postconditions (
push_config): exit 0 iffacked_count > 0anderror == ""(2.6); otherwise exit non-zero witherrordisplayed (2.7). - Postconditions (
probe_config): exit 0 and"OK: controller will accept this config (generation N→M)"iffaccepted(6.2); otherwise exit non-zero withreasondisplayed (6.3).
xceadmin.discovery
| Field | Detail |
|---|---|
| Intent | Generic, opcode-agnostic redirect-following against RequestReply.redirectHost/redirectPort. |
| Requirements | 3.1, 3.2, 3.3, 3.4 |
Responsibilities & Constraints
- Inspects any
Replyfor non-emptyredirectHost/redirectPort; reconnects and resends up to 3 hops (3.1, 3.2), then errors. - Displays the redirect target when a standby redirects (3.3).
- 30s "no primary elected" timeout is a property of each connection attempt, not the overall discovery loop (3.4, delegated to
xcecommon.wire.connect). - Implemented generically against the
RequestReplyschema shape (notupgradeConfig-specific) so it is correct today forOPCode_join-style redirects and automatically correct forupgradeConfigonce the controller wiresfill_redirect_reply()for that opcode too (Simplification — no opcode-specific branching to remove later).
Dependencies
- Outbound:
xcecommon.wire(P0).
Contracts: Service [x]
Service Interface
def follow_redirects(initial_host: str, initial_port: int, cert_dir: str, send_fn) -> tuple[str, int, bytes]:
"""send_fn(host, port) -> reply_bytes. Returns (final_host, final_port, reply_bytes)
after following <=3 redirects. Raises TooManyRedirectsError / NoPrimaryError."""
class TooManyRedirectsError(XceAdminError): hops: int
class NoPrimaryError(XceAdminError): host: str; port: int
xceadmin.cli — Implementation Note
Thin click layer: @click.group() cli() → @cli.group() config() → @config.command() build(...)/push(...).
Shared exception contract (Design Review correction — an earlier draft had 7 unrelated exception types with no unifying dispatch mechanism): xcecommon.errors defines one base, class XceAdminError(Exception): message: str, and every raising exception across xcecommon/xceadmin (SchemaNotFoundError, ValidationError, ReservedNameError, DuplicateEntryError, ConnectionTimeoutError, ReplyDecodeError, TooManyRedirectsError, NoPrimaryError) subclasses it. xceadmin.cli has exactly one except XceAdminError as e: raise click.ClickException(e.message) — a single dispatch surface, per-type message formatting happens in each exception's __str__/message construction at raise time, not in the CLI layer (7.1, 7.2, 7.3, 7.4). A SIGINT handler closes any open socket before re-raising as a "cancelled" message (7.5). No business logic lives here — covered by the summary row.
Data Models
Domain Model
No persistent entities or aggregates — xceadmin is a stateless, one-shot transform (YAML → bytes) and a stateless one-shot RPC client (bytes → controller → result). The only domain concept is the ClusterConfig field set, whose provenance (which YAML file owns which field) is the core invariant this design enforces.
Data Contracts & Integration
YAML section → ClusterConfig field mapping (all 13 fields, controller.capnp:694-708):
capnp field (@N) |
YAML source | Kind | Req |
|---|---|---|---|
clusterName (@0), heartbeatInterval (@1), inactiveHeartbeatCount (@2), fetchJobInterval (@3), netioTimeout (@4) |
xce.yaml (flat keys cluster_name, heartbeat_interval, inactive_heartbeat_count, fetch_job_interval, netio_timeout) |
scalar | 1.11 |
resources (@5) |
resources.yaml → resources[] |
list | 1.1 |
queues (@6) |
queues.yaml → queues[] |
list | 1.1 |
ugroups (@7) |
users.yaml → usergroups[] |
list | 1.1 |
hgroups (@8) |
hosts.yaml → hostgroups[] |
list | 1.1 |
hostConfigs (@9) |
hosts.yaml → hosts[] |
list | 1.1 |
hproviders (@10) |
scaler.yaml → scalers[] |
list | 1.12 |
limits (@11) |
limits.yaml → limits[] |
list | 1.1 |
configGeneration (@12) |
none — controller-assigned | excluded | 1.9 |
hosts.yaml is the one file feeding two capnp fields (hgroups + hostConfigs) — xcecommon.mapping's SECTION_MAP has two entries pointing at the same yaml_file with different yaml_key/capnp_field pairs; this is the only many-to-one case in the table and is called out explicitly here so it is not mistaken for a bug during implementation.
Per-field YAML-key-to-capnp-field-name renames (e.g., groupname→GroupConfig.name, exclude_members→exmembers) are enumerated in research.md (too granular for this table) and enforced by xcecommon.mapping's per-section spec, not restated here.
Logical Data Model
cluster.bin has no indexing/sharding concerns (single-file capnp blob, read once at controller startup/config-push time). Byte-for-byte determinism (9.1, 9.2, 9.3) is the only "consistency" property that matters, guaranteed by: (a) SECTION_MAP iterating YAML list order verbatim, never re-sorting; (b) no wall-clock/host/user value ever entering a mapped field; (c) capnp's own canonical single-segment encoding for a message built once and never mutated after .to_bytes().
Error Handling
Error Strategy
Every xcecommon/xceadmin internal exception is caught once, at the xceadmin.cli boundary, and re-raised as click.ClickException(message) — giving a single place that guarantees "clear, actionable error, non-zero exit" (7.1, 7.2, 7.3, 7.4) without scattering sys.exit() calls through the library layer (library code raises; only the CLI layer exits).
Error Categories and Responses
| Category | Trigger | Response | Req |
|---|---|---|---|
| Connection | TLS connect fails | "cannot connect to <host:port>: <reason>", exit non-zero |
7.1 |
| Protocol | Reply.status != 0 |
status code + reply error text, exit non-zero | 7.2 |
| Decode | corrupt/truncated capnp bytes | "invalid cluster.bin: <reason>", exit non-zero |
7.3 |
| Environment | schema file missing at expected path | missing path + installation hint | 7.4 |
| Interrupt | SIGINT during push |
close TLS socket, "cancelled", graceful exit |
7.5 |
| Timeout | any network op > 30s | "timeout connecting to <host:port>" |
7.6 |
Monitoring
Out of scope — xceadmin is a one-shot CLI with stderr/exit-code as its only observability surface; no logging framework, no metrics endpoint (Simplification — a persistent log/metrics story belongs to a daemon, not a CLI invoked from a shell or a skill).
Testing Strategy
- Unit — schema bundling:
default_schema_path()/default_schema_dir()resolve to a real, non-empty, parseablecontroller.capnp/JSON schema set under whatever install is currently active — trivial in isolation, but only meaningful when it actually runs against a non-editable install; every other test in this list runs fine against an editable install even ifpackage_datapackaging is broken, since editable installs keep resolving into the checked-out source tree regardless. This is what/xce-build xceadmin release(8.2) validates that/xce-build xceadmin(8.1) cannot — a gap discovered post-design-approval (seeresearch.md) that the schema-bundling fix itself would otherwise ship untested. - Unit — mapping:
SECTION_MAPlist-section and scalar-section mappers each produce the expected capnp field values from a minimal in-memory dict (no file I/O) — one test perkind, plus thehosts.yaml-feeds-two-fields case explicitly. - Unit — wire framing:
send_frame/recv_frameround-trip against a fixed byte sequence, and against a loopbacksocket.socketpair()— verifies native-order packing matchesstruct.pack("=Q", ...)semantics without needing a real controller. - Integration — config build (golden-file):
minimal/full/invalidfixture directories (File Structure Plan) —fullnow covers all 7 YAML files including the newly-addedxce.yaml/scaler.yaml; byte-comparesbuild_cluster_config()output against a committedexpected.binand its SHA-256 (1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 1.10, 1.11, 1.12, 9.1, 9.2, 9.3). Plainassert produced == expected_bytesfixture comparison —pytest-regressionswas evaluated (seeresearch.md) but not adopted, since this project's nix-pinned dependency list would need independent verification and the byte-equality need here is simple enough not to warrant it (Simplification). - Integration — validation: one test per
invalid/fixture case — schema violation, reserved name, duplicate name (including a limit/queue name collision, proving 4.3's intentionally-narrow scope), undefined reference (warn, not fail). - Integration — config push (mock controller): a Python TLS server speaking just enough of the wire protocol (frame in, decode
Requestor direct-structProbeConfigTagRequest, encode a cannedReply/ProbeConfigTagReply) to exercisepush_config()(2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 3.1, 3.2, 3.3 redirect, 7.1, 7.2, 7.3) andprobe_config()(6.1, 6.2, 6.3 — accepted and rejected probe replies) against the same mock, since both sharexcecommon.wire's framing/send-receive primitives. This is the only realistic way to test config push today — not merely "preferred for the dev loop" asbrief.mdoriginally framed it, but load-bearing: the controller-side dispatch-gate defect found during discovery (Boundary Commitments) means a real 3-node cluster currently rejectsOP_upgradeConfigfrom every role, so live-cluster E2E (brief.md's "Option B") is blocked, not merely deferred, until that controller defect is fixed (Revalidation Trigger). - E2E (deferred): real-cluster
config pushround-trip viaxce-test-ha— tracked inbrief.md, explicitly gated on both the controller dispatch fix andconfig-upgrade-snapshotlanding.
Security Considerations
- Transport is encryption-only, not authenticated — no peer-certificate validation exists anywhere in XCE's controller-to-controller TLS today (
SSL_VERIFY_NONE, noX509_check_host);xceadminmatches this exactly rather than silently adding (and thereby implying) a stronger guarantee the rest of the system doesn't have. An attacker with network access to the controller port can MITM either direction. This is a pre-existing, project-wide posture (product.md— "trust OS boundary"), not something this spec introduces or should unilaterally fix. - Input validation at the boundary — every YAML file is JSON-Schema-validated before any byte of it reaches capnp encoding (4.1), consistent with
security.md's "validate at the API boundary, never trust external data." - No secrets in xceadmin — no JWT, no RBAC, no credential storage; trust model is "same TLS cert bundle as the controller cluster," matching
xce_test's existing zero-config posture. Nothing new to rotate or leak. - Reply decoding must not crash on malformed input —
xcecommon.wire.send_request'sReply.from_bytes()call is wrapped to convert a pycapnp decode exception into7.3's "invalid" error path rather than an unhandled traceback, even though the reply source (a controller within the operator's own cluster) is semi-trusted.
Supporting References
- Full per-field YAML-key ↔ capnp-field-name rename table, pycapnp API usage notes, and the two controller-side defect write-ups:
research.md. - Nix package list, harness hook JSON (ruff format/check, YAML schema guard), and settings allowlist:
brief.mdHarness Enhancement section (unchanged, referenced not restated here).