deploy-ha.sh no longer takes a <data-dir> argument or --build-config
flag. Every deploy now generates a fresh self-signed TLS cert/key
(openssl, RSA 2048, 10y validity) directly into the deploy dir's
certs/, and builds cluster.bin directly into controller/ctrl-1/conf/
via xceadmin -- no intermediate staging directory, nothing preserved
across runs. run-ha-soak.sh (/xce-test-ha) updated to match: dropped
its own XCE_HA_DATADIR-based precondition entirely. Updated 4 callers
(run-ha-soak.sh, test-heartbeat-tolerance.sh, test-commit-durability.sh,
both xce-deploy-ha/xce-test-ha SKILL.md docs).
Separately: xcecommon.wire.connect() was loading a client-side TLS
cert, diverging from the real reference client -- netio.c's
create_ssl_context() presents no certificate at all when acting as a
client (cert_file/key_file both NULL, bare TLS_client_method()).
Fixed connect() to match, and since nothing else consumed cert_dir
(verify_mode=CERT_NONE throughout, no CA check either), removed it
end-to-end: discovery.follow_redirects(), push.push_config()/
probe_config(), and the --cert-dir CLI flag.
Server-side cert loading (mock TLS test servers) is unaffected --
moved into a shared session-scoped tls_certs fixture (new
tests/conftest.py, openssl-generated) replacing 3 duplicated
REPO_HA_DATA_DIR/skipif blocks. Net result: those tests no longer skip
under a hermetic nix build sandbox (which never had ha/data/) --
verified 165/165 passed, 0 skipped, via both meson compile and a full
nix build --option sandbox false .#xcecore_debug rebuild (previously
128 passed / 37 skipped there).
ha/data/{xce.crt,xce.key} removed from git tracking. Requirements
2.3/2.4 and 8.3/8.4/8.5 marked superseded in requirements.md with
dated correction notes.
28 KiB
Implementation Plan
-
1. Foundation: package scaffolding, shared errors, build/toolchain integration
-
1.1 Set up the xceadmin Python package skeleton
- Editable install registers a real
xceadminconsole-script entry point backed by an empty, working click group (no subcommands yet —config build/config pushare added by task 4.1, not this task) xcecommonis importable as a standalone package independent ofxceadmin's CLI- Observable: running the entry point after an editable install prints usage/help text without error, showing zero commands
- Requirements: 5.4
- Editable install registers a real
-
1.2 Establish a shared error-handling foundation
- One common base exception type carries a human-readable message
- Every specific failure mode (schema not found, validation failure, reserved name, duplicate entry, connection timeout, corrupt reply, too many redirects, no primary available) has its own subclass
- Observable: raising any of these subclasses and catching the common base type recovers the message
- Requirements: 7.1, 7.2, 7.3, 7.4
-
1.3 Wire xceadmin into the project's build and dev-shell tooling
- Nix dev shell provides the Python interpreter and all dependencies needed to build and test xceadmin
- Running the build command for xceadmin performs an editable install by default and a pinned non-editable install under the release profile, running the test suite after whichever install just happened
- The build step keeps the packaged copies of the wire-protocol schema and the validation schemas in sync with their source locations elsewhere in the repo, and packages both correctly under either install mode
- Build/test command documentation, permission allowlist entries, and automatic formatting/linting on Python file edits all exist for the new component
- Observable: running the build command for this component from a clean checkout succeeds and reports test pass/fail
- Requirements: 5.5, 8.1, 8.2, 8.6
-
2. Core: xcecommon shared library
-
2.1 (P) Validate operator-authored YAML against the bundled schemas
- Each of the seven config file types can be checked against its own bundled validation schema, independent of anything the operator supplies
- Reserved queue/host names are rejected even when otherwise schema-valid
- Duplicate names within a single file/section are rejected
- Cross-references to undefined groups or resources are flagged as warnings without blocking the build
- Observable: validating a config directory with no schema files present anywhere in it still succeeds
- Requirements: 4.1, 4.2, 4.3, 4.4, 4.5, 5.5
- Boundary: xcecommon.validation
-
2.2 (P) Load the cluster wire-protocol schema
- The wire-protocol schema loads once per process and exposes every message type xceadmin needs to build or decode
- Loading works identically whether xceadmin was installed in editable or pinned mode
- A missing or corrupted schema copy produces a clear, actionable error rather than a crash
- Observable: loading the schema under a pinned (non-editable) install still resolves the bundled copy correctly
- Requirements: 5.1, 5.5, 7.4
- Boundary: xcecommon.schema
-
2.3 (P) Map validated YAML into cluster configuration fields
- Every field the cluster configuration binary format defines is either populated from validated YAML or deliberately left untouched, with no field silently dropped
- List-based sections preserve the exact order they appeared in the YAML
- No timestamp, hostname, or username value ever enters the mapped output
- Building the same input twice produces identical output
- Requirements: 1.1, 1.7, 1.8, 1.9, 1.10, 1.11, 1.12, 5.3, 9.1, 9.2, 9.3
- Boundary: xcecommon.mapping
- Depends: 2.2
-
2.4 (P) Establish the TLS connection and wire framing to the controller
- A single request can be sent and a single reply received over a TLS connection using the cluster's existing certificate trust model
- Message framing matches the controller's own byte-for-byte, verified against known-good byte sequences
- The connection always closes cleanly, including on error or interruption
- Requirements: 2.1, 2.3, 2.4, 5.2, 6.1, 7.1, 7.3, 7.6
- Boundary: xcecommon.wire
- Depends: 2.2
-
3. Core: xceadmin CLI orchestration
-
3.1 (P) Compile a configs directory into a deployable cluster binary
- Running the build command against a directory of YAML files produces a binary output file with a printed checksum
- A missing individual YAML file is treated as an empty section, not an error
- Unrecognized-but-valid YAML fields produce a warning, not a failure
- Output can be directed to a specific path or to standard output
- Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.9, 8.3, 8.4
- Boundary: xceadmin.build
- Depends: 2.1, 2.3
-
3.2 (P) Follow controller redirects to find the primary
- A reply pointing at a different host/port is followed automatically, up to a bounded number of hops
- Exceeding the hop limit or waiting past the connection timeout produces a clear error naming the last host contacted
- Observable: a chain of up to three redirects resolves to a final destination without operator intervention
- Requirements: 3.1, 3.2, 3.3, 3.4
- Boundary: xceadmin.discovery
- Depends: 2.4
-
3.3 Push a cluster binary to the controller, with a dry-run mode
- A real push reports the assigned configuration version, acknowledgement count, and any error, exiting non-zero on failure
- A dry-run reports whether the controller would accept the config without ever committing it
- Every connection failure, protocol-level rejection, corrupt input, or missing schema maps to a specific, distinct error
- Requirements: 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 6.1, 6.2, 6.3, 7.1, 7.2, 7.3, 7.4, 7.5, 7.6
- Boundary: xceadmin.push
- Depends: 2.4, 3.2
-
4. Integration: wire the CLI entry point and hook xceadmin into cluster deployment
-
4.1 Expose config build and config push as a single command-line tool
xceadmin config buildandxceadmin config push(with--dry-run) are both reachable from one installed command- Any failure surfaces as a specific, human-readable message with a non-zero exit code, not a raw traceback
- Interrupting a push mid-flight leaves no dangling connection and reports a clean cancellation
- Requirements: 1.2, 1.3, 2.2, 2.4, 7.1, 7.2, 7.3, 7.4, 7.5, 7.6, 8.1, 8.2
- Boundary: xceadmin.cli
- Depends: 3.1, 3.3
-
4.2 Build cluster configuration automatically during HA deployment
- Deploying a cluster with the build-config option produces a fresh binary from the current YAML before startup
- Deploying with no binary present and no flag still produces one automatically
- Deploying with an existing binary and no flag leaves it untouched, matching prior behavior
- Requirements: 8.3, 8.4, 8.5
- Boundary: deploy-ha.sh
- Depends: 4.1
-
5. Validation: test coverage across build, push, and packaging
-
5.1 (P) Unit test the field-mapping logic in isolation
- Both mapper shapes (list sections and the scalar section) are verified against in-memory input with no file I/O
- The one file that feeds two separate output fields is verified explicitly
- Requirements: 1.1, 1.7, 1.8, 1.9, 1.10, 1.11, 1.12, 5.3, 9.1, 9.2, 9.3
- Boundary: xcecommon.mapping
- Depends: 2.3
-
5.2 (P) Unit test wire framing
- Framing round-trips correctly against a fixed, known byte sequence and against a local loopback connection
- Requirements: 2.1
- Boundary: xcecommon.wire
- Depends: 2.4
-
5.3 (P) Verify bundled schemas resolve under every install mode
- Both the wire-protocol schema and the validation schemas resolve to real, parseable files under the currently active install
- A deliberately corrupted or absent packaged schema produces the expected installation-hint error rather than a crash
- Meaningful confirmation of the pinned install path requires running this under the release build profile
- Requirements: 5.5, 7.4
- Boundary: xcecommon.schema, xcecommon.validation
- Depends: 2.1, 2.2, 1.3
-
5.4 (P) Golden-file test config build determinism
- Building from a minimal fixture, a full fixture covering all seven file types, and a set of intentionally invalid fixtures each produce the expected result
- Building the same fixture twice produces byte-identical output and an identical checksum
- Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 1.10, 1.11, 1.12, 9.1, 9.2, 9.3
- Boundary: xceadmin.build
- Depends: 3.1
-
5.5 (P) Test every YAML validation failure mode
- Each invalid-fixture case (schema violation, reserved name, duplicate name including a cross-type collision, undefined reference) produces the documented outcome — hard failure or warning, matching the requirement
- Requirements: 4.1, 4.2, 4.3, 4.4, 4.5
- Boundary: xcecommon.validation
- Depends: 2.1
-
5.6 (P) Test config push and dry-run against a mock controller
- A mock TLS server standing in for the controller confirms the full push flow: successful push, redirect-following, and every documented error path
- The same mock confirms dry-run reports both an accepted and a rejected outcome correctly
- Requirements: 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 3.1, 3.2, 3.3, 3.4, 6.1, 6.2, 6.3, 7.1, 7.2, 7.3
- Boundary: xceadmin.push, xceadmin.discovery
- Depends: 3.3
Implementation Notes
-
1.3: nix's bare
python3does not putpipon PATH —python3Packages.piphad to be added tonativeBuildInputsalongside the packages design.md's Technology Stack table lists; not a spec gap, just an operational dependency the table didn't need to spell out. -
1.3: Claude Code
PostToolUsehooks receive tool input via stdin JSON (tool_input.file_path), not a$FILE_PATHenv var — brief.md's ruff-hook sketch used the env-var form, which would have silently no-op'd. The actual hooks added to.claude/settings.jsonfollow this repo's existing.wolf/hooks/post-write.jsstdin-JSON convention instead. Any future hook work in this repo should follow the same pattern, not brief.md's original sketch. -
1.3:
src/xceadmin/src/xcecommon/schema_files/(synced capnp + JSON schema copies) is a build artifact, regenerated everyxce-build.sh xceadminrun — never hand-edit it, never commit it (src/xceadmin/.gitignoreexcludes it). -
2.1:
check_undefined_refsdeliberately does not parse resource names out ofres_req/gpu_reqfree-form LSF-style expression strings (e.g."mem>1000 rusage[mem=1000]") — design.md'sxcecommon.validationcomponent scopes this check to queue→host-group and limit→group/resnameonly.limits.yaml'sprojectsfield andscaler.yamlname collisions are likewise out of scope — noprojects.yamlcatalog exists among the 7 config types, and Requirement 4.3 only enumerates queue/host/resource/group/limit names.src/configs/hosts.yamlhas 2 pre-existing undefined refs (resourcemg, hosth3) that now surface as warnings — expected, not a regression; fixing the example fixture is outside this task's boundary. -
2.2: The real
controller.capnpnames its reply structRequestReply, notReplyas design.md's postcondition states — an inexact design pseudo-type, corrected empirically (verified directly againstsrc/schema/controller.capnp, not assumed). pycapnp 2.0.0/capnproto 1.2.0 abort the whole process (uncaught C++ exception, not a Python exception) on a missing cross-file import or malformed.capnpcontent passed tocapnp.load()— a baretry/exceptcannot catch this.xcecommon.schema.load_schema()works around it by pre-validating in an isolated subprocess before ever callingcapnp.load()in the caller's own process, converting any abort into a cleanSchemaNotFoundError. Any future code that callscapnp.load()directly (bypassing this loader) reintroduces the crash risk — always go throughxcecommon.schema.load_schema(). -
2.3:
map_all_sections()(aSECTION_MAP-driven orchestrator, not explicitly named in design.md's Service Interface but consistent with its Contracts) is the single callxceadmin.build(task 3.1) needs — it treats a missing YAML file identically to a present-but-empty section (byte-for-byte), satisfying Requirement 1.5 structurally.target.init(field, 0)and never touching an empty/absent list field produce different serialized bytes for the same logical "nothing here" value (verified empirically) — the mapper always picks "never touch" for byte-determinism (9.1). Calling.to_bytes()twice on the same capnp builder triggers a spurious "message already written"UserWarningeven when no field was mutated between calls — capture the bytes once and reuse them rather than calling.to_bytes()again. -
2.4:
errors.py's exception subclasses (task 1.2) all takemessageas an explicit first constructor argument in addition to their structured fields (e.g.ConnectionTimeoutError(message, host, port),ReplyDecodeError(message, reason)) — design.md's abbreviatedService Interfacetype stubs (class ConnectionTimeoutError(XceAdminError): host: str; port: int) read as if only the structured fields were needed; always check the real constructor inerrors.py, not the design doc's shorthand.xcecommon.wireis deliberately opcode-agnostic —send_request/send_frame/recv_framemove rawbytesonly and never import a capnp type; callers own.to_bytes()/.from_bytes(). Reusedha/data/xce.crt/xce.key(the repo's existing dev cert pair, same onedeploy-ha.shalready copies for controller TLS) as the test fixture for both the TLS test server and the clientcert_dir— no synthetic cert generation needed. -
3.1: Requirement 1.7's "unrecognized-but-schema-valid field" warning is real and fires on essentially every real build using
src/configs/resources.yaml—resources.yaml'stypekey is required byresources.schema.json(required: [name, type]) but has noClusterConfigdestination (perxcecommon.mapping's documented decision not to guess thevaltype/ordtype/flagsinteger encoding), so every valid resource entry triggers a "field 'type' ... ignored" warning unconditionally, not just for unusual input. This is a disclosed spec tradeoff, not a bug — flagged here so a future pass overxceadmin.build's warning UX (e.g. only warning once per field-name per run instead of once per item) knows the noise is expected, not a regression.xceadmin.build's unmapped-field table (_UNMAPPED_BUT_VALID_ITEM_KEYS) was derived by cross-referencingxcecommon.mapping's per-item builder functions against the realsrc/configs/schema/*.schema.jsonproperty lists directly (not from prose alone) to avoid drift. -
3.2: design.md's Dependencies list for
xceadmin.discoverynames onlyxcecommon.wire, but decoding a reply to inspectRequestReply.redirectHost/.redirectPortgenuinely requiresxcecommon.schematoo (verified real field names againstcontroller.capnp:594-595, which matched design.md exactly this time — unlike task 2.2'sReply/RequestReplystruct-name slip). Treated as a design-doc omission, not license to hand-roll capnp decoding.follow_redirects'scert_dirparameter is accepted for signature parity with design.md but unused internally — all I/O, including TLS setup, is delegated to the caller'ssend_fnclosure, which is expected to already carry its owncert_dir. A connection timeout to any hop (including a non-initial one reached via redirect) raisesNoPrimaryErrornaming that specific hop, not the original target. -
bug-1461 (out-of-band, between 3.2 and 3.3):
controller.capnphad a real, confirmed schema bug — a locally-declaredstruct Data(anOPCode_getquery-result tagged union) shadowed Cap'n Proto's builtinDatablob type for every later: Datafield in the file, miscompilingUpgradeConfigRequest.sha256/.bytes(and 5 other fields acrossReplConfigReply/ReplConfigPushRequest/ReplEnvelope/SnapshotTransfer/WalCatchupEntry/CtrlHelloRequest, several HA-critical) as a struct-pointer type instead of a byte blob — blockingpush_config()from constructing a valid request via pycapnp. Fixed at the root (renamed the local struct toXceData,$C.mapname("xce_data_t")unchanged so the C typedef is stable) across 20 files spanning controller/lib/scheduler/Go — see commite53c086and.wolf/buglog.jsonbug-1461 for the full investigation and fix. This was NOT an admin-config-tool-boundary change; it's a pre-existing, cross-component schema defect that happened to block this task.push_config()'s real controller counterpart remains untested live — a separate, already-documented dispatch-gate defect (ha_dispatch_gate()/cluster.c, spec.jsonblocked_on) still blocks that independently of bug-1461. -
3.3:
probe_config()does NOT go throughxceadmin.discovery.follow_redirects—ProbeConfigTagReplyis a direct struct with onlyconfigTag, noredirectHost/.Portfields exist on it at all, so there is no redirect signal forfollow_redirects(which unconditionally decodes replies asRequestReply) to read; onlypush_config()'sRequest/RequestReply-wrapped exchange supports redirects.ProbeConfigTagReplyalso carries no accept/reject signal —DryRunResult.acceptedis a client-side inference (true on any successful round-trip), matching the real C reference client (config_repl.c'sxce_probe_config_tag, which also always sends an emptynameand returns only a bare tag). pycapnp union field assignment for Void/Text-typed variants (e.g.reply.data.nodata = None,reply.data.debugmsg = "...") must use direct assignment, not.init(name)—.init()without a size is only valid for struct/list/data-typed variants, confirmed viaKjException: init() without a size is only valid for struct and object fields. -
4.1:
xcecommon.wire.connect()deliberately letsConnectionRefusedError/other bareOSErrors propagate unwrapped (documented inwire.py's own module docstring) —xceadmin.cliis the layer that catchesOSErrorand formats it per Requirement 7.1, alongside itsexcept XceAdminErrordispatch surface; a task 4.1 implementer must not assumeXceAdminErroralone covers every network failure mode. click 8.3.1'sCliRunnerno longer acceptsmix_stderr=;result.stdout/.stderrare separated by default, butCliRunner's output capture decodes as text and is lossy for arbitrary binary (confirmed:bytes(range(256))does not round-trip through the captured.outputstring) — do not assert exact byte content of a-o -binary write throughCliRunner; that's already covered at thewrite_output()unit level (task 3.1'stest_config_build.py). A CLI test that omits-ofalls through to the real default (cluster.bin, relative to CWD) — always pass an explicit-o <tmp_path>in tests that reach the write step, or the test will write a stray artifact into the actual repo working directory (caught viagit statusbefore commit, not by the test itself). -
4.2:
deploy-ha.shresolves bothxceadmin($SCRIPT_DIR/../xceadmin/.venv/bin/xceadmin) and the YAML configs source ($SCRIPT_DIR/../configs) relative to the script's own location ($(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)), not the caller's CWD — matches the project's other relative-path scripts and keeps the deploy working regardless of where it's invoked from.--build-configis parsed out of"$@"before positional-arg assignment so it can appear anywhere in the argument list, not just a fixed position. Verified live against the realxceadminbuild andsrc/configs/: missingcluster.binauto-builds (8.4), an existing one with no flag is left byte-for-byte untouched — same mtime, same SHA (8.5), and--build-configagainst an existingcluster.binforces a rebuild that reproduces byte-identical content (8.3, confirming 9.1's determinism guarantee holds through this path too). -
5.1, 5.2: Already fully satisfied by task 2.3's
test_mapping.pyand task 2.4'stest_wire.pyrespectively — same requirement lists, same acceptance bullets (in-memory-only mapper coverage + the explicit two-field case for 5.1; a fixed known byte sequence + real loopback round-trips for 5.2). No new tests written; would have been pure duplication. -
5.3: Found and fixed a real gap while writing
test_schema_bundling.py—xcecommon.validation.validate_yaml()'sjson.loads(schema_path.read_text())call (loading the bundled JSON validation schema itself, not the operator's YAML) had no error handling, so a corrupted bundled JSON schema crashed with a rawjson.JSONDecodeErrorinstead of a cleanValidationError— the same "crash vs. clean error" class of bug task 2.2 already fixed forcontroller.capnp, just on the JSON-schema side, and not caught by task 2.1 since that task's tests only exercised operator-YAML failure modes, never a corrupted schema file. Fixed by wrapping it and raisingValidationError(..., constraint="schema-corrupted"), matching the existing missing-schema handling's pattern one function above it. Gotcha for any future task touchingxcecommon/xceadminsource: editing a source file does NOT propagate to a running.venvuntil reinstalled — always re-runxce-build.sh xceadminafter a source edit before trusting a bare.venv/bin/python -m pytestresult; a stale venv (especially one left inrelease/pinned mode by a prior profile run) will silently test the old code and produce a false failure (or false pass) that looks like a real bug. -
5.4:
tests/fixtures/{minimal,full}/configs/*.yaml+ checked-inexpected.bingolden files added totest_config_build.py(not a new file — design.md's File Structure Plan lists the SAME requirement set fortest_config_build.pyunder both 3.1 and 5.4).full/configs/*.yamlis a byte-for-byte copy ofsrc/configs/*.yaml; its golden SHA (860cfd4f59ef...) independently matches the tagdeploy-ha.shproduced in task 4.2's live verification against the same source files — an unplanned but welcome additional confirmation of Requirement 9's cross-context determinism guarantee. Golden-file tests catch representation drift (bytes change even though decoded content is still semantically correct) that purely structural/decode-based assertions elsewhere in the suite cannot detect; regenerateexpected.bindeliberately (viabuild_cluster_config(), sanity-checked by decoding it back) whenever a real, intentional field-mapping change lands — never hand-edit the binary. -
5.5: Already fully satisfied by task 2.1's
test_validation.py— every sub-case (schema violation, reserved name, cross-type duplicate collision viatest_check_duplicates_allows_limit_name_to_collide_with_queue_name, undefined reference) already has an explicit test with the exact same requirement list. No new tests written. -
5.6:
probe_config()'s dry-run has no reachableaccepted=Falsepath —ProbeConfigTagReplygenuinely carries no rejection signal on the wire (see task 3.3's note). 5.6's bullet asking for both an accepted and a rejected outcome cannot be satisfied without inventing undocumented client-side rejection logic; instead addedtest_probe_config_has_no_wire_level_rejection_signal_by_design(parametrized over severalconfigTagvalues) that locks in and documents the limitation explicitly rather than leaving it silently uncovered. Added two new end-to-end propagation tests throughpush_config's real composition withxceadmin.discovery(not just discovery.py's own unit tests): a real 4-mock-server redirect chain provingTooManyRedirectsErrorsurfaces correctly, and aNoPrimaryError-on-timeout test that monkeypatchesxcecommon.wire.connectto raiseConnectionTimeoutErrorimmediately rather than waiting out a real 30s socket timeout. -
Post-implementation, out-of-band (2026-07-22): reopened and reversed the 2026-07-21 "not wired into meson" Design Review correction — see
design.md's File Structure Plan Amendment for full rationale and verification. Summary: addedsrc/xceadmin/meson.build(custom_target(), stamp-file output,build_always_stale: true) +src/xceadmin/meson-build.sh(out-of-tree install+test,repo-mirror/symlink staging sosrc/configs/,src/schema/,ha/data/relative-path test assumptions still resolve from a copied location);src/meson.buildgainedsubdir('xceadmin');src/tools/xce-build.sh'sxceadmin)case now delegates tomeson compile -C "$BUILD_DIR" xceadmininstead of its own pip/venv logic. Verified bothmeson compile(165/165 passed against a real dev checkout with certs present) andnix build --option sandbox false .#xcecore_debug(128 passed / 37 skipped — cert-dependent tests correctly skip under nix's git-tracked-only hermetic source, sinceha/data/xce.crt/.keyare intentionally not committed). AppImage bundling (README 3.5) explicitly deferred, not part of this change. Net effect on Requirement 8: the editable-vs-pinned distinction (8.1 vs. 8.2) collapses — everyxce-build.sh xceadminrun is now the pinned/meson-driven path, a strictly stronger guarantee than the old default. -
Post-implementation, out-of-band (2026-07-22, second pass):
meson-build.sh/meson-install.shno longer resolvepython3via a bare PATH lookup —meson.buildnow pins it once viaimport('python').find_installation('python3')(same mechanism asfind_program('go')/capnp/packcc elsewhere in this build, visible atmeson setuptime asProgram python3 found: YES (/nix/store/...)) and threads the resolved path into both scripts explicitly. Verified:meson setupprints the pinned interpreter, rebuild stays 165/165 green, and the built venv's ownpython3resolves through to that exact store path. -
Post-implementation, out-of-band (2026-07-22, third pass — removes
ha/data/entirely):deploy-ha.shno longer takes a<data-dir>argument at all (--build-configflag also removed) — every invocation now generates a fresh self-signed TLS cert/key (openssl req -x509 -newkey rsa:2048 -nodes -days 3650, 10y validity) directly into the deploy dir'scerts/, and buildscluster.bindirectly intocontroller/ctrl-1/conf/viaxceadmin, with no intermediate staging directory.run-ha-soak.sh(/xce-test-ha) updated to match: dropped its ownXCE_HA_DATADIR-based precondition entirely, sincedeploy-ha.shnow handles both cert and config generation itself. Updated 4 callers (run-ha-soak.sh,test-heartbeat-tolerance.sh,test-commit-durability.sh, plus bothxce-deploy-ha/xce-test-haSKILL.md docs) to the new 3-arg signature.ha/data/{xce.crt,cluster.bin}removed from git tracking (cluster.binwas already gitignored from the prior pass; the certs were a real checked-in TLS private key, now gone from history going forward).Investigating this surfaced a real, independent finding:
xcecommon.wire.connect()was loading a client-side TLS cert (context.load_cert_chain(...)on thePROTOCOL_TLS_CLIENTcontext) — verified against the actual reference implementation (netio.c'screate_ssl_context()) that the real client presents no certificate at all whencert_file/key_fileareNULL(bareTLS_client_method(), noSSL_CTX_use_certificate_chain_file/SSL_CTX_use_PrivateKey_filecall).xceadmin's client diverged from the reference client it's meant to mirror. Fixed:connect()no longer loads a client cert or takes acert_dirparameter at all. Since nothing else consumedcert_dir(no CA verification happens either —verify_mode=CERT_NONEthroughout), removed it end-to-end:xceadmin.discovery.follow_redirects()'scert_dirparam (already dead/noqa'd — signature-parity leftover),xceadmin.push.push_config()/probe_config(), and the--cert-dirCLI flag onconfig push. Server-side cert loading (the mock TLS servers intest_wire.py/test_cli.py/test_config_push_mock.py) is unaffected and still needed — moved into a shared session-scopedtls_certsfixture in a newtests/conftest.py(openssl-generated, same approach asdeploy-ha.sh), replacing 3 duplicated module-levelREPO_HA_DATA_DIR/pytest.mark.skipif(not ha/data/xce.crt...)blocks. Net result: those 3 files' TLS-dependent tests no longer skip under a hermeticnix buildsandbox (which never hadha/data/in its git-tracked-only source) — verified 165/165 passed, 0 skipped, both viameson compileand via a fullnix build --option sandbox false .#xcecore_debugrebuild (previously 128 passed / 37 skipped there). Requirements 2.3/2.4 and 8.3/8.4/8.5 marked superseded inrequirements.mdwith dated correction notes rather than silently rewritten.