Rongsong Shen
77ee831e7a
TLS connection + 8-byte native-order length-prefixed framing to the controller, verified byte-for-byte against request.c's own framing and myapp.c's client TLS posture (SSL_VERIFY_NONE, no CN check). Opcode-agnostic: moves raw bytes only, so the same send_request() serves both OP_upgradeConfig and ProbeConfigTag's direct-struct variant. Connection always closes on error, including handshake timeout.
152 lines
14 KiB
Markdown
152 lines
14 KiB
Markdown
# Implementation Plan
|
|
|
|
- [ ] 1. Foundation: package scaffolding, shared errors, build/toolchain integration
|
|
- [x] 1.1 Set up the xceadmin Python package skeleton
|
|
- Editable install registers a real `xceadmin` console-script entry point backed by an empty, working click group (no subcommands yet — `config build`/`config push` are added by task 4.1, not this task)
|
|
- `xcecommon` is importable as a standalone package independent of `xceadmin`'s CLI
|
|
- Observable: running the entry point after an editable install prints usage/help text without error, showing zero commands
|
|
- _Requirements: 5.4_
|
|
|
|
- [x] 1.2 Establish a shared error-handling foundation
|
|
- One common base exception type carries a human-readable message
|
|
- Every specific failure mode (schema not found, validation failure, reserved name, duplicate entry, connection timeout, corrupt reply, too many redirects, no primary available) has its own subclass
|
|
- Observable: raising any of these subclasses and catching the common base type recovers the message
|
|
- _Requirements: 7.1, 7.2, 7.3, 7.4_
|
|
|
|
- [x] 1.3 Wire xceadmin into the project's build and dev-shell tooling
|
|
- Nix dev shell provides the Python interpreter and all dependencies needed to build and test xceadmin
|
|
- Running the build command for xceadmin performs an editable install by default and a pinned non-editable install under the release profile, running the test suite after whichever install just happened
|
|
- The build step keeps the packaged copies of the wire-protocol schema and the validation schemas in sync with their source locations elsewhere in the repo, and packages both correctly under either install mode
|
|
- Build/test command documentation, permission allowlist entries, and automatic formatting/linting on Python file edits all exist for the new component
|
|
- Observable: running the build command for this component from a clean checkout succeeds and reports test pass/fail
|
|
- _Requirements: 5.5, 8.1, 8.2, 8.6_
|
|
|
|
- [ ] 2. Core: xcecommon shared library
|
|
- [x] 2.1 (P) Validate operator-authored YAML against the bundled schemas
|
|
- Each of the seven config file types can be checked against its own bundled validation schema, independent of anything the operator supplies
|
|
- Reserved queue/host names are rejected even when otherwise schema-valid
|
|
- Duplicate names within a single file/section are rejected
|
|
- Cross-references to undefined groups or resources are flagged as warnings without blocking the build
|
|
- Observable: validating a config directory with no schema files present anywhere in it still succeeds
|
|
- _Requirements: 4.1, 4.2, 4.3, 4.4, 4.5, 5.5_
|
|
- _Boundary: xcecommon.validation_
|
|
|
|
- [x] 2.2 (P) Load the cluster wire-protocol schema
|
|
- The wire-protocol schema loads once per process and exposes every message type xceadmin needs to build or decode
|
|
- Loading works identically whether xceadmin was installed in editable or pinned mode
|
|
- A missing or corrupted schema copy produces a clear, actionable error rather than a crash
|
|
- Observable: loading the schema under a pinned (non-editable) install still resolves the bundled copy correctly
|
|
- _Requirements: 5.1, 5.5, 7.4_
|
|
- _Boundary: xcecommon.schema_
|
|
|
|
- [x] 2.3 (P) Map validated YAML into cluster configuration fields
|
|
- Every field the cluster configuration binary format defines is either populated from validated YAML or deliberately left untouched, with no field silently dropped
|
|
- List-based sections preserve the exact order they appeared in the YAML
|
|
- No timestamp, hostname, or username value ever enters the mapped output
|
|
- Building the same input twice produces identical output
|
|
- _Requirements: 1.1, 1.7, 1.8, 1.9, 1.10, 1.11, 1.12, 5.3, 9.1, 9.2, 9.3_
|
|
- _Boundary: xcecommon.mapping_
|
|
- _Depends: 2.2_
|
|
|
|
- [x] 2.4 (P) Establish the TLS connection and wire framing to the controller
|
|
- A single request can be sent and a single reply received over a TLS connection using the cluster's existing certificate trust model
|
|
- Message framing matches the controller's own byte-for-byte, verified against known-good byte sequences
|
|
- The connection always closes cleanly, including on error or interruption
|
|
- _Requirements: 2.1, 2.3, 2.4, 5.2, 6.1, 7.1, 7.3, 7.6_
|
|
- _Boundary: xcecommon.wire_
|
|
- _Depends: 2.2_
|
|
|
|
- [ ] 3. Core: xceadmin CLI orchestration
|
|
- [ ] 3.1 (P) Compile a configs directory into a deployable cluster binary
|
|
- Running the build command against a directory of YAML files produces a binary output file with a printed checksum
|
|
- A missing individual YAML file is treated as an empty section, not an error
|
|
- Unrecognized-but-valid YAML fields produce a warning, not a failure
|
|
- Output can be directed to a specific path or to standard output
|
|
- _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.9, 8.3, 8.4_
|
|
- _Boundary: xceadmin.build_
|
|
- _Depends: 2.1, 2.3_
|
|
|
|
- [ ] 3.2 (P) Follow controller redirects to find the primary
|
|
- A reply pointing at a different host/port is followed automatically, up to a bounded number of hops
|
|
- Exceeding the hop limit or waiting past the connection timeout produces a clear error naming the last host contacted
|
|
- Observable: a chain of up to three redirects resolves to a final destination without operator intervention
|
|
- _Requirements: 3.1, 3.2, 3.3, 3.4_
|
|
- _Boundary: xceadmin.discovery_
|
|
- _Depends: 2.4_
|
|
|
|
- [ ] 3.3 Push a cluster binary to the controller, with a dry-run mode
|
|
- A real push reports the assigned configuration version, acknowledgement count, and any error, exiting non-zero on failure
|
|
- A dry-run reports whether the controller would accept the config without ever committing it
|
|
- Every connection failure, protocol-level rejection, corrupt input, or missing schema maps to a specific, distinct error
|
|
- _Requirements: 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 6.1, 6.2, 6.3, 7.1, 7.2, 7.3, 7.4, 7.5, 7.6_
|
|
- _Boundary: xceadmin.push_
|
|
- _Depends: 2.4, 3.2_
|
|
|
|
- [ ] 4. Integration: wire the CLI entry point and hook xceadmin into cluster deployment
|
|
- [ ] 4.1 Expose config build and config push as a single command-line tool
|
|
- `xceadmin config build` and `xceadmin config push` (with `--dry-run`) are both reachable from one installed command
|
|
- Any failure surfaces as a specific, human-readable message with a non-zero exit code, not a raw traceback
|
|
- Interrupting a push mid-flight leaves no dangling connection and reports a clean cancellation
|
|
- _Requirements: 1.2, 1.3, 2.2, 2.4, 7.1, 7.2, 7.3, 7.4, 7.5, 7.6, 8.1, 8.2_
|
|
- _Boundary: xceadmin.cli_
|
|
- _Depends: 3.1, 3.3_
|
|
|
|
- [ ] 4.2 Build cluster configuration automatically during HA deployment
|
|
- Deploying a cluster with the build-config option produces a fresh binary from the current YAML before startup
|
|
- Deploying with no binary present and no flag still produces one automatically
|
|
- Deploying with an existing binary and no flag leaves it untouched, matching prior behavior
|
|
- _Requirements: 8.3, 8.4, 8.5_
|
|
- _Boundary: deploy-ha.sh_
|
|
- _Depends: 4.1_
|
|
|
|
- [ ] 5. Validation: test coverage across build, push, and packaging
|
|
- [ ] 5.1 (P) Unit test the field-mapping logic in isolation
|
|
- Both mapper shapes (list sections and the scalar section) are verified against in-memory input with no file I/O
|
|
- The one file that feeds two separate output fields is verified explicitly
|
|
- _Requirements: 1.1, 1.7, 1.8, 1.9, 1.10, 1.11, 1.12, 5.3, 9.1, 9.2, 9.3_
|
|
- _Boundary: xcecommon.mapping_
|
|
- _Depends: 2.3_
|
|
|
|
- [ ] 5.2 (P) Unit test wire framing
|
|
- Framing round-trips correctly against a fixed, known byte sequence and against a local loopback connection
|
|
- _Requirements: 2.1_
|
|
- _Boundary: xcecommon.wire_
|
|
- _Depends: 2.4_
|
|
|
|
- [ ] 5.3 (P) Verify bundled schemas resolve under every install mode
|
|
- Both the wire-protocol schema and the validation schemas resolve to real, parseable files under the currently active install
|
|
- A deliberately corrupted or absent packaged schema produces the expected installation-hint error rather than a crash
|
|
- Meaningful confirmation of the pinned install path requires running this under the release build profile
|
|
- _Requirements: 5.5, 7.4_
|
|
- _Boundary: xcecommon.schema, xcecommon.validation_
|
|
- _Depends: 2.1, 2.2, 1.3_
|
|
|
|
- [ ] 5.4 (P) Golden-file test config build determinism
|
|
- Building from a minimal fixture, a full fixture covering all seven file types, and a set of intentionally invalid fixtures each produce the expected result
|
|
- Building the same fixture twice produces byte-identical output and an identical checksum
|
|
- _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 1.10, 1.11, 1.12, 9.1, 9.2, 9.3_
|
|
- _Boundary: xceadmin.build_
|
|
- _Depends: 3.1_
|
|
|
|
- [ ] 5.5 (P) Test every YAML validation failure mode
|
|
- Each invalid-fixture case (schema violation, reserved name, duplicate name including a cross-type collision, undefined reference) produces the documented outcome — hard failure or warning, matching the requirement
|
|
- _Requirements: 4.1, 4.2, 4.3, 4.4, 4.5_
|
|
- _Boundary: xcecommon.validation_
|
|
- _Depends: 2.1_
|
|
|
|
- [ ] 5.6 (P) Test config push and dry-run against a mock controller
|
|
- A mock TLS server standing in for the controller confirms the full push flow: successful push, redirect-following, and every documented error path
|
|
- The same mock confirms dry-run reports both an accepted and a rejected outcome correctly
|
|
- _Requirements: 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 3.1, 3.2, 3.3, 3.4, 6.1, 6.2, 6.3, 7.1, 7.2, 7.3_
|
|
- _Boundary: xceadmin.push, xceadmin.discovery_
|
|
- _Depends: 3.3_
|
|
|
|
## Implementation Notes
|
|
|
|
- **1.3**: nix's bare `python3` does not put `pip` on PATH — `python3Packages.pip` had to be added to `nativeBuildInputs` alongside the packages design.md's Technology Stack table lists; not a spec gap, just an operational dependency the table didn't need to spell out.
|
|
- **1.3**: Claude Code `PostToolUse` hooks receive tool input via stdin JSON (`tool_input.file_path`), not a `$FILE_PATH` env var — brief.md's ruff-hook sketch used the env-var form, which would have silently no-op'd. The actual hooks added to `.claude/settings.json` follow this repo's existing `.wolf/hooks/post-write.js` stdin-JSON convention instead. Any future hook work in this repo should follow the same pattern, not brief.md's original sketch.
|
|
- **1.3**: `src/xceadmin/src/xcecommon/schema_files/` (synced capnp + JSON schema copies) is a build artifact, regenerated every `xce-build.sh xceadmin` run — never hand-edit it, never commit it (`src/xceadmin/.gitignore` excludes it).
|
|
- **2.1**: `check_undefined_refs` deliberately does not parse resource names out of `res_req`/`gpu_req` free-form LSF-style expression strings (e.g. `"mem>1000 rusage[mem=1000]"`) — design.md's `xcecommon.validation` component scopes this check to queue→host-group and limit→group/`resname` only. `limits.yaml`'s `projects` field and `scaler.yaml` name collisions are likewise out of scope — no `projects.yaml` catalog exists among the 7 config types, and Requirement 4.3 only enumerates queue/host/resource/group/limit names. `src/configs/hosts.yaml` has 2 pre-existing undefined refs (resource `mg`, host `h3`) that now surface as warnings — expected, not a regression; fixing the example fixture is outside this task's boundary.
|
|
- **2.2**: The real `controller.capnp` names its reply struct `RequestReply`, not `Reply` as design.md's postcondition states — an inexact design pseudo-type, corrected empirically (verified directly against `src/schema/controller.capnp`, not assumed). pycapnp 2.0.0/capnproto 1.2.0 abort the whole process (uncaught C++ exception, not a Python exception) on a missing cross-file import or malformed `.capnp` content passed to `capnp.load()` — a bare `try/except` cannot catch this. `xcecommon.schema.load_schema()` works around it by pre-validating in an isolated subprocess before ever calling `capnp.load()` in the caller's own process, converting any abort into a clean `SchemaNotFoundError`. Any future code that calls `capnp.load()` directly (bypassing this loader) reintroduces the crash risk — always go through `xcecommon.schema.load_schema()`.
|
|
- **2.3**: `map_all_sections()` (a `SECTION_MAP`-driven orchestrator, not explicitly named in design.md's Service Interface but consistent with its Contracts) is the single call `xceadmin.build` (task 3.1) needs — it treats a missing YAML file identically to a present-but-empty section (byte-for-byte), satisfying Requirement 1.5 structurally. `target.init(field, 0)` and never touching an empty/absent list field produce *different* serialized bytes for the same logical "nothing here" value (verified empirically) — the mapper always picks "never touch" for byte-determinism (9.1). Calling `.to_bytes()` twice on the same capnp builder triggers a spurious "message already written" `UserWarning` even when no field was mutated between calls — capture the bytes once and reuse them rather than calling `.to_bytes()` again.
|
|
- **2.4**: `errors.py`'s exception subclasses (task 1.2) all take `message` as an explicit first constructor argument in addition to their structured fields (e.g. `ConnectionTimeoutError(message, host, port)`, `ReplyDecodeError(message, reason)`) — design.md's abbreviated `Service Interface` type stubs (`class ConnectionTimeoutError(XceAdminError): host: str; port: int`) read as if only the structured fields were needed; always check the real constructor in `errors.py`, not the design doc's shorthand. `xcecommon.wire` is deliberately opcode-agnostic — `send_request`/`send_frame`/`recv_frame` move raw `bytes` only and never import a capnp type; callers own `.to_bytes()`/`.from_bytes()`. Reused `ha/data/xce.crt`/`xce.key` (the repo's existing dev cert pair, same one `deploy-ha.sh` already copies for controller TLS) as the test fixture for both the TLS test server and the client `cert_dir` — no synthetic cert generation needed.
|