# Implementation Plan - [ ] 1. Schema & Struct Foundation - [x] 1.1 Remove obsolete fields and the `MasterCandidate` struct from the `ClusterConfig` schema, regenerate bindings - Remove `port` (@1), `candidates` (@2), `vcpus` (@7), `jobmgr` (@13), `workermgr` (@14), `qmgr` (@15), `resmgr` (@16), `xstrings` (@17) from `ClusterConfig` in `controller.capnp` - Remove the `MasterCandidate` struct entirely - Confirm `clusterName`, `heartbeatInterval`, `inactiveHeartbeatCount`, `fetchJobInterval`, `netioTimeout`, `resources`, `queues`, `ugroups`, `hgroups`, `hostConfigs`, `hproviders`, `limits`, `configGeneration` remain unchanged in type and semantics; renumber them to a contiguous `@0..@12` range (Cap'n Proto forbids ordinal gaps — verified empirically, see `research.md`'s "Design Revision 2" — so direct removal of the 8 fields necessarily renumbers every retained field declared after a removed one) - Regenerate the capnp C bindings via the existing meson codegen step; do not hand-edit generated output - Observable: capnp compiler succeeds against the edited schema; the generated `ClusterConfig` reader/builder exposes exactly 13 fields, ordinals `@0..@12`, and no `MasterCandidate` type - _Requirements: 1.1, 1.2, 1.6, 2.1, 3.1, 4.2_ - _Boundary: Schema Component_ - [x] 1.2 Remove the matching members from `cluster_config_t` - Delete `port`, `ncandidates`, `candidates`/`master_candidate_t**`, `vcpus`, `jobmgr`, `workermgr`, `qmgr`, `resmgr`, `xstrings` members from `cluster_config_t` in `clusterconfig.h` - Leave all retained members (matching the schema's surviving 13 fields) untouched - Observable: `clusterconfig.h`'s field list matches the regenerated schema field-for-field; any remaining reference to a removed member elsewhere in the tree is now a compile error, not a silent gap - _Requirements: 1.5, 2.3, 2.4_ - _Depends: 1.1_ - _Boundary: C Struct Component_ - [x] 1.3 Remove the orphaned `HashZArrayConfig`/`ResmgrConfig` capnp structs - Discovered during task 2.1's partial compile check: these structs (used only by the fields task 1.1 already removed) were deliberately left in `controller.capnp`, but task 1.2 removed their corresponding hand-written C typedefs (`hash_zarray_config_t`, `resmgr_config_t`) — the capnp-generated bindings still reference those now-nonexistent types via `$C.mapname(...)`, breaking compilation - Remove the `HashZArrayConfig` and `ResmgrConfig` struct definitions entirely from `controller.capnp` (grep-verify first that neither is referenced anywhere else in the schema, same method used for `MasterCandidate` in task 1.1) - Regenerate the capnp C bindings - Observable: capnp compiler succeeds; the generated bindings no longer emit any `HashZArrayConfig`/`ResmgrConfig`/`hash_zarray_config_t`/`resmgr_config_t` reference anywhere - _Requirements: 2.5_ - _Depends: 1.1, 1.2_ - _Boundary: Schema Component_ - [ ] 2. Core: cluster.c Port Resolution, Debug-Dump & Port-Defaulting Cleanup - [x] 2.1 Update `cluster_open_listener`'s port-resolution fallback - Change the fallback branch from reading `cluster_config->port` to a new hardcoded `XCE_CONTROLLER_DEFAULT_PORT` constant (9990), reached only when `--peers` was never passed at all (`peers_cfg.candidates_count == 0`) - Do NOT add a new CLI flag or `myconf_t` field — the `--peers` candidate-match branch (including its existing single self-referencing `--peers =:` usage for single-node mode) is left completely unchanged; this is the only mechanism an operator uses to get a non-default single-node port - Preserve the exact precedence and branch structure: a `--peers` match with nonzero port binds to it unchanged; a matched candidate with port == 0 falls through to the hardcoded-default tier exactly as today's tolerated case; `--peers` passed but local name absent still returns `-EINVAL` and refuses to start, unchanged - Observable: a controller started with no `--peers` binds to 9990; a controller started with a single self-referencing `--peers` entry binds to that entry's port; a controller started with `--peers` not containing the local name still exits with the existing `-EINVAL` startup error - _Requirements: 1.3, 1.7_ - _Depends: 1.2_ - _Boundary: Listener/Port-Resolution Component (cluster.c)_ - [x] 2.2 Remove the `candidates` debug-dump mirror - Delete the `candidates`/`ncandidates` KV-mirror loop from `cluster_build_config` - Confirm `init_controller_tunables()` itself is left unchanged — the hardcoded tunable constants and their consumption are untouched by this spec - Observable: `cluster_build_config` compiles without referencing `candidates` or `ncandidates`; the controller's internal debug/status config dump no longer includes those keys - _Requirements: 1.4, 2.2_ - _Depends: 1.2_ - _Boundary: Debug-Dump Component (cluster.c)_ - [x] 2.3 Remove the now-dead `ClusterConfig` port defaulting in `myconf.c` and `config_repl.c` - Delete the `SET_DEFAULT_IF_UNSET(c->port, 0, 9990)` call in `myconf.c` - Delete the equivalent `SET_DEFAULT_IF_UNSET(c->port, 0, 9990)` call in `config_repl.c` - Observable: `myconf.c` and `config_repl.c` compile without referencing `cluster_config_t`'s `port` member - _Requirements: 1.7_ - _Depends: 2.1_ - _Boundary: myconf.c, config_repl.c_ - [x] 3. (P) Update struct-literal test fixtures for the shrunk `cluster_config_t` - Drop the `cfg.port = 9990;` literal at `test_commit_barrier_rollout.c:1359` - Drop the `cfg.port = 9991;` literal at `test_config_repl.c:983` - Grep the full tree for any other direct `cluster_config_t` struct-literal reference to a removed field and update each the same way - Observable: both known test files compile against the shrunk struct with no remaining reference to a removed field; a repo-wide grep for the removed field names against struct-literal assignments returns no further hits - _Requirements: 5.1_ - _Depends: 1.2_ - _Boundary: Test Fixture Updates_ - [x] 4. Build the controller and run the full unit test suite - Build `controller_unit_test` (meson target) with the regenerated capnp bindings and all updated source/test files - Run the full existing suite and compare against the pre-spec baseline - Observable: build completes with zero compile errors; `controller_unit_test` passes with no new failures attributable to the removed fields - _Requirements: 5.1, 5.2_ - _Depends: 1.1, 1.2, 1.3, 2.1, 2.2, 2.3, 3_ - [ ] 5. Validation: Live HA Verification - [x] 5.1 Regenerate the `ha/data/cluster.bin` fixture against the renumbered schema - Decode the current (pre-spec) `ha/data/cluster.bin`'s retained-field values under the OLD (pre-renumbering) schema — build against a pre-1.1 checkout or otherwise capture the pre-spec field values before they become unreadable - Write and run a small throwaway capnp-encoding tool (not part of the shipped codebase — discard it after use) that constructs a `ClusterConfig` with those same retained-field values and serializes it under the NEW (post-task-1.1) schema - Overwrite `ha/data/cluster.bin` with the newly-encoded blob - Observable: the regenerated `ha/data/cluster.bin` decodes correctly under the post-spec schema with every retained field matching the pre-spec blob's semantic values; `deploy-ha.sh`'s existing verbatim-copy usage (`deploy-ha.sh:33-36`) requires no changes and keeps working - _Requirements: 3.2, 4.1, 5.3_ - _Depends: 4_ - [x] 5.2 3-node HA cluster config-push verification - Deploy a 3-node HA cluster using the existing deploy tooling (which seeds ctrl-1 from the regenerated `ha/data/cluster.bin`) and confirm peer discovery via `--peers`, listener binding, and worker-liveness tracking behave identically to pre-spec - Worker-liveness tracking (Req 3.3) is verified indirectly: `heartbeatInterval`/`inactiveHeartbeatCount` are retained, unmodified fields, and this spec touched no worker-liveness code path (`cluster.c`'s delay-task scheduling consuming them, e.g. around the `check_worker_timeout_`/liveness-poll call sites) — the observable is that the regenerated config activates cleanly on all 3 nodes with no error from that subsystem, not a live worker daemon exercising a timeout - Observable: the 3-node cluster reaches steady state with unchanged peer/listener/worker-liveness behavior, using the regenerated fixture with no error or warning - _Requirements: 3.3, 5.4_ - _Depends: 4, 5.1_ - [x] 5.3 Single-node startup, self-referencing-peers port override, and preserved error-path verification - Start a controller with no `--peers` flag and confirm it binds its listener to the hardcoded default port (9990) from task 2.1 - Start a controller with a single self-referencing `--peers =:` entry (own `-n`/`--myname` matching, custom port) and confirm it binds to that port instead of the default — this is the sole replacement mechanism for a non-default single-node port under this spec - Start a controller with a `--peers` list that does not contain the controller's own name and confirm it still refuses to start with the existing `-EINVAL` error, unchanged from pre-spec behavior - Observable: the no-peers controller reaches a running state bound to port 9990; the self-referencing-peers controller reaches a running state bound to its specified custom port; the name-absent-from-peers controller exits with the same startup error it produced before this spec - _Requirements: 5.5_ - _Depends: 4_ - [ ] 6. (Optional — Requirement 6, nice-to-have, deferrable without blocking the rest of this spec) CLI-Configurable Performance Tunables - [x] 6.1 Add CLI flags for the hash-table tunables - Add `--jobmgr-nhash`, `--workermgr-nhash`, `--qmgr-nhash`, `--resmgr-nhash`, `--xstrings-nhash` (and corresponding `-delta`/`-inst` variants where the tunable has more than one dimension) following the same `OPT_*` + `struct arguments` + `myconf_t` pattern already used for `--vcpus` - Observable: `xce_controller --help` lists all new flags; passing one populates the corresponding `myconf_t` field - _Requirements: 6.1_ - _Depends: 1.2_ - _Boundary: CLI Tunables Component (main.c, myconf.h)_ - [x] 6.2 Wire the tunables into `init_controller_tunables()` - Read the new `myconf_t` fields in `init_controller_tunables()`, falling back to the existing hardcoded default for any field left at 0 (unset) - Observable: passing a tunable flag changes the corresponding controller tunable at startup; omitting all of them reproduces today's exact hardcoded values - _Requirements: 6.2_ - _Depends: 6.1_ - _Boundary: Tunables Component (cluster.c)_