myxce/.kiro/specs/clusterconfig-schema-split/requirements.md

Requirements Document

Introduction

ClusterConfig (src/schema/controller.capnp:711-733, generated as C struct cluster_config_t) is the capnp struct pushed cluster-wide via OP_upgradeConfig. It currently carries several fields that were bundled in without a principled reason: port, candidates (peer/standby list), vcpus, and the jobmgr/workermgr/qmgr/resmgr/xstrings performance-tuning fields. This spec removes those fields from the schema and the C code that reads them, so ClusterConfig contains only fields that are genuinely cluster-shared, per the classification in .kiro/steering/config-design.md.

Correction (2026-07-20, second review pass): candidates, vcpus, and the perf-tuning fields ARE already dead weight at runtime (verified — see Adjacent Expectations). port is NOT — cluster_open_listener (cluster.c:1216-1258) has a live fallback to cluster_config->port that is the sole port-sourcing mechanism for single-node deployments (no --peers flag is passed at all in single-node mode — main.c:293-294). Removing the port field without adding a replacement mechanism would break single-node controller startup. This spec now requires that replacement as part of the port removal (Requirement 1.7), not as an assumed-safe deletion.

This closes an assumption flagged in .kiro/specs/admin-config-tool/requirements.md (the "per-instance field preservation" risk) — not by verifying controller behavior, but by making the scenario structurally impossible: fields that don't exist in the schema can't be silently zeroed on a config push.

Boundary Context

• In scope:

  • Remove port (@1) and candidates (@2, List(MasterCandidate)) from ClusterConfig.
  • Remove vcpus (@7) from ClusterConfig — already marked deprecated in the schema comment ("use --vcpus CLI flag; kept for config blob backward compatibility"); this is the schema revision that comment anticipated.
  • Remove jobmgr (@13), workermgr (@14), qmgr (@15), resmgr (@16), xstrings (@17) from ClusterConfig.
  • Remove the C code that reads the above fields from cluster_config_t: the candidates/ncandidates KV-mirror loop in cluster_build_config (cluster.c) is dead-code removal (no behavior change); the port-fallback branch in cluster_open_listener (cluster.c:1216-1258) is NOT dead code (see Introduction correction) and its removal requires a replacement mechanism per Requirement 1.7, not a bare deletion. Also remove the corresponding fields in cluster_config_t (clusterconfig.h) and the SET_DEFAULT_IF_UNSET(c->port, ...) calls in myconf.c and config_repl.c (only after the replacement mechanism lands).
  • Regenerate capnp bindings after the schema change.
  • Direct field removal (no deprecate-in-place transition, no field-number reservation) — the product has not reached 1.0, so capnp wire/on-disk compatibility for these fields is not a constraint (user decision, 2026-07-20).

• Out of scope:

  • heartbeatInterval (@3) and inactiveHeartbeatCount (@4) — these govern controller→worker liveness, not the HA/Raft election heartbeat (see Adjacent Expectations). Investigated and deliberately kept in ClusterConfig (operational-uniformity override — see .kiro/steering/config-design.md); this spec does not touch them.
  • fetchJobInterval (@5) and netioTimeout (@6) — flagged by the prior src/docs/cluster-config-split-analysis.md as candidates for the same treatment, but not yet decided in this pass. Left untouched; a candidate for a future iteration of this spec or a follow-up.
  • clusterName (@0), resources, queues, ugroups, hgroups, hostConfigs, hproviders, limits — cluster-shared business config, unaffected.
  • configGeneration (@20) — controller-assigned protocol state, not operator config; unaffected, not part of this spec's field-removal scope.
  • Adding a real HA/Raft election-heartbeat CLI parameter (--heartbeat-interval-ms/--heartbeat-timeout-ms) to replace the currently-hardcoded HA_DEFAULT_HEARTBEAT_INTERVAL_MS/HA_DEFAULT_HEARTBEAT_TIMEOUT_MS — tracked as a separate roadmap item (.kiro/steering/roadmap.md, "Raft/HA heartbeat interval+timeout as CLI parameters"), not part of this spec. This spec does not add any new ClusterConfig field for HA heartbeat timing, and per Rule 0 in config-design.md never should.
  • Peer list replication protocol (joint-consensus membership changes) — a much larger future item noted in the prior analysis doc's Phase 5; not needed here since candidates is simply being deleted, not replaced with a new distributed mechanism (the live peer source, --peers CLI, already exists and is unaffected).
  • entity-bearing-snapshots / config-upgrade-snapshot — unrelated concern (snapshot consistency at config-change boundaries); this spec has no dependency on either.

• Adjacent expectations:

  • Blast-radius finding (corrected 2026-07-20): candidates is fully dead in ClusterConfig at runtime — the live peer list comes exclusively from the --peers CLI flag (parsed into a separate struct, myconf.peers_cfg), and the blob's candidates is read only into a debug-only KV mirror (cluster_build_config, cluster.c:851-885) with no other readers. jobmgr/workermgr/qmgr/resmgr/xstrings/vcpus are likewise dead — hardcoded constants in cluster.c's init_controller_tunables() (cluster.c:1093-1108), with existing TODOs acknowledging the blob values are ignored; vcpus is sourced only from the separate CLI-set myconf->vcpus, never from the blob. port is the exception: cluster_open_listener (cluster.c:1216-1258) falls back to cluster_config->port whenever the local controller's name isn't matched in --peers — and single-node mode passes no --peers at all (main.c:293-294, "No --peers flag; running as single-node primary"), making this the sole port-sourcing path for that mode today. Also: xce_parse_peers only WARNs (not rejects) on a --peers candidate with port == 0 (main.c:289-291), so a malformed --peers entry for the local node hits the same live fallback. Full analysis: src/docs/cluster-config-split-analysis.md (note: that doc's own effort/risk table calls this "Low risk... no fallback needed," which this spec's investigation found does not hold for single-node mode — treat the doc's file:line citations as reliable, its risk conclusion for port specifically as superseded by this correction).
  • Two distinct "heartbeat" concepts (do not conflate when implementing): (1) the HA/Raft controller-to-controller election heartbeat (ha->config.heartbeat_interval_ms, governs failover timing, currently hardcoded, no ClusterConfig field, out of scope here per Rule 0 in config-design.md); (2) the controller→worker liveness heartbeat (ClusterConfig.heartbeatInterval/inactiveHeartbeatCount, governs when a controller marks its own worker unavailable, stays in ClusterConfig, out of scope here because it's unchanged, not because it's unrelated).
  • admin-config-tool dependency: xceadmin config push is soft-blocked on this spec landing (see admin-config-tool requirements.md Adjacent Expectations) — not because xceadmin's code depends on it, but because until these fields are removed, nobody has verified the controller preserves them correctly when a pushed config leaves them zero-valued.
  • MasterCandidate capnp struct: verified (grep, 2026-07-20) as referenced only by ClusterConfig.candidates and nowhere else in controller.capnp — removal is required by Requirement 1.6, not an open design question.
  • Stale citation, verified and dropped: the prior analysis doc cites testsuite/test.c:124,266 for an unrelated get_setting("port") (scheduler/worker connect-to-primary port) as a "don't touch this" landmine. That call no longer exists in test.c as of this spec's discovery pass — the citation has drifted. Two other test files DO reference a removed field directly: test_commit_barrier_rollout.c:1359 and test_config_repl.c:983 (see Requirement 5.1).
  • Prior analysis document: src/docs/cluster-config-split-analysis.md (2026-06-08) already scoped this exact change (its "Phase 1"), verified the zero-blast-radius claim, and listed the exact file:line change sites — treat it as a primary design input, not something to re-derive from scratch. Some line numbers in it have drifted; current locations are in this spec's discovery notes.

Requirements

Requirement 1: Remove Dead Peer/Listener Fields

Objective: As an XCE maintainer, I want port and candidates removed from ClusterConfig, so that the schema no longer implies these are cluster-shared values when the live peer list is already sourced from --peers.

Acceptance Criteria

1. The ClusterConfig capnp struct shall not contain a port field.
2. The ClusterConfig capnp struct shall not contain a candidates field.
3. When the controller opens its listener and its name is found in the --peers CLI-parsed candidate list, the system shall determine the bind port from that list, exactly as it does today.
4. When the controller builds its internal debug/status config dump, the system shall not attempt to read candidates or ncandidates from cluster_config_t (the field no longer exists).
5. The cluster_config_t C struct (src/include/clusterconfig.h) shall not contain port, ncandidates, or a candidates/master_candidate_t** member, matching the capnp struct.
6. The MasterCandidate capnp struct shall be removed from controller.capnp (verified: referenced only by ClusterConfig.candidates, nowhere else in the schema — its removal is not a design-phase open question, it's a direct consequence of Requirement 1.2's candidates removal).
7. Where the controller's name is NOT found in --peers (including single-node mode, where --peers is not passed at all), the system shall provide a way to determine the controller's listen port that does not depend on a ClusterConfig field. This spec does not mandate the specific mechanism (e.g., a new --port CLI flag, or requiring single-node mode to pass a self-referencing --peers entry) — selecting it is a design-phase decision — but Requirement 1.1's removal of the port field SHALL NOT ship without this replacement in place, and single-node controller startup SHALL continue to work exactly as it does today once the replacement lands.

Requirement 2: Remove Dead Performance-Tuning Fields

Objective: As an XCE maintainer, I want the already-hardcoded performance-tuning fields removed from ClusterConfig, so that the schema doesn't advertise configurability that doesn't exist.

Acceptance Criteria

1. The ClusterConfig capnp struct shall not contain vcpus, jobmgr, workermgr, qmgr, resmgr, or xstrings fields.
2. While these fields are absent from the schema, the controller shall continue to use its existing hardcoded tunable values (c->tunables.* defaults in cluster.c) with no behavior change from before this spec.
3. When the controller starts, the system shall not attempt to read vcpus, jobmgr, workermgr, qmgr, resmgr, or xstrings from cluster_config_t (the fields no longer exist).
4. The cluster_config_t C struct (src/include/clusterconfig.h) shall not contain vcpus, jobmgr, workermgr, qmgr, resmgr, or xstrings members, matching the capnp struct.
5. The HashZArrayConfig and ResmgrConfig capnp structs shall be removed from controller.capnp (discovered during /kiro-impl task 2.1, 2026-07-20: verified via grep that these structs are referenced only by the removed jobmgr/workermgr/qmgr/xstrings fields and the removed resmgr field respectively, nowhere else in the schema — their removal is a direct consequence of Criterion 2.1's field removal, the same class of cleanup as Requirement 1.6's MasterCandidate removal. Left in place, their $C.mapname(...) bindings reference C typedefs that Requirement 2.4's struct-member removal deletes, breaking the capnp-generated bindings' compile).

Requirement 3: Preserve Cluster-Shared Fields Unchanged

Objective: As an XCE operator, I want the fields that are genuinely cluster-shared to remain exactly as they are today, so that this schema cleanup doesn't regress any existing config-push behavior.

Acceptance Criteria

1. The ClusterConfig capnp struct shall retain clusterName, heartbeatInterval, inactiveHeartbeatCount, fetchJobInterval, netioTimeout, resources, queues, ugroups, hgroups, hostConfigs, hproviders, limits, and configGeneration unchanged in type and semantics. Field numbers (@N) for these fields ARE expected to change (compacted to a contiguous 0..12 range): Cap'n Proto requires struct ordinals to be a gapless sequence, so removing 8 fields via direct deletion (Requirement 4.2 — not in-place reservation) necessarily renumbers every retained field declared after a removed one. This was verified empirically during /kiro-impl (2026-07-20): a schema with an ordinal gap fails to compile (error: Skipped ordinal @N. Ordinals must be sequential with no holes.).
2. Because retained fields are renumbered per 3.1, a cluster.bin built before this spec is NOT expected to decode its retained fields correctly when pushed to a controller built after this spec — Cap'n Proto's field layout is ordinal-position-based, so renumbering shifts wire byte-offsets for every field after the first removed one, not just the removed fields' own bytes. This is an accepted compatibility break for this pre-1.0 product (see Requirement 4.1). Any cluster.bin artifact that predates this spec — including the checked-in ha/data/cluster.bin used by /xce-deploy-ha and /xce-test-ha — must be rebuilt against the new schema before being used with a post-spec controller (Requirement 5.3).
3. The controller→worker liveness behavior driven by heartbeatInterval/inactiveHeartbeatCount shall be unaffected by this spec.

Requirement 4: Compatibility Posture

Objective: As an XCE maintainer, I want the compatibility expectations for this change stated explicitly, so implementation and review don't over- or under-build compatibility handling.

Acceptance Criteria

1. Because this spec performs direct field removal via ordinal renumbering rather than in-place reservation (4.2), a cluster.bin built before this spec is not binary-compatible with a controller built after this spec — not only are the removed fields' values (port, candidates, vcpus, jobmgr, workermgr, qmgr, resmgr, xstrings) discarded without error, warning, or migration, but retained-field values will not decode correctly if such a pre-spec blob is pushed as-is, since their wire byte-offsets shifted (see 3.1/3.2). This is accepted given the product has not reached 1.0; no backward-read guarantee is provided for any pre-spec cluster.bin, and any such artifact — including the repo's own fixture — must be rebuilt against the new schema (Requirement 5.3).
2. The system shall not implement a deprecate-in-place transition (field-number reservation, dual-read-old-and-new) for the fields removed by this spec — direct removal (with the resulting renumbering of retained fields) is the chosen approach.

Requirement 5: Build and Test Verification

Objective: As an XCE maintainer, I want this schema change verified as behavior-preserving, so the "zero blast radius" claim from the prior analysis is proven, not assumed.

Acceptance Criteria

1. When the controller AND its test suite are built after this spec's changes, the system shall compile without errors using the regenerated capnp bindings. This includes test code that directly constructs a cluster_config_t literal and sets a removed field — confirmed present at test_commit_barrier_rollout.c:1359 (cfg.port = 9990;) and test_config_repl.c:983 (cfg.port = 9991;); both shall be updated to drop the now-nonexistent assignment. Any other direct struct-literal reference to a removed field found during implementation shall be updated the same way.
2. When the existing controller unit test suite is run after this spec's changes, the system shall show no new failures attributable to the removed fields.
3. Because field-number renumbering breaks binary compatibility with any pre-spec cluster.bin (Requirement 3.2/4.1), the system shall regenerate the checked-in ha/data/cluster.bin fixture against the new (post-renumbering) schema, preserving the semantic values of all retained fields from the pre-spec blob, so /xce-deploy-ha and /xce-test-ha continue to work for this and every future spec that relies on that fixture.
4. When a 3-node HA cluster is deployed and run through a config-push cycle after this spec's changes — using the regenerated ha/data/cluster.bin from Criterion 3 — the system shall behave identically to before this spec (peer discovery via --peers, listener binding, worker liveness tracking) — verified live, not just by unit tests, given this touches cluster.c/ha.c/main.c.
5. When a single-node controller (no --peers flag) is started after this spec's changes, the system shall successfully bind its listener and operate correctly using Requirement 1.7's replacement port-sourcing mechanism — this is the scenario Finding #2 of this spec's review identified as the actual regression risk, and it shall be verified live, not assumed from the multi-node soak in Criterion 4.

Requirement 6 (Nice-to-Have): CLI-Configurable Performance Tunables

Objective: As an XCE operator running a high-memory or high-latency controller node, I want the previously-hardcoded hash-table-size tunables to become CLI-configurable, so I can tune an individual controller's performance without needing a schema field that was never wired up anyway.

Acceptance Criteria

1. Where this nice-to-have is implemented, the system shall accept --jobmgr-nhash, --workermgr-nhash, --qmgr-nhash, --resmgr-nhash, --xstrings-nhash (and corresponding -delta/-inst variants as applicable) CLI flags, following the same OPT_* + struct arguments + myconf_t field pattern already used for --vcpus.
2. Where a tunable CLI flag is not specified (or is 0), the system shall fall back to the existing hardcoded default for that tunable, with no behavior change from Requirement 2.