myxce/.kiro/specs/clusterconfig-schema-split/design.md

Sections may be reordered when it improves clarity; this is a backend/systems (C daemon + capnp schema) change, not a web app — "Service Interface"/API/Event contract subsections are adapted to C function-signature contracts where used, and omitted where not applicable.

Overview

Purpose: Removes eight fields from the ClusterConfig capnp struct (port, candidates, vcpus, jobmgr, workermgr, qmgr, resmgr, xstrings) that are either already dead at runtime or (in port's case) load-bearing for single-node startup but conceptually misplaced in cluster-shared config. port's replacement source is the already-existing --peers candidate-match mechanism plus a hardcoded default — no new CLI flag.

Users: XCE maintainers (schema/controller code owners) and, indirectly, XCE operators running single-node or --peers-mismatched controllers, whose startup behavior must not change.

Impact: ClusterConfig shrinks from 21 to 13 fields. cluster_config_t (the generated C struct) loses the matching members. cluster_open_listener's port-resolution fallback moves from a capnp-blob field to a two-tier shape: --peers candidate match (unchanged, including the already-supported single self-referencing --peers <name>=<host>:<port> entry for single-node mode) → hardcoded constant when --peers was never passed at all.

Goals

• Remove the 8 fields and the MasterCandidate struct from controller.capnp, and their mirrors in cluster_config_t.
• Replace port's ClusterConfig-sourced fallback with the existing --peers candidate-match mechanism (unchanged) plus a hardcoded default for the true zero---peers case, preserving single-node startup behavior exactly with no new CLI flag.
• Prove behavior-preservation live (single-node startup, 3-node soak against a regenerated ha/data/cluster.bin fixture) — not just by unit tests, given the touched files (cluster.c, ha.c, main.c) are HA-critical.
• (Nice-to-have) Add CLI flags for the hash-tuning tunables, closing the existing cluster.c TODOs.

Non-Goals

• heartbeatInterval/inactiveHeartbeatCount (controller→worker liveness) — unchanged, stay in ClusterConfig.
• fetchJobInterval/netioTimeout — unresolved, left untouched, a candidate for a future pass.
• A real HA/Raft election-heartbeat CLI parameter — separate roadmap item, not this spec.
• Peer list replication protocol / joint consensus — future, unrelated to a field deletion.
• Any change to entity-bearing-snapshots / config-upgrade-snapshot.

Boundary Commitments

This Spec Owns

• ClusterConfig and MasterCandidate struct shape in controller.capnp.
• cluster_config_t's field list in clusterconfig.h.
• The port-resolution fallback branch in cluster_open_listener (cluster.c): --peers candidate match (unchanged) → hardcoded default constant.
• The dead-code removals that don't need a replacement: candidates KV-mirror loop (cluster_build_config), SET_DEFAULT_IF_UNSET(c->port, ...) call sites, and the (already-inert) jobmgr/workermgr/qmgr/resmgr/xstrings/vcpus blob reads.
• The two known test-fixture updates (test_commit_barrier_rollout.c, test_config_repl.c) and any others a full-repo grep surfaces during implementation.
• Regenerating capnp bindings.
• (Optional, Requirement 6) CLI-configurable perf tunables — five new OPT_* flags mirroring the existing --vcpus, deferrable without blocking the rest of this spec.

Out of Boundary

• heartbeatInterval/inactiveHeartbeatCount, fetchJobInterval/netioTimeout — no field or code change.
• A ClusterConfig field or CLI flag for the HA/Raft election heartbeat — structurally disallowed (config-design.md Rule 0); tracked as its own roadmap item.
• Peer list replication / joint-consensus membership — unrelated, candidates is deleted, not replaced with a distributed mechanism.
• config-upgrade-snapshot, entity-bearing-snapshots — no dependency either direction.
• admin-config-tool's xceadmin implementation — this spec closes the risk that tool depends on (per its requirements.md), but does not implement or block on any of that tool's own code.

Allowed Dependencies

• Existing build tooling: capnp compiler + c-capnproto codegen (schema → C bindings), already wired into the meson build.
• argp CLI parsing (already used by every other controller flag, including --vcpus).
• No new external dependencies, no new libraries.

Revalidation Triggers

• Any future proposal to add a field to ClusterConfig must be checked against .kiro/steering/config-design.md's inclusion rules before landing.
• admin-config-tool's xceadmin config build/push implementation, when it lands, should target the post-split (13-field) schema — if it's implemented against the pre-split schema instead, that's a signal this spec's sequencing assumption was violated.
• If the controller's invocation contract changes elsewhere (e.g., --peers becomes mandatory for another reason), revisit whether the hardcoded-default fallback added here is still needed.

Architecture

Existing Architecture Analysis

ClusterConfig today mixes three concerns in one capnp struct: cluster-wide business config (queues, resources, groups — genuinely shared), HA peer membership (candidates, already superseded in practice by the --peers CLI flag into a separate xce_cluster_config_t/peers_cfg struct), and per-controller performance tunables (jobmgr/workermgr/qmgr/resmgr/xstrings/vcpus, already hardcoded in cluster.c's init_controller_tunables() with TODOs acknowledging the blob values are dead). port sits in the same struct but, unlike the others, still has one live reader: cluster_open_listener's fallback branch, reached whenever the local controller's name isn't matched in --peers (including single-node mode, which passes no --peers at all).

Architecture Pattern & Boundary Map

graph TB
    Schema[ClusterConfig capnp schema]
    Codegen[capnp codegen]
    ConfigStruct[cluster_config_t]
    Blob[cluster.bin blob]

    PeersFlag[peers CLI flag]
    PeersCfg[peers_cfg struct]

    Listener[cluster_open_listener]
    Tunables[init_controller_tunables]
    Business[cluster_init_phase2 business config]

    Schema --> Codegen --> ConfigStruct
    Blob --> ConfigStruct
    ConfigStruct --> Business

    PeersFlag --> PeersCfg --> Listener
    Listener --> DefaultConst[hardcoded default constant]

    Tunables --> HardcodedTunables[hardcoded tunable constants]

Architecture Integration:

• Selected pattern: no new CLI surface — port resolution reuses the existing --peers candidate-match loop (a single self-referencing --peers <name>=<host>:<port> entry already gives single-node mode a custom port today) plus a hardcoded fallback constant for the true zero---peers case.
• Domain/feature boundaries: ClusterConfig (schema) retains only fields that are genuinely cluster-shared (per config-design.md); peers_cfg (CLI-sourced, per-process, already existing) owns port resolution for everything this spec touches.
• Existing patterns preserved: --peers remains the sole port/membership source; init_controller_tunables()'s hardcoded constants are untouched (Requirement 2.2 — no behavior change).
• New components: one new hardcoded constant (cluster.c or clusterconfig.h) — the minimum needed to satisfy Requirement 1.7 without any invocation-contract or CLI-surface change.
• Steering compliance: enforces .kiro/steering/config-design.md's independence/frequency/Rule-0 classification; follows src/controller/CLAUDE.md's HA protocol verification checklist for the touched files.

Technology Stack

Layer	Choice / Version	Role in Feature	Notes
Schema	Cap'n Proto (existing, controller.capnp)	Defines ClusterConfig/MasterCandidate shape	No version change
Codegen	c-capnproto (existing)	Schema → C struct/reader/builder	Regenerated, not upgraded
Backend	C (gnu17, existing)	cluster.c/myconf.c/config_repl.c changes	No new C dependency
CLI	argp (existing)	No new flag for Req 1.7; reused unchanged for optional Req 6 tunables, same pattern as --vcpus	No new dependency

File Structure Plan

Modified Files

• src/schema/controller.capnp — remove port (@1), candidates (@2), vcpus (@7), jobmgr (@13), workermgr (@14), qmgr (@15), resmgr (@16), xstrings (@17) from ClusterConfig; remove the MasterCandidate struct entirely; remove the now-orphaned HashZArrayConfig/ResmgrConfig structs (used only by the removed fields — discovered during /kiro-impl task 2.1, their $C.mapname bindings broke the capnp-generated code once task 1.2 removed the C typedefs they referenced). Cap'n Proto requires gapless ordinals, so the 13 retained fields are renumbered to a contiguous 0..12 range as a direct consequence — see research.md's ordinal-gap finding (Req 1.1, 1.2, 1.6, 2.1, 2.5, 3.1).
• ha/data/cluster.bin (binary fixture, not source) — regenerated in place against the renumbered schema, preserving the retained fields' semantic values from the pre-spec blob (Req 3.2, 5.3).
• src/include/clusterconfig.h — remove the matching cluster_config_t members: port, ncandidates, candidates/master_candidate_t**, vcpus, jobmgr, workermgr, qmgr, resmgr, xstrings (Req 1.5, 2.4).
• src/controller/cluster.c — (a) remove the candidates/ncandidates KV-mirror loop in cluster_build_config (Req 1.4); (b) change cluster_open_listener's fallback branch from reading cluster_config->port to a hardcoded default constant, reached only when --peers was never passed at all — the --peers candidate-match branch itself is unchanged (Req 1.3, 1.7); init_controller_tunables() itself is unchanged (Req 2.2).
• src/controller/myconf.c — remove the SET_DEFAULT_IF_UNSET(c->port, 0, 9990) call (the field it defaults no longer exists) (part of Req 1.7's cleanup).
• src/controller/config_repl.c — remove the equivalent SET_DEFAULT_IF_UNSET(c->port, 0, 9990) call (same reason).
• src/controller/testsuite/test_commit_barrier_rollout.c — drop the cfg.port = 9990; literal at line 1359 (Req 5.1).
• src/controller/testsuite/test_config_repl.c — drop the cfg.port = 9991; literal at line 983 (Req 5.1).
• Capnp-generated bindings (build output, not hand-edited) — regenerated as part of the normal build after the schema change (Req 1.1-1.2, 1.6, 2.1).

Verification Infrastructure (existing, no new files)

• controller_unit_test (meson target, existing) — runs the full existing suite including the two updated test files above (Req 5.2).
• A small throwaway capnp-encoding tool (written and discarded during implementation, not part of the shipped codebase) — decodes ha/data/cluster.bin's retained-field values under the pre-spec schema and re-encodes them under the post-spec (renumbered) schema, overwriting the fixture (Req 3.2, 5.3).
• /xce-deploy-ha skill (existing, src/tools/deploy-ha.sh) — deploys the 3-node live cluster for the config-push cycle, using the regenerated ha/data/cluster.bin (Req 5.4).
• Direct xce_controller invocation with no --peers flag (existing supported mode, main.c:293-295) — the single-node startup verification (Req 5.5).
• Direct xce_controller invocation with a single self-referencing --peers <name>=<host>:<port> entry (existing supported mode) — verifies the custom-port replacement mechanism itself (Req 5.5).

Optional (Requirement 6, nice-to-have)

• src/controller/main.c — five new OPT_* flags (--jobmgr-nhash, --workermgr-nhash, --qmgr-nhash, --resmgr-nhash, --xstrings-nhash, plus -delta/-inst variants where the tunable has more than one dimension), same pattern as the existing --vcpus.
• src/include/myconf.h — matching myconf_t fields.
• src/controller/cluster.c — init_controller_tunables() reads the new myconf_t fields with the existing hardcoded values as fallback (0 = unset), rather than always using the hardcoded constant.

No new files are created by this spec — every change is to an existing file.

System Flows

Port resolution at controller startup — corrected during design review (see research.md Design Decisions for the two fixes below):

flowchart TD
    Start[cluster_open_listener starts]
    HasPeers{peers_cfg candidates_count greater than 0}
    SearchName[search peers_cfg for local name]
    NameFound{local name found}
    PortValid{matched candidate port greater than 0}
    UseMatchedPort[use matched candidate port]
    UseFallback[use hardcoded default port]
    RefuseStart[log error and refuse to start]
    Bind[bind listener]

    Start --> HasPeers
    HasPeers -->|no, --peers never passed| UseFallback --> Bind
    HasPeers -->|yes| SearchName --> NameFound
    NameFound -->|no| RefuseStart
    NameFound -->|yes| PortValid
    PortValid -->|yes| UseMatchedPort --> Bind
    PortValid -->|no, malformed entry| UseFallback

Key decisions (corrected from the initial draft during /kiro-validate-design; the --port CLI flag was further revised away in favor of the mechanism below during /kiro-impl kickoff — see research.md's "Design Revision" section):

• Fallback trigger is port value, not just match presence. A matched candidate whose port field is 0 (a malformed --peers entry — already WARNed about at main.c:289-291) falls through to the hardcoded-default tier exactly like today's if (my_port == 0) check, rather than binding to literal port 0. This preserves Requirement 1.3's "exactly as it does today" for the already-tolerated malformed-entry case.
• Error path is preserved, but only for one specific case. candidates_count == 0 can only happen when --peers was never passed at all — a passed-but-empty or malformed --peers string is rejected earlier by xce_parse_peers itself (src/lib/cluster.c:58 returns -1 unless candidates_count > 0), which main.c treats as a startup error before cluster_open_listener is ever reached (corrected citation, 2026-07-20: this guarantee comes from xce_parse_peers's contract, not validate_peer_count, which only rejects even non-zero counts) — that's genuine single-node mode, and gets the hardcoded-default fallback per Requirement 1.7. An operator who wants a non-default port in this mode passes a single self-referencing --peers <name>=<host>:<port> entry instead — an already-existing, already-supported invocation (this resolves via the unchanged NameFound/PortValid branches above, not a new mechanism). But when --peers WAS passed and the local name simply isn't in it, that's a likely operator misconfiguration (own name typo'd or omitted) — this spec keeps today's -EINVAL/refuse-to-start behavior for that case rather than silently defaulting, per the project's fail-loud convention (src/controller/CLAUDE.md process-lifecycle invariant). Decided interactively during design review rather than defaulting to "simplest single code path."

Requirements Traceability

Requirement	Summary	Components	Files
1.1	Remove port from schema	Schema Component	controller.capnp
1.2	Remove candidates from schema	Schema Component	controller.capnp
1.3	Listener uses --peers match when available	Listener/Port-Resolution Component	cluster.c
1.4	Debug dump stops reading candidates	Debug-Dump Component	cluster.c
1.5	cluster_config_t drops port/candidates members	C Struct Component	clusterconfig.h
1.6	Remove MasterCandidate struct	Schema Component	controller.capnp
1.7	Replacement port-sourcing mechanism (existing --peers self-reference + hardcoded default)	Listener/Port-Resolution Component	cluster.c, myconf.c, config_repl.c
2.1	Remove 6 perf-tuning fields from schema	Schema Component	controller.capnp
2.5	Remove orphaned HashZArrayConfig/ResmgrConfig structs	Schema Component	controller.capnp
2.2	Hardcoded tunables unchanged	Tunables Component	cluster.c (no change)
2.3	Controller stops attempting to read removed tunable fields	Tunables Component	cluster.c (implicit — field no longer exists)
2.4	cluster_config_t drops tunable members	C Struct Component	clusterconfig.h
3.1	Retained fields unchanged in type/number	Schema Component	controller.capnp (no change to retained fields)
3.2	Renumbering breaks pre-spec cluster.bin binary-compat (accepted); fixture rebuilt	Schema Component, Verification	controller.capnp, ha/data/cluster.bin regeneration
3.3	Worker-liveness behavior unaffected	Tunables/Business Component	cluster.c (no change)
4.1	Discard removed-field values from legacy blobs silently	Schema Component	controller.capnp (capnp default behavior)
4.2	No deprecate-in-place transition	Schema Component	controller.capnp
5.1	Build + test-fixture compile cleanly	Test Fixture Updates	test_commit_barrier_rollout.c, test_config_repl.c
5.2	No new unit test failures	Verification	controller unit test suite
5.3	Regenerate ha/data/cluster.bin fixture against the new schema	Verification	ha/data/cluster.bin
5.4	3-node live soak (config-push, peer/listener/worker-liveness)	Verification	live HA cluster
5.5	Single-node + bare-peers port-override + -EINVAL live verification	Verification	live single-node controller
6.1	(Nice-to-have) CLI tunable flags	CLI Tunables Component (optional)	main.c, myconf.h
6.2	(Nice-to-have) Fallback to hardcoded default	Tunables Component (optional)	cluster.c

Components and Interfaces

Component	Domain	Intent	Req Coverage	Key Dependencies	Contracts
Schema Component	capnp	Defines ClusterConfig/MasterCandidate shape	1.1, 1.2, 1.6, 2.1, 3.1, 4.1, 4.2	capnp compiler (P0)	State
C Struct Component	C headers	cluster_config_t field list matches schema	1.5, 2.4	Schema Component (P0)	State
Listener/Port-Resolution Component	C, cluster.c	Resolves listen port via --peers match (unchanged) with a hardcoded-default fallback	1.3, 1.7	C Struct Component (P0)	Service
Debug-Dump Component	C, cluster.c	Internal status/debug KV dump, no longer mirrors candidates	1.4	C Struct Component (P0)	State
Tunables Component	C, cluster.c	Hardcoded (or optionally CLI-sourced) perf tunables	2.2, 2.3, 3.3, 6.2	CLI Tunables Component (P1, optional path)	State
CLI Tunables Component (optional)	C, main.c/myconf.h	Requirement 6 only: parses tunable flags, populates myconf_t, same pattern as the existing --vcpus	6.1	argp (P0)	Service
Test Fixture Updates	C tests	Two known struct-literal sites updated to compile	5.1	C Struct Component (P0)	—
Verification	Build + live cluster	Proves behavior preservation	5.1, 5.2, 5.3, 5.4, 5.5, 3.2, 4.1	All components above (P0)	—

Only the Listener/Port-Resolution Component introduces new logic (the two-tier fallback); every other component is either schema/struct shrinkage or already-hardcoded behavior left untouched, so only that one gets a full contract block below.

Controller

Listener/Port-Resolution Component

Field	Detail
Intent	Determine the controller's own listen port without depending on a ClusterConfig field
Requirements	1.3, 1.7

Responsibilities & Constraints

• Primary responsibility: resolve exactly one port value before cluster_open_listener binds.
• Precedence is strict and non-configurable: --peers match (if local name found) > hardcoded default constant. No tier is skipped or reordered by configuration. No CLI flag mediates this — an operator who needs a non-default port in the no-other-peers case supplies it via a single self-referencing --peers <name>=<host>:<port> entry, which resolves through the same match branch as any other --peers candidate.
• Must not depend on cluster_config_t in any form (the field this component replaces no longer exists after this spec).

Dependencies

• Inbound: main.c startup sequence — invokes this resolution before/during cluster_open_listener (P0)
• Outbound: myconf->peers_cfg (existing, unchanged parsing) (P0)
• Outbound: hardcoded default constant (same file or clusterconfig.h) — no external dependency (P0)

Contracts: Service [x]

Service Interface

/* cluster_open_listener — port resolution, updated contract */
/* Precondition: myconf->peers_cfg is populated (possibly empty, iff
 * --peers was never passed). */
/* Postcondition, --peers never passed (peers_cfg.candidates_count == 0):
 * returns 0 with the listener bound to the hardcoded default (9990).
 * Always succeeds. An operator wanting a non-default port here passes
 * a single self-referencing --peers entry instead (see below).
 * Postcondition, --peers given: if the local name is matched with
 * a nonzero port, binds to it (today's unchanged happy path — this
 * is also how a single-node operator gets a custom port). If
 * matched with a zero port (malformed entry), falls through to the
 * hardcoded-default resolution. If the local name is NOT matched at
 * all, returns -EINVAL and refuses to start — unchanged from today,
 * preserved deliberately (see System Flows "Error path is preserved"
 * decision). */
int cluster_open_listener(cluster_t *c);

• Preconditions: myconf->peers_cfg populated (empty iff --peers never passed).
• Postconditions: see the branches above — no---peers and matched-with-zero-port always resolve to a bound listener; --peers-given-but-unmatched still returns -EINVAL.
• Invariants: precedence order (--peers match > hardcoded default) never changes based on runtime state; the -EINVAL case is determined solely by "was --peers passed AND local name absent," never by port value.

Implementation Notes

• Integration: the match/no-match branch structure is unchanged from today's cluster_open_listener; only the final fallback's source changed (from cluster_config->port to a hardcoded XCE_CONTROLLER_DEFAULT_PORT constant). No new CLI flag, no new myconf_t field — see research.md's "Design Revision" section for why this replaced the originally-designed --port flag.
• Validation: no new validation needed — the hardcoded constant requires no runtime validation, and --peers candidate validation is unchanged.
• Risks: none identified beyond what Requirement 5.5's live single-node test is designed to catch — this is the one path that removes a previously-live capability, and this component IS the requirement's specified replacement. Requirement 5.5's live verification must cover BOTH the true zero---peers case AND the self-referencing single-candidate --peers case (the latter is now the sole way to get a non-default single-node port, so it's load-bearing, not incidental). The preserved -EINVAL path for "peers given, no match" should also get its own live/unit test coverage during tasks (not separately required by requirements.md, but a natural extension of Requirement 5.5's spirit — flag during task generation).

Error Handling

Error Strategy

No new error categories are introduced, and the existing -EINVAL error path (cluster_open_listener's "no port could be determined," cluster.c:1226 today) is preserved, not removed — corrected during design review, since single-code-path simplicity would have traded a clear startup error for a silent wrong-default in a likely-misconfiguration scenario (--peers given, local name absent). It now applies to exactly one case: --peers was passed and the local name isn't in it. The genuinely-new case (--peers never passed at all, i.e. single-node) and the pre-existing-tolerated case (name matched, that entry's port is 0) both resolve via the hardcoded-default fallback and never error — unchanged from today's tolerance for the latter, newly-defined for the former. All other error handling (schema decode failures, capnp field-absence for legacy blobs) is capnp's existing default behavior: an old cluster.bin with data in now-unused field slots decodes without error, and those values are simply never read (Requirement 4.1) — no new error branch needed for that case either.

Monitoring

No new logging beyond what already exists for --peers parsing (main.c:284-291) and controller startup (INFO/WARN/ERROR via the existing mylog macros). No new metrics.

Testing Strategy

• Unit: build-time enforcement is the primary "test" for Requirements 1.1-1.2, 1.5-1.6, 2.1, 2.4 — a removed field/struct member that's still referenced anywhere is a compile error, not a silent gap (this is why Requirement 5.1's test-fixture fix is explicit rather than assumed).
• Unit: controller_unit_test full suite run (Requirement 5.2) — existing coverage for cluster_build_config, init_controller_tunables, and the two touched test files, now compiling against the shrunk struct.
• Integration: regenerate the ha/data/cluster.bin fixture (Requirement 3.2, 5.3) — since retained-field renumbering breaks binary compatibility with any pre-spec blob, decode the current fixture's semantic field values under the pre-spec schema and re-encode them under the new schema with a small throwaway tool; confirm the regenerated blob decodes correctly under the post-spec schema and deploy-ha.sh's verbatim-copy usage keeps working unmodified.
• Live/E2E: 3-node HA cluster deploy + config-push cycle (Requirement 5.4) — using the regenerated fixture; peer discovery via --peers, listener binding, worker-liveness tracking all behave identically to pre-spec.
• Live/E2E: single-node controller startup with no --peers flag, AND with a single self-referencing --peers entry for a custom port (Requirement 5.5) — the actual regression risk this spec's design resolves; must be verified live, not inferred from the 3-node soak (single-node never exercises the --peers-match branch at all).
• Live/E2E (optional, Requirement 6): if the nice-to-have CLI tunable flags are implemented, verify each flag overrides its corresponding hardcoded default and that omitting all of them reproduces today's exact hardcoded values.

Migration Strategy

No data migration is required for the fields this spec targets — Requirement 4.2 explicitly rules out a deprecate-in-place transition, and Requirement 4.1 confirms retained-field renumbering is an accepted, unguarded compatibility break for this pre-1.0 product: a cluster.bin built before this spec is not expected to be usable as-is against a controller built after it, and no backward-read path is supported or verified for such an artifact. The one thing this spec DOES migrate is the checked-in ha/data/cluster.bin test fixture (Requirement 3.2, 5.3): its retained-field values are decoded under the pre-spec schema and re-encoded under the new schema with a small throwaway tool, so /xce-deploy-ha and dependent live-verification tooling keep working. There is no forward path either (a post-spec cluster.bin is not expected to be readable by a pre-spec controller, but that direction isn't a supported downgrade path in this project regardless of this spec).

Supporting References

• src/docs/cluster-config-split-analysis.md — prior team analysis; primary design input for the dead-field classification.
• .kiro/specs/clusterconfig-schema-split/research.md — port-replacement-mechanism investigation, alternatives considered, and the decision rationale in full.
• .kiro/steering/config-design.md — the general shared-vs-local config rule this spec enforces.