# Brief: controller-peers-env-fallback ## Problem Operators and deploy/test scripts must retype the full `--peers` candidate list (or, for xceadmin, `--controller`) on every single invocation of every XCE component -- there is no way to set a default once per session/ environment. xceadmin already solved this for its own `--controller` flag (`XCEADMIN_CONTROLLER` env var, bug-1566), but the four `--peers`-based daemons (controller, scheduler, worker, harness/xce_test) have no equivalent: `--peers` is optional (omitting it runs standalone/non-HA), but there is no way to supply a default candidate list via the environment for someone who wants the convenience of not retyping it, as opposed to someone deliberately omitting it to opt out of HA. ## Current State - xceadmin (Python, click): `--controller` already falls back to `$XCEADMIN_CONTROLLER`, then a hardcoded `localhost:9991` default. Fully implemented and tested (bug-1566), used consistently by every `--controller`-accepting subcommand (`config push`, `config probe`, `scheduler record start/stop/status`) -- confirmed via source read, `src/xceadmin/src/xceadmin/cli.py`. - **Correction, discovered mid-session (bug-1821, 2026-09-17)**: `--controller` is NOT single-target-only -- it already accepts a comma-separated list of bare `host:port` candidates (`xceadmin.discovery.resolve_controllers()` -> `follow_redirects_any()`), tried in order until one is reachable, mirroring the client-side master-rotation failover `xcecli`/`xce_test` already use. `resolve_controllers()` does a plain `part.rsplit(":", 1)` per comma-separated entry with **no** `name=` prefix handling -- a `name=host:port`-shaped entry (the `--peers`/`xce_parse_peers()` format) would mis-parse as a literal (unresolvable) hostname `"name=host"`. This directly shapes the merge design below: xceadmin cannot consume a `name=host:port,...` value as-is, but stripping the `name=` prefixes and handing the WHOLE resulting candidate list to its own existing `resolve_controllers`/`follow_redirects_any` gets proper multi-candidate failover for free, rather than arbitrarily trusting just one entry. - controller, scheduler, worker (C, argp/getopt_long) and harness/xce_test (C, custom `flagspec.c` parser) each take an optional `--peers ` flag with **no** environment-variable fallback at all -- confirmed via source grep, zero `getenv()` calls for this anywhere in these four components. - All four of these C binaries already link `libxce` (`src/lib/`) -- confirmed via each component's `meson.build` (`libxce_dep`). - apiserver (Go) takes its controller peer list from a YAML config file (`app.yaml`'s `backend.peers`), not a CLI flag -- no `--peers`/ `--controller` equivalent exists there. - xcecli (Go) takes `--url` to reach the apiserver -- a different target ("which apiserver") than "which controller(s)" entirely. ## Desired Outcome - **A single, merged environment variable, `XCE_CONTROLLER_PEERS`**, provides the default target for all five components. Its value format is exactly what `xce_parse_peers()` already accepts (`src/lib/cluster.c`): either a bare `host:port` (single candidate, no name) or a full `name=host:port,name=host:port,...` candidate list -- confirmed by reading `xce_parse_peers()` itself, which already tolerates both shapes with no name required (a strict superset relationship, not an assumption). - controller, scheduler, worker, harness/xce_test: when `--peers` is not passed, use `$XCE_CONTROLLER_PEERS` as-is (no extra parsing needed -- it's already exactly `xce_parse_peers()`'s native input shape). Precedence: `--peers` flag > `$XCE_CONTROLLER_PEERS` > today's no-peers behavior (standalone/non-HA). - xceadmin: when `--controller` is not passed, read the *same* `$XCE_CONTROLLER_PEERS`, strip any `name=` prefix from each comma-separated entry (producing the bare `host:port,host:port,...` shape `resolve_controllers()` already natively accepts), and feed the WHOLE resulting string through xceadmin's existing `--controller` value path -- i.e. functionally as if the user had typed `--controller "host1:port1,host2:port2,..."` themselves. This inherits xceadmin's own already-tested multi-candidate failover (`resolve_controllers` + `follow_redirects_any`, bug-1821) for free, rather than arbitrarily trusting a single entry regardless of whether it's alive. The existing `XCEADMIN_CONTROLLER` env var (bare `host:port` or comma-list, unchanged) keeps working as a lower-priority legacy fallback -- no breaking change to the already-shipped, tested, `deploy-ha.sh`-integrated behavior. Precedence: `--controller` flag > `$XCE_CONTROLLER_PEERS` (name= stripped, full candidate list) > `$XCEADMIN_CONTROLLER` (legacy) > `localhost:9991` bare default. - This merges what were originally proposed as two separate new env vars (`XCE_CONTROLLER_PEERS` and `XCE_CONTROLLER`) into one, since the peers-list format is a strict superset of the single-target format -- reduces the operator-facing surface to one variable to set, shared by every component. ## Approach Add one small shared helper in `src/lib/` (all four C targets already link `libxce`) that resolves "the CLI-provided value, or else `$XCE_CONTROLLER_PEERS`" -- called once by each of `controller/main.c`, `scheduler/src/main.c`, `worker/src/main.c`, and `harness/test.c` right after their own (unchanged) argp/getopt_long/flagspec parsing of `--peers`, replacing four near-identical inline duplications with one shared, TDD-covered implementation. This helper does no format-specific parsing itself -- it just returns whichever raw string wins precedence, which then flows into each component's existing `xce_parse_peers()` call exactly as `--peers` would have. xceadmin's click-based `--controller` option gains a new small conversion helper (Python): given `$XCE_CONTROLLER_PEERS`'s raw value, strip an optional `name=` prefix from EVERY comma-delimited entry (not just the first) and return the resulting bare `host:port,host:port,...` string unchanged in shape from what `--controller`/`resolve_controllers()` already parses today. Wired in ahead of the existing `envvar="XCEADMIN_CONTROLLER"` in click's precedence chain (click's `envvar=` only reads a raw string per name, so the name-stripping conversion needs a small `callback=`/default resolution wrapper around the option rather than relying on click's built-in multi-envvar list mechanism, since the two source vars now have different value shapes and only one of them needs transformation). No change to `resolve_controllers()`/`follow_redirects_any()` themselves -- they already do exactly the right thing once handed a bare-host candidate list, regardless of whether that list came from an explicit `--controller` flag or this new env-var conversion. ## Scope - **In**: - New `XCE_CONTROLLER_PEERS` env var fallback for `--peers` in controller, scheduler, worker, harness/xce_test (used as-is, no transformation). - The same `XCE_CONTROLLER_PEERS` env var, `name=`-stripped and handed to xceadmin's existing `resolve_controllers`/`follow_redirects_any` multi-candidate failover, as a new, higher-precedence fallback for xceadmin's `--controller` option, ahead of the existing `XCEADMIN_CONTROLLER` (kept working, noted as legacy). - TDD test coverage for all five call sites (4 C components' own testsuites + xceadmin's existing click-based test file), including the `name=`-stripping conversion specifically (bare `host:port` input, `name=host:port` input, multi-candidate input with a mix of named and bare entries, and that the resulting failover still tries every candidate in order). - **Out**: - apiserver (YAML-config-based, no CLI flag for this) -- explicit user decision. - xcecli's `--url` (different target: apiserver, not controller) -- explicit user decision. - `src/tools/deploy-ha.sh` and its generated wrapper scripts -- stay exactly as they are today (explicit `--peers`/`--controller` baked into every generated script); this feature is purely additive, available for other use, not adopted into the deploy skill itself -- explicit user decision. - Any change to what happens today when `--peers`/`--controller` is omitted and no env var is set either (standalone/bare-default behavior is unchanged). ## Boundary Candidates - Shared libxce helper -- one implementation, one seam all 4 C daemons call through. - xceadmin's click option definitions -- already-existing seam in `cli.py`, just add an env var name to the existing list. - Per-component test coverage -- controller/scheduler/worker/harness each already have their own testsuite; this feature adds one test per component to each, not new test infrastructure. ## Out of Boundary - apiserver and xcecli, per user decision (different flag shapes/targets, not part of this feature). - `deploy-ha.sh` adoption, per user decision (kept purely additive; no change to existing deploy/test skill behavior). - Any redesign of what "no `--peers` and no env var" means today (standalone mode semantics untouched). ## Upstream / Downstream - **Upstream**: none -- this is additive to existing, independent argument-parsing in each of the 5 components. - **Downstream**: future deploy/test tooling (including, later, an opt-in update to `deploy-ha.sh`, explicitly deferred here) could adopt `XCE_CONTROLLER_PEERS` once this ships; not a requirement of this spec. ## Existing Spec Touchpoints - **Extends**: none directly -- `admin-config-tool` is where xceadmin's CLI (including `XCEADMIN_CONTROLLER`) originally shipped, but this change is additive/small enough not to require reopening that spec; noted here for cross-reference only. `xceadmin.discovery.resolve_controllers()`/ `follow_redirects_any()` (bug-1821, 2026-09-17, not tied to any open spec) is reused as-is, not modified -- this feature only needs to produce a value in the shape those functions already accept. - **Adjacent**: `structure.md`/`build-deployment.md` steering (component build/CLI conventions) -- no changes expected, just awareness. ## Constraints - TDD mandatory per project rules: a test in each of the 4 C components' `testsuite/` dirs, and an addition to xceadmin's existing click-based test coverage, before/alongside the implementation. - Must not change behavior when `--peers`/`--controller` IS explicitly passed on the command line (env var is purely a fallback, never an override). - Open question, not resolved in discovery: what happens for an operator who has `XCE_CONTROLLER_PEERS` set globally in their shell but wants ONE specific invocation to run standalone/non-HA? Needs an explicit answer at requirements/design time (e.g. is "unset the env var for that one invocation" considered sufficient, or does a component need an explicit opt-out?). - Naming: a single `XCE_CONTROLLER_PEERS` env var shared by all five components (merged from an earlier two-variable proposal, per explicit user decision, once `xce_parse_peers()`'s existing bare-`host:port` tolerance confirmed the peers-list format is a strict superset of the single-target format); `XCEADMIN_CONTROLLER` stays supported as a legacy, lower-priority fallback, not removed. - The `name=`-stripping conversion (xceadmin side only) must be exercised by tests covering: bare `host:port`, `name=host:port`, a multi-candidate list with a mix of named and bare entries, and that the converted value still drives `resolve_controllers`/`follow_redirects_any`'s existing in-order failover across every candidate (not just the first) -- this is new logic (not just a new `envvar=` name) and is the one place this design has room for a subtle bug (e.g. mis-splitting on an `=` inside a hostname, though none of `xce_parse_peers()`'s own accepted characters make that ambiguous in practice).