myxce/.kiro/specs/controller-peers-env-fallback/brief.md

# Brief: controller-peers-env-fallback

## Problem

Operators and deploy/test scripts must retype the full `--peers` candidate
list (or, for xceadmin, `--controller`) on every single invocation of every
XCE component -- there is no way to set a default once per session/
environment. xceadmin already solved this for its own `--controller` flag
(`XCEADMIN_CONTROLLER` env var, bug-1566), but the four `--peers`-based
daemons (controller, scheduler, worker, harness/xce_test) have no
equivalent: `--peers` is optional (omitting it runs standalone/non-HA), but
there is no way to supply a default candidate list via the environment for
someone who wants the convenience of not retyping it, as opposed to someone
deliberately omitting it to opt out of HA.

## Current State

- xceadmin (Python, click): `--controller` already falls back to
  `$XCEADMIN_CONTROLLER`, then a hardcoded `localhost:9991` default. Fully
  implemented and tested (bug-1566), used consistently by every
  `--controller`-accepting subcommand (`config push`, `config probe`,
  `scheduler record start/stop/status`) -- confirmed via source read,
  `src/xceadmin/src/xceadmin/cli.py`.
- **Correction, discovered mid-session (bug-1821, 2026-09-17)**: `--controller`
  is NOT single-target-only -- it already accepts a comma-separated list of
  bare `host:port` candidates (`xceadmin.discovery.resolve_controllers()` ->
  `follow_redirects_any()`), tried in order until one is reachable, mirroring
  the client-side master-rotation failover `xcecli`/`xce_test` already use.
  `resolve_controllers()` does a plain `part.rsplit(":", 1)` per
  comma-separated entry with **no** `name=` prefix handling -- a
  `name=host:port`-shaped entry (the `--peers`/`xce_parse_peers()` format)
  would mis-parse as a literal (unresolvable) hostname `"name=host"`. This
  directly shapes the merge design below: xceadmin cannot consume a
  `name=host:port,...` value as-is, but stripping the `name=` prefixes and
  handing the WHOLE resulting candidate list to its own existing
  `resolve_controllers`/`follow_redirects_any` gets proper multi-candidate
  failover for free, rather than arbitrarily trusting just one entry.
- controller, scheduler, worker (C, argp/getopt_long) and harness/xce_test
  (C, custom `flagspec.c` parser) each take an optional
  `--peers <name=host:port,...>` flag with **no** environment-variable
  fallback at all -- confirmed via source grep, zero `getenv()` calls for
  this anywhere in these four components.
- All four of these C binaries already link `libxce` (`src/lib/`) --
  confirmed via each component's `meson.build` (`libxce_dep`).
- apiserver (Go) takes its controller peer list from a YAML config file
  (`app.yaml`'s `backend.peers`), not a CLI flag -- no `--peers`/
  `--controller` equivalent exists there.
- xcecli (Go) takes `--url` to reach the apiserver -- a different target
  ("which apiserver") than "which controller(s)" entirely.

## Desired Outcome

- **A single, merged environment variable, `XCE_CONTROLLER_PEERS`**,
  provides the default target for all five components. Its value format is
  exactly what `xce_parse_peers()` already accepts (`src/lib/cluster.c`):
  either a bare `host:port` (single candidate, no name) or a full
  `name=host:port,name=host:port,...` candidate list -- confirmed by
  reading `xce_parse_peers()` itself, which already tolerates both shapes
  with no name required (a strict superset relationship, not an assumption).
- controller, scheduler, worker, harness/xce_test: when `--peers` is not
  passed, use `$XCE_CONTROLLER_PEERS` as-is (no extra parsing needed --
  it's already exactly `xce_parse_peers()`'s native input shape).
  Precedence: `--peers` flag > `$XCE_CONTROLLER_PEERS` > today's no-peers
  behavior (standalone/non-HA).
- xceadmin: when `--controller` is not passed, read the *same*
  `$XCE_CONTROLLER_PEERS`, strip any `name=` prefix from each
  comma-separated entry (producing the bare `host:port,host:port,...` shape
  `resolve_controllers()` already natively accepts), and feed the WHOLE
  resulting string through xceadmin's existing `--controller` value path --
  i.e. functionally as if the user had typed
  `--controller "host1:port1,host2:port2,..."` themselves. This inherits
  xceadmin's own already-tested multi-candidate failover
  (`resolve_controllers` + `follow_redirects_any`, bug-1821) for free,
  rather than arbitrarily trusting a single entry regardless of whether
  it's alive. The existing `XCEADMIN_CONTROLLER` env var (bare `host:port`
  or comma-list, unchanged) keeps working as a lower-priority legacy
  fallback -- no breaking change to the already-shipped, tested,
  `deploy-ha.sh`-integrated behavior. Precedence: `--controller` flag >
  `$XCE_CONTROLLER_PEERS` (name= stripped, full candidate list) >
  `$XCEADMIN_CONTROLLER` (legacy) > `localhost:9991` bare default.
- This merges what were originally proposed as two separate new env vars
  (`XCE_CONTROLLER_PEERS` and `XCE_CONTROLLER`) into one, since the
  peers-list format is a strict superset of the single-target format --
  reduces the operator-facing surface to one variable to set, shared by
  every component.

## Approach

Add one small shared helper in `src/lib/` (all four C targets already link
`libxce`) that resolves "the CLI-provided value, or else `$XCE_CONTROLLER_PEERS`"
-- called once by each of `controller/main.c`, `scheduler/src/main.c`,
`worker/src/main.c`, and `harness/test.c` right after their own (unchanged)
argp/getopt_long/flagspec parsing of `--peers`, replacing four
near-identical inline duplications with one shared, TDD-covered
implementation. This helper does no format-specific parsing itself -- it
just returns whichever raw string wins precedence, which then flows into
each component's existing `xce_parse_peers()` call exactly as `--peers`
would have.

xceadmin's click-based `--controller` option gains a new small conversion
helper (Python): given `$XCE_CONTROLLER_PEERS`'s raw value, strip an
optional `name=` prefix from EVERY comma-delimited entry (not just the
first) and return the resulting bare `host:port,host:port,...` string
unchanged in shape from what `--controller`/`resolve_controllers()` already
parses today. Wired in ahead of the existing `envvar="XCEADMIN_CONTROLLER"`
in click's precedence chain (click's `envvar=` only reads a raw string per
name, so the name-stripping conversion needs a small `callback=`/default
resolution wrapper around the option rather than relying on click's
built-in multi-envvar list mechanism, since the two source vars now have
different value shapes and only one of them needs transformation). No
change to `resolve_controllers()`/`follow_redirects_any()` themselves --
they already do exactly the right thing once handed a bare-host candidate
list, regardless of whether that list came from an explicit `--controller`
flag or this new env-var conversion.

## Scope

- **In**:
  - New `XCE_CONTROLLER_PEERS` env var fallback for `--peers` in
    controller, scheduler, worker, harness/xce_test (used as-is, no
    transformation).
  - The same `XCE_CONTROLLER_PEERS` env var, `name=`-stripped and handed to
    xceadmin's existing `resolve_controllers`/`follow_redirects_any`
    multi-candidate failover, as a new, higher-precedence fallback for
    xceadmin's `--controller` option, ahead of the existing
    `XCEADMIN_CONTROLLER` (kept working, noted as legacy).
  - TDD test coverage for all five call sites (4 C components' own
    testsuites + xceadmin's existing click-based test file), including the
    `name=`-stripping conversion specifically (bare `host:port` input,
    `name=host:port` input, multi-candidate input with a mix of named and
    bare entries, and that the resulting failover still tries every
    candidate in order).
- **Out**:
  - apiserver (YAML-config-based, no CLI flag for this) -- explicit user
    decision.
  - xcecli's `--url` (different target: apiserver, not controller) --
    explicit user decision.
  - `src/tools/deploy-ha.sh` and its generated wrapper scripts -- stay
    exactly as they are today (explicit `--peers`/`--controller` baked into
    every generated script); this feature is purely additive, available for
    other use, not adopted into the deploy skill itself -- explicit user
    decision.
  - Any change to what happens today when `--peers`/`--controller` is
    omitted and no env var is set either (standalone/bare-default behavior
    is unchanged).

## Boundary Candidates

- Shared libxce helper -- one implementation, one seam all 4 C daemons call
  through.
- xceadmin's click option definitions -- already-existing seam in `cli.py`,
  just add an env var name to the existing list.
- Per-component test coverage -- controller/scheduler/worker/harness each
  already have their own testsuite; this feature adds one test per
  component to each, not new test infrastructure.

## Out of Boundary

- apiserver and xcecli, per user decision (different flag shapes/targets,
  not part of this feature).
- `deploy-ha.sh` adoption, per user decision (kept purely additive; no
  change to existing deploy/test skill behavior).
- Any redesign of what "no `--peers` and no env var" means today
  (standalone mode semantics untouched).

## Upstream / Downstream

- **Upstream**: none -- this is additive to existing, independent
  argument-parsing in each of the 5 components.
- **Downstream**: future deploy/test tooling (including, later, an opt-in
  update to `deploy-ha.sh`, explicitly deferred here) could adopt
  `XCE_CONTROLLER_PEERS` once this ships; not a requirement of this spec.

## Existing Spec Touchpoints

- **Extends**: none directly -- `admin-config-tool` is where xceadmin's CLI
  (including `XCEADMIN_CONTROLLER`) originally shipped, but this change is
  additive/small enough not to require reopening that spec; noted here for
  cross-reference only. `xceadmin.discovery.resolve_controllers()`/
  `follow_redirects_any()` (bug-1821, 2026-09-17, not tied to any open spec)
  is reused as-is, not modified -- this feature only needs to produce a
  value in the shape those functions already accept.
- **Adjacent**: `structure.md`/`build-deployment.md` steering (component
  build/CLI conventions) -- no changes expected, just awareness.

## Constraints

- TDD mandatory per project rules: a test in each of the 4 C components'
  `testsuite/` dirs, and an addition to xceadmin's existing click-based test
  coverage, before/alongside the implementation.
- Must not change behavior when `--peers`/`--controller` IS explicitly
  passed on the command line (env var is purely a fallback, never an
  override).
- Open question, not resolved in discovery: what happens for an operator
  who has `XCE_CONTROLLER_PEERS` set globally in their shell but wants ONE
  specific invocation to run standalone/non-HA? Needs an explicit answer at
  requirements/design time (e.g. is "unset the env var for that one
  invocation" considered sufficient, or does a component need an explicit
  opt-out?).
- Naming: a single `XCE_CONTROLLER_PEERS` env var shared by all five
  components (merged from an earlier two-variable proposal, per explicit
  user decision, once `xce_parse_peers()`'s existing bare-`host:port`
  tolerance confirmed the peers-list format is a strict superset of the
  single-target format); `XCEADMIN_CONTROLLER` stays supported as a
  legacy, lower-priority fallback, not removed.
- The `name=`-stripping conversion (xceadmin side only) must be exercised
  by tests covering: bare `host:port`, `name=host:port`, a multi-candidate
  list with a mix of named and bare entries, and that the converted value
  still drives `resolve_controllers`/`follow_redirects_any`'s existing
  in-order failover across every candidate (not just the first) -- this is
  new logic (not just a new `envvar=` name) and is the one place this
  design has room for a subtle bug (e.g. mis-splitting on an `=` inside a
  hostname, though none of `xce_parse_peers()`'s own accepted characters
  make that ambiguous in practice).