chore: persist OpenWolf state, controller-peers-env-fallback spec, and scheduler backlog notes
- .wolf/anatomy-index.json, anatomy.md, memory.md: OpenWolf hook-maintained project index and session log, refreshed by this session's activity. - .kiro/specs/controller-peers-env-fallback/: spec scaffold (brief, requirements, design, research) from prior spec-init work. - issues/new_requirements.md: scheduler simulator enhancement backlog (info-command improvements, pend-reason detail, decision tracing). Excluded as non-persistent: issues/Issues.md (empty placeholder), src/lib/testsuite/test_xcectx_reconnect_race.c (self-documented as an ASAN stress reproduction only, not a committed regression test, and not wired into meson.build), scheduler-replay-20260922-095207.replay (4MB binary debug capture, no .replay files are tracked in this repo). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
7901db57d4
commit
668e3f6893
9 changed files with 1130 additions and 41 deletions
214
.kiro/specs/controller-peers-env-fallback/brief.md
Normal file
214
.kiro/specs/controller-peers-env-fallback/brief.md
Normal file
|
|
@ -0,0 +1,214 @@
|
|||
# Brief: controller-peers-env-fallback
|
||||
|
||||
## Problem
|
||||
|
||||
Operators and deploy/test scripts must retype the full `--peers` candidate
|
||||
list (or, for xceadmin, `--controller`) on every single invocation of every
|
||||
XCE component -- there is no way to set a default once per session/
|
||||
environment. xceadmin already solved this for its own `--controller` flag
|
||||
(`XCEADMIN_CONTROLLER` env var, bug-1566), but the four `--peers`-based
|
||||
daemons (controller, scheduler, worker, harness/xce_test) have no
|
||||
equivalent: `--peers` is optional (omitting it runs standalone/non-HA), but
|
||||
there is no way to supply a default candidate list via the environment for
|
||||
someone who wants the convenience of not retyping it, as opposed to someone
|
||||
deliberately omitting it to opt out of HA.
|
||||
|
||||
## Current State
|
||||
|
||||
- xceadmin (Python, click): `--controller` already falls back to
|
||||
`$XCEADMIN_CONTROLLER`, then a hardcoded `localhost:9991` default. Fully
|
||||
implemented and tested (bug-1566), used consistently by every
|
||||
`--controller`-accepting subcommand (`config push`, `config probe`,
|
||||
`scheduler record start/stop/status`) -- confirmed via source read,
|
||||
`src/xceadmin/src/xceadmin/cli.py`.
|
||||
- **Correction, discovered mid-session (bug-1821, 2026-09-17)**: `--controller`
|
||||
is NOT single-target-only -- it already accepts a comma-separated list of
|
||||
bare `host:port` candidates (`xceadmin.discovery.resolve_controllers()` ->
|
||||
`follow_redirects_any()`), tried in order until one is reachable, mirroring
|
||||
the client-side master-rotation failover `xcecli`/`xce_test` already use.
|
||||
`resolve_controllers()` does a plain `part.rsplit(":", 1)` per
|
||||
comma-separated entry with **no** `name=` prefix handling -- a
|
||||
`name=host:port`-shaped entry (the `--peers`/`xce_parse_peers()` format)
|
||||
would mis-parse as a literal (unresolvable) hostname `"name=host"`. This
|
||||
directly shapes the merge design below: xceadmin cannot consume a
|
||||
`name=host:port,...` value as-is, but stripping the `name=` prefixes and
|
||||
handing the WHOLE resulting candidate list to its own existing
|
||||
`resolve_controllers`/`follow_redirects_any` gets proper multi-candidate
|
||||
failover for free, rather than arbitrarily trusting just one entry.
|
||||
- controller, scheduler, worker (C, argp/getopt_long) and harness/xce_test
|
||||
(C, custom `flagspec.c` parser) each take an optional
|
||||
`--peers <name=host:port,...>` flag with **no** environment-variable
|
||||
fallback at all -- confirmed via source grep, zero `getenv()` calls for
|
||||
this anywhere in these four components.
|
||||
- All four of these C binaries already link `libxce` (`src/lib/`) --
|
||||
confirmed via each component's `meson.build` (`libxce_dep`).
|
||||
- apiserver (Go) takes its controller peer list from a YAML config file
|
||||
(`app.yaml`'s `backend.peers`), not a CLI flag -- no `--peers`/
|
||||
`--controller` equivalent exists there.
|
||||
- xcecli (Go) takes `--url` to reach the apiserver -- a different target
|
||||
("which apiserver") than "which controller(s)" entirely.
|
||||
|
||||
## Desired Outcome
|
||||
|
||||
- **A single, merged environment variable, `XCE_CONTROLLER_PEERS`**,
|
||||
provides the default target for all five components. Its value format is
|
||||
exactly what `xce_parse_peers()` already accepts (`src/lib/cluster.c`):
|
||||
either a bare `host:port` (single candidate, no name) or a full
|
||||
`name=host:port,name=host:port,...` candidate list -- confirmed by
|
||||
reading `xce_parse_peers()` itself, which already tolerates both shapes
|
||||
with no name required (a strict superset relationship, not an assumption).
|
||||
- controller, scheduler, worker, harness/xce_test: when `--peers` is not
|
||||
passed, use `$XCE_CONTROLLER_PEERS` as-is (no extra parsing needed --
|
||||
it's already exactly `xce_parse_peers()`'s native input shape).
|
||||
Precedence: `--peers` flag > `$XCE_CONTROLLER_PEERS` > today's no-peers
|
||||
behavior (standalone/non-HA).
|
||||
- xceadmin: when `--controller` is not passed, read the *same*
|
||||
`$XCE_CONTROLLER_PEERS`, strip any `name=` prefix from each
|
||||
comma-separated entry (producing the bare `host:port,host:port,...` shape
|
||||
`resolve_controllers()` already natively accepts), and feed the WHOLE
|
||||
resulting string through xceadmin's existing `--controller` value path --
|
||||
i.e. functionally as if the user had typed
|
||||
`--controller "host1:port1,host2:port2,..."` themselves. This inherits
|
||||
xceadmin's own already-tested multi-candidate failover
|
||||
(`resolve_controllers` + `follow_redirects_any`, bug-1821) for free,
|
||||
rather than arbitrarily trusting a single entry regardless of whether
|
||||
it's alive. The existing `XCEADMIN_CONTROLLER` env var (bare `host:port`
|
||||
or comma-list, unchanged) keeps working as a lower-priority legacy
|
||||
fallback -- no breaking change to the already-shipped, tested,
|
||||
`deploy-ha.sh`-integrated behavior. Precedence: `--controller` flag >
|
||||
`$XCE_CONTROLLER_PEERS` (name= stripped, full candidate list) >
|
||||
`$XCEADMIN_CONTROLLER` (legacy) > `localhost:9991` bare default.
|
||||
- This merges what were originally proposed as two separate new env vars
|
||||
(`XCE_CONTROLLER_PEERS` and `XCE_CONTROLLER`) into one, since the
|
||||
peers-list format is a strict superset of the single-target format --
|
||||
reduces the operator-facing surface to one variable to set, shared by
|
||||
every component.
|
||||
|
||||
## Approach
|
||||
|
||||
Add one small shared helper in `src/lib/` (all four C targets already link
|
||||
`libxce`) that resolves "the CLI-provided value, or else `$XCE_CONTROLLER_PEERS`"
|
||||
-- called once by each of `controller/main.c`, `scheduler/src/main.c`,
|
||||
`worker/src/main.c`, and `harness/test.c` right after their own (unchanged)
|
||||
argp/getopt_long/flagspec parsing of `--peers`, replacing four
|
||||
near-identical inline duplications with one shared, TDD-covered
|
||||
implementation. This helper does no format-specific parsing itself -- it
|
||||
just returns whichever raw string wins precedence, which then flows into
|
||||
each component's existing `xce_parse_peers()` call exactly as `--peers`
|
||||
would have.
|
||||
|
||||
xceadmin's click-based `--controller` option gains a new small conversion
|
||||
helper (Python): given `$XCE_CONTROLLER_PEERS`'s raw value, strip an
|
||||
optional `name=` prefix from EVERY comma-delimited entry (not just the
|
||||
first) and return the resulting bare `host:port,host:port,...` string
|
||||
unchanged in shape from what `--controller`/`resolve_controllers()` already
|
||||
parses today. Wired in ahead of the existing `envvar="XCEADMIN_CONTROLLER"`
|
||||
in click's precedence chain (click's `envvar=` only reads a raw string per
|
||||
name, so the name-stripping conversion needs a small `callback=`/default
|
||||
resolution wrapper around the option rather than relying on click's
|
||||
built-in multi-envvar list mechanism, since the two source vars now have
|
||||
different value shapes and only one of them needs transformation). No
|
||||
change to `resolve_controllers()`/`follow_redirects_any()` themselves --
|
||||
they already do exactly the right thing once handed a bare-host candidate
|
||||
list, regardless of whether that list came from an explicit `--controller`
|
||||
flag or this new env-var conversion.
|
||||
|
||||
## Scope
|
||||
|
||||
- **In**:
|
||||
- New `XCE_CONTROLLER_PEERS` env var fallback for `--peers` in
|
||||
controller, scheduler, worker, harness/xce_test (used as-is, no
|
||||
transformation).
|
||||
- The same `XCE_CONTROLLER_PEERS` env var, `name=`-stripped and handed to
|
||||
xceadmin's existing `resolve_controllers`/`follow_redirects_any`
|
||||
multi-candidate failover, as a new, higher-precedence fallback for
|
||||
xceadmin's `--controller` option, ahead of the existing
|
||||
`XCEADMIN_CONTROLLER` (kept working, noted as legacy).
|
||||
- TDD test coverage for all five call sites (4 C components' own
|
||||
testsuites + xceadmin's existing click-based test file), including the
|
||||
`name=`-stripping conversion specifically (bare `host:port` input,
|
||||
`name=host:port` input, multi-candidate input with a mix of named and
|
||||
bare entries, and that the resulting failover still tries every
|
||||
candidate in order).
|
||||
- **Out**:
|
||||
- apiserver (YAML-config-based, no CLI flag for this) -- explicit user
|
||||
decision.
|
||||
- xcecli's `--url` (different target: apiserver, not controller) --
|
||||
explicit user decision.
|
||||
- `src/tools/deploy-ha.sh` and its generated wrapper scripts -- stay
|
||||
exactly as they are today (explicit `--peers`/`--controller` baked into
|
||||
every generated script); this feature is purely additive, available for
|
||||
other use, not adopted into the deploy skill itself -- explicit user
|
||||
decision.
|
||||
- Any change to what happens today when `--peers`/`--controller` is
|
||||
omitted and no env var is set either (standalone/bare-default behavior
|
||||
is unchanged).
|
||||
|
||||
## Boundary Candidates
|
||||
|
||||
- Shared libxce helper -- one implementation, one seam all 4 C daemons call
|
||||
through.
|
||||
- xceadmin's click option definitions -- already-existing seam in `cli.py`,
|
||||
just add an env var name to the existing list.
|
||||
- Per-component test coverage -- controller/scheduler/worker/harness each
|
||||
already have their own testsuite; this feature adds one test per
|
||||
component to each, not new test infrastructure.
|
||||
|
||||
## Out of Boundary
|
||||
|
||||
- apiserver and xcecli, per user decision (different flag shapes/targets,
|
||||
not part of this feature).
|
||||
- `deploy-ha.sh` adoption, per user decision (kept purely additive; no
|
||||
change to existing deploy/test skill behavior).
|
||||
- Any redesign of what "no `--peers` and no env var" means today
|
||||
(standalone mode semantics untouched).
|
||||
|
||||
## Upstream / Downstream
|
||||
|
||||
- **Upstream**: none -- this is additive to existing, independent
|
||||
argument-parsing in each of the 5 components.
|
||||
- **Downstream**: future deploy/test tooling (including, later, an opt-in
|
||||
update to `deploy-ha.sh`, explicitly deferred here) could adopt
|
||||
`XCE_CONTROLLER_PEERS` once this ships; not a requirement of this spec.
|
||||
|
||||
## Existing Spec Touchpoints
|
||||
|
||||
- **Extends**: none directly -- `admin-config-tool` is where xceadmin's CLI
|
||||
(including `XCEADMIN_CONTROLLER`) originally shipped, but this change is
|
||||
additive/small enough not to require reopening that spec; noted here for
|
||||
cross-reference only. `xceadmin.discovery.resolve_controllers()`/
|
||||
`follow_redirects_any()` (bug-1821, 2026-09-17, not tied to any open spec)
|
||||
is reused as-is, not modified -- this feature only needs to produce a
|
||||
value in the shape those functions already accept.
|
||||
- **Adjacent**: `structure.md`/`build-deployment.md` steering (component
|
||||
build/CLI conventions) -- no changes expected, just awareness.
|
||||
|
||||
## Constraints
|
||||
|
||||
- TDD mandatory per project rules: a test in each of the 4 C components'
|
||||
`testsuite/` dirs, and an addition to xceadmin's existing click-based test
|
||||
coverage, before/alongside the implementation.
|
||||
- Must not change behavior when `--peers`/`--controller` IS explicitly
|
||||
passed on the command line (env var is purely a fallback, never an
|
||||
override).
|
||||
- Open question, not resolved in discovery: what happens for an operator
|
||||
who has `XCE_CONTROLLER_PEERS` set globally in their shell but wants ONE
|
||||
specific invocation to run standalone/non-HA? Needs an explicit answer at
|
||||
requirements/design time (e.g. is "unset the env var for that one
|
||||
invocation" considered sufficient, or does a component need an explicit
|
||||
opt-out?).
|
||||
- Naming: a single `XCE_CONTROLLER_PEERS` env var shared by all five
|
||||
components (merged from an earlier two-variable proposal, per explicit
|
||||
user decision, once `xce_parse_peers()`'s existing bare-`host:port`
|
||||
tolerance confirmed the peers-list format is a strict superset of the
|
||||
single-target format); `XCEADMIN_CONTROLLER` stays supported as a
|
||||
legacy, lower-priority fallback, not removed.
|
||||
- The `name=`-stripping conversion (xceadmin side only) must be exercised
|
||||
by tests covering: bare `host:port`, `name=host:port`, a multi-candidate
|
||||
list with a mix of named and bare entries, and that the converted value
|
||||
still drives `resolve_controllers`/`follow_redirects_any`'s existing
|
||||
in-order failover across every candidate (not just the first) -- this is
|
||||
new logic (not just a new `envvar=` name) and is the one place this
|
||||
design has room for a subtle bug (e.g. mis-splitting on an `=` inside a
|
||||
hostname, though none of `xce_parse_peers()`'s own accepted characters
|
||||
make that ambiguous in practice).
|
||||
Loading…
Add table
Add a link
Reference in a new issue