From f5a762c3f7e84672007cdf9f77c4cbfc8e0ac372 Mon Sep 17 00:00:00 2001 From: Rongsong Shen Date: Sat, 3 Oct 2026 08:23:32 +0000 Subject: [PATCH] docs(controller-peers-env-fallback): document XCE_CONTROLLER_PEERS in the build/deployment steering (task 6.1) Co-Authored-By: Claude Opus 5.5 (1M context) --- .../controller-peers-env-fallback/tasks.md | 2 +- .kiro/steering/build-deployment.md | 40 +++++++++++++++++++ 2 files changed, 41 insertions(+), 1 deletion(-) diff --git a/.kiro/specs/controller-peers-env-fallback/tasks.md b/.kiro/specs/controller-peers-env-fallback/tasks.md index f5e408ed..d60b2fdd 100644 --- a/.kiro/specs/controller-peers-env-fallback/tasks.md +++ b/.kiro/specs/controller-peers-env-fallback/tasks.md @@ -114,7 +114,7 @@ - _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 4.1, 4.2, 4.3_ - [ ] 6. Documentation -- [ ] 6.1 (P) Document the variable in the build/deployment steering +- [x] 6.1 (P) Document the variable in the build/deployment steering - Describe which components read it, its value format, its precedence against each explicit value, the one-invocation opt-out, the controller's own-name rule, and that `sudo` and systemd do not pass it on (with `--preserve-env` and `Environment=`) - Done when the steering document has a section covering every item above - _Requirements: 3.1_ diff --git a/.kiro/steering/build-deployment.md b/.kiro/steering/build-deployment.md index b4cc80e9..906d9b95 100644 --- a/.kiro/steering/build-deployment.md +++ b/.kiro/steering/build-deployment.md @@ -137,6 +137,46 @@ kill -9 $(pgrep xce_scheduler) 2>/dev/null || true # Then re-start all nodes (and worker/scheduler, if you had them running) ``` +## Controller Peer List from the Environment (`XCE_CONTROLLER_PEERS`) + +`XCE_CONTROLLER_PEERS` supplies the controller candidate list to any component started without an explicit one, so it does not have to be repeated on every command line. + +**Value format**: `[name=]host:port,...`, the same as `--peers` (e.g. `ctrl-1=node1:9991,ctrl-2=node2:9991,ctrl-3=node3:9991`). An empty value is the same as unset for every component. + +**Who reads it, and what wins**: + +| Component | Explicit value (always wins) | Used when the explicit value is absent | Neither present | +|-----------|------------------------------|----------------------------------------|-----------------| +| `xce_controller` | `--peers` | `XCE_CONTROLLER_PEERS` | standalone single-node primary | +| `xce_scheduler` | `--peers` / `-P` | `XCE_CONTROLLER_PEERS` (not read in `--sim` mode) | single-controller path | +| `xcecd` / `xcecd_virtual` | `--peers` | `XCE_CONTROLLER_PEERS` | single-controller path (`localhost:9990`) | +| `xce_test` (harness) | `set masters` (typed, scripted, or in the rc file) | `XCE_CONTROLLER_PEERS` as the `masters` default, read once at startup | `localhost:9990` | +| `xceadmin` (`--controller` options) | `--controller` | `XCE_CONTROLLER_PEERS`, then the legacy `XCEADMIN_CONTROLLER` | `localhost:9991` | + +xceadmin strips any `name=` prefix from each entry and treats the list as failover candidates. When a list came from the variable, the daemons' "HA mode"/"HA cluster config" log lines end with ` (from XCE_CONTROLLER_PEERS)`, and parse/validation errors name `XCE_CONTROLLER_PEERS` instead of `--peers`. xceadmin's controller errors end with ` (from $XCE_CONTROLLER_PEERS)` (or ` (from $XCEADMIN_CONTROLLER)`) when the value came from that variable. + +**Opting out for one invocation**: unset or empty it on that command, e.g. `XCE_CONTROLLER_PEERS= xce_controller ...` (same for `xce_scheduler`, `xcecd`, `xce_test`). Passing an explicit `--peers`/`set masters`/`--controller` also overrides it. + +**Controller own-name rule**: a list taken from the variable must contain a candidate whose name (or, for an entry without `name=`, whose host) equals the controller's `--myname`, which defaults to the host name. Otherwise the controller refuses to start, before any network activity: +``` +xce_controller: XCE_CONTROLLER_PEERS () has no candidate named ''; pass --myname or unset XCE_CONTROLLER_PEERS to run standalone +``` +This stops a stray variable from joining a controller to a cluster it is not part of. An explicit `--peers` keeps its existing handling. Even-count and parse-error checks apply to both sources. + +**`sudo` and systemd do not pass it on**: `sudo` drops it under the default `env_reset`, and a systemd service never sees the login shell's environment. Pass it explicitly: +```bash +sudo --preserve-env=XCE_CONTROLLER_PEERS xcecd ... +``` +```ini +# in the unit's [Service] section +Environment=XCE_CONTROLLER_PEERS=ctrl-1=node1:9991,ctrl-2=node2:9991,ctrl-3=node3:9991 +``` +This matters most for the production worker (`xcecd`), which runs as root and so is usually started through `sudo` or a unit. If the variable was dropped, the component logs `No --peers flag and XCE_CONTROLLER_PEERS unset or empty; ...` at startup and takes the standalone/single-controller path. That line means it never received the list. + +**`deploy-ha.sh` / `/xce-deploy-ha`**: the generated `start.sh`, `start-worker.sh` and `start-scheduler.sh` still pass `--peers` explicitly, so the variable has no effect on them. The generated `scripts/xceadmin` wrapper exports `XCE_CONTROLLER_PEERS` set to the deployment's own full peer list (plus `XCEADMIN_CONTROLLER` set to its first candidate), so a value left in the caller's shell for another cluster cannot redirect it. + +**Regression check**: `src/tools/verify-controller-peers-env.sh` covers the variable's handling in the controller, scheduler, worker and harness, plus the `deploy-ha.sh` xceadmin wrapper (xceadmin's own precedence is covered by its pytest suite). + ## Go Frontend Build Go builds use Meson for both development and production — apiserver and xcecli have no separate build system: