docs(controller-peers-env-fallback): note XCE_CONTROLLER_PEERS in the HA-deploy and multi-host skill docs (task 6.2)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-03 08:25:13 +00:00
parent f5a762c3f7
commit 8579e3cefc
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
3 changed files with 27 additions and 7 deletions

View file

@ -238,7 +238,12 @@ both fail over across the whole candidate list on their own), start a worker
and/or the scheduler with the **identical `--peers` configuration** the
controllers were deployed with — `deploy-ha.sh` bakes the exact same
`ctrl-1=host:port,ctrl-2=host:port,...` string into both generated scripts,
so there is nothing to keep in sync by hand:
so there is nothing to keep in sync by hand. (The daemons can also take the
list from `XCE_CONTROLLER_PEERS` when started without `--peers`, but the
generated `start.sh`/`start-worker.sh`/`start-scheduler.sh` still pass
`--peers` explicitly, so the variable has no effect on them; see
`.kiro/steering/build-deployment.md`, "Controller Peer List from the
Environment".)
```bash
"$BASH_BIN" "$TOP/scripts/start-worker.sh" worker-1 -vvv & # name defaults to worker-1
@ -327,10 +332,15 @@ a warning if no wheel exists in the same build dir as `$BINARY`, same as
- **Defaults `--controller` to this deployment's own first `--peers`
candidate**, not xceadmin's generic bare fallback (`localhost:9991`,
unrelated to any real deployment's actual ports). The wrapper exports
`XCE_CONTROLLER_PEERS="<this deployment's full --peers list>"` and
`XCEADMIN_CONTROLLER="<first --peers candidate>"`, which every
`--controller`-accepting xceadmin subcommand (`config push`, `scheduler
record start`/`stop`/`status`) reads via `envvar=` fallback — an explicit
`--controller host:port` on the command line still overrides it. Before
record start`/`stop`/`status`) reads via `envvar=` fallback, in that
order — an explicit `--controller host:port` on the command line still
overrides both. Pinning `XCE_CONTROLLER_PEERS` means a value left in your
shell for another cluster cannot redirect this wrapper (see
`.kiro/steering/build-deployment.md`, "Controller Peer List from the
Environment"). Before
this (bug-1566), a bare `scripts/xceadmin scheduler record status` with no
`--controller` silently tried port 9991 and failed with a connection-
refused that had nothing to do with the actual deployment.
@ -515,7 +525,7 @@ If you need a full redeploy (different peers or new binary), run steps 3–5 ins
| `grep`/`tail` against `$TOP/logs/xcecd.log` says `Permission denied` | Confirmed live: the real worker's log file comes out `root:root`, mode `640` — not `chmod`'d permissive like `xcecd_worker.log`/the domain socket are | Prefix with `sudo` (`sudo grep ...`/`sudo tail ...`) |
| Deploy picked the wrong worker binary (real when you wanted virtual, or vice versa) | Auto-detection follows `/run/systemd/system`, which may not match what you actually want on this host | Set `XCE_REAL_WORKER=0` or `XCE_REAL_WORKER=1` before Step 3 to force it, then redeploy |
| A generated script fails with `bad interpreter` or silently uses the wrong bash | Redeploying with a stale `$BASH_BIN` captured outside `nix develop`, or a shell that shadows `bash` | Re-resolve `BASH_BIN=$(nix develop ./src --command which bash)` right before Step 3 and re-deploy |
| `scripts/xceadmin ...` (no `--controller`) fails `cannot connect to localhost:9991: Connection refused` | bug-1566, fixed — a wheel built before this fix, or a `deploy-ha.sh` from before it, doesn't export `XCEADMIN_CONTROLLER` | Rebuild (`/xce-build xceadmin --incremental`) and redeploy (Step 3); check `$TOP/scripts/xceadmin` — it should `export XCEADMIN_CONTROLLER="<first --peers candidate>"` before the `exec` line |
| `scripts/xceadmin ...` (no `--controller`) fails `cannot connect to localhost:9991: Connection refused` | bug-1566, fixed — a wheel built before this fix, or a `deploy-ha.sh` from before it, doesn't export `XCEADMIN_CONTROLLER` | Rebuild (`/xce-build xceadmin --incremental`) and redeploy (Step 3); check `$TOP/scripts/xceadmin` — it should `export XCEADMIN_CONTROLLER="<first --peers candidate>"` and `export XCE_CONTROLLER_PEERS="<full --peers list>"` before the `exec` line (if only the first is there, an ambient `XCE_CONTROLLER_PEERS` in your shell takes precedence and points it elsewhere — redeploy) |
| `scripts/xceadmin` not generated | No wheel in `$BINARY`'s build dir (`deploy-ha.sh` logs a `Warning:`, not an error) | `/xce-build xceadmin` (add `--incremental` if you don't want to lose other already-built binaries in that dir — see `/xce-build`'s SKILL.md), then redeploy (Step 3) |
| `scripts/start-apiserver.sh` not generated | No binary at `<build dir>/frontend/apiserver/xce_apiserver` or `/tmp/apiserver` (`deploy-ha.sh` logs a `Warning:`, not an error) | `/xce-build apiserver`, then redeploy (Step 3) — or place a binary at `/tmp/apiserver` (legacy path, not guaranteed to be the same build generation) |
| Asked to use `xce_test` against this deployment but `<build dir>/harness/xce_test` doesn't exist | Harness never built — `/xce-build controller` no longer produces `xce_test` as a side effect, and `deploy-ha.sh` doesn't build or reference it either (see Prerequisites) | `/xce-build harness`, then invoke `<build dir>/harness/xce_test` directly against this deployment's `--peers` (no wrapper script is generated for it, unlike worker/scheduler/xceadmin) |

View file

@ -66,7 +66,10 @@ Traces to `.kiro/specs/resreq-maxtasks-wiring/design.md`'s
named workers individually via the scripts `deploy-ha.sh` generates
(`start.sh ctrl-1`, `start-scheduler.sh`, `start-worker.sh <name>` twice
with two different names) -- deliberately NOT `start-all.sh`, which only
ever starts a single worker hardcoded to the name `worker-1`. A
ever starts a single worker hardcoded to the name `worker-1`. (Those
generated scripts pass `--peers` explicitly, so `XCE_CONTROLLER_PEERS`
-- the daemons' environment fallback, see
`.kiro/steering/build-deployment.md` -- has no effect on this run.) A
`trap`-based cleanup is registered before the first process starts, so it
fires on every exit path (a completed run, a mid-deployment abort, or an
external SIGINT/SIGTERM), matching `xce-test-resources`'s convention.
@ -244,6 +247,10 @@ processor/slot capacity: `worker/src/main.c`'s `getopt_long()` table is
`xcecd_virtual` means it skips real job execution/cgroups, **not** that it
fakes a different processor count -- each worker reports the real host's
actual core count (confirmed live in this sandbox: `nproc` reports 22).
(`--peers` can also come from `XCE_CONTROLLER_PEERS` when the flag is
absent -- see `.kiro/steering/build-deployment.md` -- but this skill's
generated `start-worker.sh` always passes `--peers`, so that does not
apply here.)
`compute_task_count()` in `run-multihost-verify.sh` therefore queries
`nproc` live at test-run time (never hardcoded) and picks the test job's
@ -350,7 +357,10 @@ allocation decision, not a copy of the plain job's.
- `xce-deploy-ha`'s `deploy-ha.sh` and its generated `start.sh` /
`start-scheduler.sh` / `start-worker.sh` (single-controller-node path,
two named worker instances against the same `--peers` config).
two named worker instances against the same `--peers` config). These
scripts, and this script's own `xce_test` sessions (`set masters`), pass
the controller list explicitly, so `XCE_CONTROLLER_PEERS` (see
`.kiro/steering/build-deployment.md`) does not change this verification.
- Controller, scheduler, worker, and harness (`xce_test`) binaries in the
same build dir (see Prerequisites above).

View file

@ -120,7 +120,7 @@
- _Requirements: 3.1_
- _Boundary: steering docs_
- [ ] 6.2 (P) Note the variable in the HA-deploy and multi-host skill docs
- [x] 6.2 (P) Note the variable in the HA-deploy and multi-host skill docs
- HA-deploy skill: the variable's availability, that generated daemon scripts still pass `--peers`, and that the xceadmin wrapper now exports the variable (description and troubleshooting row)
- Multi-host skill: the same availability note at its three `--peers` locations, without implying any change to its generated invocations
- Done when both skill documents carry the notes at the listed locations