fix(resource-value-verification): close bug-1579 for real via xce_test's ! shell-escape

Previously worked around (task 6.2) by removing the one host-attrs resource
(cpuf) from the default set - a custom resource=value arg using any
host-attrs name would still false-mismatch at the controller layer, since
inject_resources() disconnected the worker before the separate controller-
check session ran, and the controller resets host-attrs resources on
disconnect.

Real fix: xce_test's `!<shell-command>` escape is handled in the single
top-level REPL loop before context-specific dispatch, so it works in any
context - including worker mode - without calling leave_ctx()/disconnecting.
inject_resources() now shells out via `!` to run the controller check
in-line, between resupdate and quit, while the worker's connection is still
open. query_controller_resvals()'s comparison logic is unchanged - this is
a pure sequencing fix. cpuf restored to DEFAULT_RESOURCE_ARGS as live proof.

Live-verified: cpuf now MATCHes at the controller layer (was always a false
MISMATCH before); adversarial shell-injection attempt via a malicious
resource name confirmed safe (printf %q quoting); SIGTERM delivered exactly
during the nested check confirmed zero orphaned/zombie processes across 4
independent runs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-08-18 11:03:44 +00:00
parent 792dfdf4df
commit b673b7c933
Signed by: rshen
SSH key fingerprint: SHA256:8Vf2pZ3h6rJa7EhGK+WaAQ9Mta/apZvteW6g9U/WKKk
2 changed files with 234 additions and 81 deletions

View file

@ -116,7 +116,8 @@
- _Depends: 7.1_
## Implementation Notes
- Task 6.3 confirmed (live, via a `cpuf` probe: controller MISMATCH + scheduler MATCH simultaneously) that the scheduler check is genuinely immune to bug-1579 — it reads task 4.1's trace log (written once, at apply-time, in the `RESVALS_DELTA` case only), never touched by a later worker-disconnect's `WORKERS_PUSH`-driven reset. **Real fix for bug-1579 identified, not yet applied** (see below) — the `!<shell-command>` escape in `xce_test` (`preprocess_and_run()`, `test.c` ~line 965) works in ANY context including `worker` mode, since it's handled in the single top-level REPL loop before context dispatch, not per-context. This means `inject_resources()` can stay in worker context (connection held open) after injecting, shell out via `!` to a separate `xce_test` client-mode process for the controller query, and only `quit` at the very end — fully closing bug-1579 (not just working around it in the default set) rather than needing a C-level `xce_test` change. Plan: retrofit this into `inject_resources()`/`query_controller_resvals()` as a follow-up before/alongside task 7.1, then restore a host-attrs resource (e.g. `cpuf`) to `DEFAULT_RESOURCE_ARGS` to prove it.
- **bug-1579 is now FULLY fixed** (retrofit on top of tasks 6.1/6.2, not a numbered task — user-requested follow-up). `inject_resources()`'s worker-context `xce_test` script now inserts a `!<shell-command>` line between `resupdate` and `quit`: `! bash <script> --internal-controller-check <top> <peers> <worker-host> <names...>` — this runs the controller-check query via `xce_test`'s `!` shell-escape (`preprocess_and_run()`, handled in the single top-level REPL loop before context dispatch, so it works in ANY context including `worker` mode without disconnecting). The controller check now runs WHILE the injecting worker's connection is still open, so host-attrs resources are never reset before being checked. `DEFAULT_RESOURCE_ARGS` now includes `cpuf` again (proof the fix works: `cpuf` now MATCHes at the controller layer, not the old false MISMATCH). `query_controller_resvals()`'s comparison logic (`controller_values_match()`) is confirmed byte-for-byte unchanged — this was a pure sequencing fix. Any future task touching `inject_resources()`/the controller-check flow should be aware of this nested-process structure (`--internal-controller-check` is a hidden internal dispatch flag, gated before `main()`/`parse_args()` runs — do not remove it thinking it's dead code).
- Non-blocking notes from the retrofit's review: `internal_controller_check()` queries ALL requested resource names (including ones that will later be rejected), not just accepted ones like the original design — forced by sequencing (accept/reject isn't known until the whole injecting session's output is parsed after it returns); confirmed benign, the final `controller-check-results.tsv` still correctly omits rejected names. Also: invoking `--internal-controller-check` directly with fewer than 3 args hits `set -u` and dies with a raw "unbound variable" message instead of a clean `fail()` — cosmetic only, this path is unreachable from any normal invocation.
- Minor non-blocking follow-up from task 6.3's review: `poll_scheduler_log()`'s output isn't filtered to the caller's `pending` list (harmless today since the only caller already filters, but worth tightening).
- Task 6.2 (round 1 REJECTED, round 2 fixed) discovered **bug-1579**: any host-attrs-derived resource (`type`/`model`/`cpuf`/`maxcpus`/`maxmem`/`maxswap`/`maxtmp`/`ndisks` — `worker.c`'s `N_HOST_ATTRS`) gets reset to baseline by the controller the instant the synthetic worker disconnects, and `inject_resources()` (6.1) always disconnects (`quit`) before each layer's separate check session runs — so a controller-layer check on any host-attrs name will ALWAYS false-MISMATCH under the current two-session structure. Fixed for the default set by swapping `DEFAULT_RESOURCE_ARGS`'s `cpuf` for `r1m` (periodic/dynamic, confirmed safe). **This risk is NOT fully closed** — a user-supplied custom `resource=value` argument using any host-attrs name will still hit this. A real fix requires either a C-level `xce_test` change (its REPL has no path from the `worker` context to the `client` context without a disconnect-equivalent `leave_ctx()`) or a bash coprocess restructuring to keep the worker connected across all layer checks — out of scope for tasks 6.1-6.4, worth a dedicated follow-up task if host-attrs resources ever need default-set coverage.
- Task 6.1 uncovered and got fixed a critical, unrelated, pre-existing controller bug (**bug-1578**, commit `b406dd7`, standalone — a dangling-pointer use-after-free in `sched_worker_change()`/`sched.c` that crashed the controller on worker disconnect once a scheduler had ever joined, 5 months old, unrelated to this feature). It's fixed now with regression coverage (`test_sched_worker_change.c`). If any later task sees a live controller die unexpectedly during a worker-disconnect sequence, confirm you're building from a commit that includes `b406dd7` before assuming it's a new bug. `inject_resources()` also fixed **bug-1577** (in-boundary): `XCE_WORKER_BIN=/bin/true` is exported before invoking `start-all.sh` to suppress the REAL `xcecd_virtual` worker daemon that `deploy-ha.sh`'s pre-existing worker branch would otherwise silently start (Requirement 2.1 violation) whenever the build dir happens to contain one.