myxce/.wolf/bugs/bug-1895.md

id	status	status_updated	status_note	component	severity	timestamp	last_seen	occurrences	files	tags	related	fix_commits
bug-1895	fixed	2026-09-30		build	wrong-result	2026-09-30T04:23:48Z	2026-09-30T04:23:48Z	1	src/tools/xce-build.sh (apiserver and all test steps)	build xce-build go tests set-e	bug-1894	4e484629

bug-1895: xce-build.sh apiserver/all: a failing apiserver Go test does not fail the build

Description

Observed: bash src/tools/xce-build.sh apiserver asan <dir> --incremental exited 0 although its Go unit tests failed:

FAIL	apiserver/backend [build failed]
FAIL	apiserver/store [build failed]
... exit=0

Expected: a failing unit test fails the build (non-zero exit), as for every C suite. Impact: apiserver Go test failures are silently ignored by /xce-build (and by all); a regression in apiserver/store, utils, controllers or backend can pass the build unnoticed. xcecli's tests are unaffected when they are the ones failing, but are skipped entirely when the apiserver tests fail. Found by: review of the build output while doing bug-1894's RED step, 2026-09-29.

Environment

• OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
• libc: glibc 2.42-47 (the nix dev shell's bash running xce-build.sh; the Go tests themselves run CGO_ENABLED=0)
• Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
• Build: asan profile (apiserver component, --incremental), src/xce.build.bug1871
• Code: scheduler-framework @ 141ac22c-dirty
• Deployment: n/a -- local build script

Reproduction

1. Make any apiserver Go test fail (e.g. reference an undefined symbol in a _test.go under src/frontend/apiserver/backend).
2. bash src/tools/xce-build.sh apiserver asan <dir> --incremental; echo $? -> prints the FAIL lines, then exits 0.

• Frequency: deterministic -- 1/1, 2026-09-29 (bug-1894's RED run)
• Evidence: the mechanism is in src/tools/xce-build.sh lines ~390 and ~437: cd frontend/apiserver && go test ... && cd ../xcecli && go test ... under set -euo pipefail; bash's errexit ignores a failure of any command in an && list except the last, so a failing apiserver go test just short-circuits the list and the script carries on to exit 0

Root cause

src/tools/xce-build.sh ran the frontend Go tests for the apiserver and all components as a single && list: cd frontend/apiserver && go test ... && cd ../xcecli && go test ..., under set -euo pipefail. Bash's errexit does not apply to a failing command in an && list unless it is the last one, so a failing apiserver go test merely short-circuited the list: the xcecli tests were skipped, the script carried on, and it exited 0 with === Done. Only an xcecli failure (the list's last command) failed the build.

Fix

src/tools/xce-build.sh: in the apiserver and all components, each Go suite now runs as its own command in a subshell -- (cd frontend/apiserver && go test ...) then (cd frontend/xcecli && go test ...) -- so a failure of either exits the script under set -e, and the cd no longer changes the script's working directory. The xcecli component's single suite got the same subshell for consistency.

Verification

• Test: live probe -- a temporary failing test (TestBug1895DeliberateFailure, t.Fatal) dropped into src/frontend/apiserver/utils/ and, separately, src/frontend/xcecli/cmd/, then xce-build.sh apiserver asan <dir> --incremental. RED before the fix: FAIL apiserver/utils yet exit 0 with === Done and no xcecli tests run. GREEN after: apiserver probe -> exit 1, no Done line; xcecli probe -> exit 1; clean tree -> exit 0 with all 10 xcecli packages tested. The probe files were removed (never committed)
• Suites: n/a -- build-script change; bash -n clean
• Live: n/a -- the three builds above are the end-to-end check

References

• Fix commit: 4e484629
• Related: bug-1894 (found during its RED step)