build: make Go and Python builds fully offline (hermetic Nix build)
The Nix build previously fetched Go modules from goproxy.cn and Python build deps from PyPI during the build phase, relying on network leaking into the sandbox. This is non-reproducible and breaks whenever the sandbox resolver is unreachable (e.g. pc13v's loopback dnsmasq). Go: - Vendor all three modules (frontend/apiserver, frontend/xcecli, autoscaler/providers/baidu/src/bcctool) and commit vendor/ trees. - tools/go_build now builds with GOFLAGS=-mod=vendor and GOPROXY=off instead of GOPROXY=https://goproxy.cn. Python (xceadmin): - Add setuptools + wheel to xce.nix nativeBuildInputs. - meson-build.sh installs with pip --no-build-isolation --no-index --no-deps so no PyPI access happens at build time. xceadmin install script: - Invoke meson-install.sh via find_program('bash') instead of relying on its #!/usr/bin/env bash shebang, which fails in the Nix install sandbox (no /usr/bin/env -> 'could not be run'). Cleanup: - Drop now-unneeded cacert and impureEnvVars=[HOME] from xce.nix and xcecli.nix (they only existed to enable the impure network fetch). Docs: - README build guide: drop '--option sandbox false' from nix build / nix bundle commands, note the build is now hermetic, and document the go mod vendor + git add vendor/ maintenance workflow. Verified: nix build .#xcecore, .#xcecore_debug, .#xcecli, and .#xce all succeed with the sandbox enabled.
This commit is contained in:
parent
a51790dcbb
commit
68bc86bc59
3333 changed files with 1438798 additions and 11 deletions
|
|
@ -135,7 +135,14 @@ fi
|
|||
echo "=== xceadmin (meson): pinned install into build-dir venv (python: $PYTHON) ==="
|
||||
rm -rf "$VENV_DIR"
|
||||
"$PYTHON" -m venv --system-site-packages "$VENV_DIR"
|
||||
"$VENV_DIR/bin/pip" install "$WORK_DIR" --quiet
|
||||
# Offline install: the Nix build sandbox has no network. Runtime deps
|
||||
# (click/pycapnp/PyYAML/jsonschema) come from the nix python env via
|
||||
# --system-site-packages; the setuptools build backend comes from the same
|
||||
# env (added to nixSupport/xce.nix nativeBuildInputs). --no-build-isolation
|
||||
# stops pip from fetching setuptools from PyPI to build in an isolated env;
|
||||
# --no-index --no-deps forbid any network index / dependency resolution.
|
||||
"$VENV_DIR/bin/pip" install "$WORK_DIR" \
|
||||
--no-build-isolation --no-index --no-deps --quiet
|
||||
|
||||
echo "=== xceadmin (meson): running tests ==="
|
||||
# tests/_live_e2e_common.py's XCEADMIN_CANDIDATES look for a sibling
|
||||
|
|
@ -164,7 +171,11 @@ DIST_DIR="$BUILD_DIR/dist"
|
|||
echo "=== xceadmin (meson): building installable wheel ==="
|
||||
rm -rf "$DIST_DIR"
|
||||
mkdir -p "$DIST_DIR"
|
||||
"$VENV_DIR/bin/pip" wheel "$WORK_DIR" --no-deps -w "$DIST_DIR" --quiet
|
||||
# --no-deps: this is xceadmin's own wheel only, not a vendored bundle of
|
||||
# click/pycapnp/etc. --no-build-isolation --no-index keep it offline in the
|
||||
# Nix sandbox, same as the pinned install above.
|
||||
"$VENV_DIR/bin/pip" wheel "$WORK_DIR" \
|
||||
--no-deps --no-build-isolation --no-index -w "$DIST_DIR" --quiet
|
||||
WHEEL_FILE=$(ls "$DIST_DIR"/xceadmin-*.whl)
|
||||
echo "=== xceadmin (meson): wheel built at $WHEEL_FILE ==="
|
||||
|
||||
|
|
|
|||
|
|
@ -68,4 +68,12 @@ xceadmin_stamp = custom_target('xceadmin',
|
|||
# (ninja install always depends on build_by_default targets first) so the
|
||||
# build-dir venv it copies from already exists. See meson-install.sh for
|
||||
# why this isn't a plain `cp` of that venv.
|
||||
meson.add_install_script('meson-install.sh', python3_prog.full_path())
|
||||
# Invoke the install script through an explicit bash (find_program('bash'),
|
||||
# the same interpreter-pinning pattern the build-phase custom_target above
|
||||
# already uses) rather than relying on the script's `#!/usr/bin/env bash`
|
||||
# shebang. In the Nix install-phase sandbox there is no /usr/bin/env, so
|
||||
# letting meson exec the script by shebang fails with "could not be run";
|
||||
# handing meson a resolved bash + the script path sidesteps that entirely.
|
||||
meson.add_install_script(find_program('bash'),
|
||||
meson.current_source_dir() / 'meson-install.sh',
|
||||
python3_prog.full_path())
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue