build: remove nix dependency layer -- apt system libs + meson wrap subprojects

The build no longer self-wraps into `nix develop`: src/tools/xce-build.sh now
runs meson+ninja directly on system packages, with a dependency preflight
that fails early naming the missing tool/pkg-config package (DEPS.md).

Dependencies (see DEPS.md for the full tables):
- 11 C libraries from apt (libuv, openssl, libedit, sqlite3, hwloc, libcgroup,
  libtirpc, libcyaml, libyaml, zlib, libsystemd) with version floors in
  meson.build mirroring the Ubuntu 24.04 baseline.
- sd-bus: default libsystemd, -Duse_elogind=true opt-back for nix-era hosts.
- 6 libraries/tools unavailable in apt become committed meson wrap
  subprojects under src/subprojects/ (source closure, ~500 files): the
  upstream trees plus packagefiles/ overlays and diff_files patches
  registering dependency/program overrides the nix .pc files used to
  provide (incl. the '<name>-raw' variable deps the Go CGO path reads).
  Fresh-clone reproducibility verified: wiped trees + `meson subprojects
  download` reproduce the committed state exactly (fixed librtti.wrap's
  dangling patch_directory and c-capnproto's empty packagefiles overlay).
- xceadmin: self-contained venv built from requirements-build.txt (pinned,
  offline-seedable via PIP_FIND_LINKS) instead of --system-site-packages.

Build/packaging:
- Makefile facade over xce-build.sh; scripts/ci-ubuntu.sh drives a bare
  Ubuntu 24.04 container through deps + build + tests.
- src/tools/xce-package.sh: release-tree tarball packager. Collects
  non-base DT_NEEDED libs from the BUILD-tree binaries (staged ldd lies:
  the dangling build RPATH makes ldd drop the very lines the naive filter
  needs), sets $ORIGIN RPATHs, rebases the xceadmin venv at install time
  (kernel does not parse quotes in shebangs), ships systemd unit samples +
  install.sh. Local smoke test: all 8 binaries resolve, xceadmin works from
  both bin/ wrapper and venv script.

Test fixes found on the way (all environmental, not build bugs):
- harness_test.c setUp(): pin XCE_CONTROLLER_PEERS to a bound-then-closed
  loopback port so the CLI-error tests assert the no-controller failure
  path even on a dev box running a live cluster on localhost:9990.
- test_test_cli.c: setpgid(0,0) in the spawned-xce_test child. libedit's
  sig_handler re-raises received signals via kill(0, sig) -- process GROUP
  wide (strace -k evidence) -- and the child previously shared the
  harness's pgrp, so an interrupted SSL handshake killed the whole suite
  (exit 130).

Also fixed while re-baselining on the de-nix toolchain (pre-existing, now
deterministic): raft_init zeroes pre-vote tally state (pre_vote_in_progress/
pre_votes_granted/pre_vote_replies were left at stack garbage);
push_config_to_standbys clears stale config acks before fan-out
(repl_reset_config_acks) -- SHA-256-derived tags are not monotonic, so a
previous push's ack numerically above the new tag fabricated a barrier
quorum; capnpc-c's generated encode_* memsets the destination first
(skipped Data fields fed garbage into capn_setp).

Validation: asan profile 161/161 meson tests green (one known p2_integration
-110 timing flake on first run, green on re-run); debug + release full
builds OK; tarball smoke-tested on the build host. Cluster rollout on
t1/t2/t3 follows.

Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
lei.chen 2026-10-20 18:04:16 +00:00
parent 1e5c85555a
commit 50cf424146
521 changed files with 106066 additions and 83 deletions

47
scripts/ci-ubuntu.sh Executable file
View file

@ -0,0 +1,47 @@
#!/usr/bin/env bash
# ci-ubuntu.sh -- CI entrypoint for the de-nix build path.
#
# Designed to run inside a bare Ubuntu 24.04 container (the same baseline
# customers use); the dependency install mirrors DEPS.md exactly. Local
# developers who already have the deps can skip straight to the build step.
#
# Usage: scripts/ci-ubuntu.sh [--deps-only | --build-only]
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
DO_DEPS=1; DO_BUILD=1
case "${1:-}" in
--deps-only) DO_BUILD=0 ;;
--build-only) DO_DEPS=0 ;;
esac
if [ "$DO_DEPS" = 1 ]; then
echo "=== Installing build toolchain + system libraries (DEPS.md) ==="
export DEBIAN_FRONTEND=noninteractive
apt-get update -qq
# toolchain
apt-get install -y -qq build-essential pkg-config ninja-build python3 python3-pip python3-venv ca-certificates git
# meson >= 1.5 (Ubuntu 24.04's meson is 1.3 -- pip upgrades it in-place)
apt-get install -y -qq meson || true
meson --version 2>/dev/null | { read v; [ "${v%%.*}" -ge 1 ] && [ "${v#*.}" -ge 5 ] || pip3 install --break-system-packages 'meson>=1.5'; } || pip3 install --break-system-packages 'meson>=1.5'
# cap'n proto (schema compiler + C++ runtime for pycapnp)
apt-get install -y -qq capnproto libcapnp-dev
# C libraries (DEPS.md "apt 可直接安装" table)
apt-get install -y -qq libuv1-dev libssl-dev libedit-dev libsqlite3-dev \
libhwloc-dev libcgroup-dev libtirpc-dev libcyaml-dev libyaml-dev \
zlib1g-dev libsystemd-dev
# Go toolchain (Ubuntu golang-go is fine for the vendored modules)
command -v go >/dev/null 2>&1 || apt-get install -y -qq golang-go
echo "=== deps installed ==="
fi
if [ "$DO_BUILD" = 1 ]; then
echo "=== Building all components (asan profile: build + unit tests) ==="
bash "$REPO_ROOT/src/tools/xce-build.sh" all asan
echo "=== Building release profile (packaging prerequisite) ==="
bash "$REPO_ROOT/src/tools/xce-build.sh" all release
echo "=== Smoke: package a tarball ==="
bash "$REPO_ROOT/src/tools/xce-package.sh" "$REPO_ROOT/src/build.release" "$REPO_ROOT/src/dist"
echo "=== CI OK ==="
fi