myxce/.kiro/specs/build-time-versioning/research.md

Research & Design Decisions

Summary

• Feature: build-time-versioning
• Discovery Scope: Extension (existing, partial C-side infrastructure; zero Go/Python infrastructure)
• Key Findings:
  • Controller and scheduler already have a working --version/-V flag (GNU argp auto-generates it from argp_program_version = MK_VERSION("xce v0.0.1-", STR(VERSION_COMMIT_ID)), both main.cs) and a project-wide VERSION_COMMIT_ID preprocessor define wired through meson (src/meson.build:30, src/meson.options's commit_id option) — but it carries no build timestamp, and xce-build.sh never sets -Dcommit_id=, so every locally-built binary (the entire /xce-build dev loop, including the one that produced the stale binary in the motivating incident) gets an empty commit id today.
  • Worker uses plain getopt_long(), not argp — it has no --version at all.
  • git is deliberately not a nix devShell buildInput (src/nixSupport/xce.nix) — the existing commit_id mechanism avoids shelling out to git entirely; the two production nix packages (xce.nix:109, xcecli.nix:30) derive it from the flake's own self.shortRev/self.dirtyShortRev introspection instead. Any new mechanism must respect this: no run_command/custom_target in meson.build may shell out to git.
  • Go (apiserver, xcecli) and Python (xceadmin) have zero version infrastructure — no -ldflags injection anywhere in src/tools/go_build or src/frontend/Makefile, no click.version_option, no dynamic pyproject.toml version.

Research Log

Where does xce-build.sh (the actual dev-loop path) get its build metadata from?

• Context: the motivating incident happened via /xce-build scheduler debug, not nix build — need to know which computation path is actually load-bearing for fixing it.
• Findings: xce-build.sh's MESON_OPTS array (set per-profile at xce-build.sh:70-72) never includes -Dcommit_id= or anything build-time related; meson setup is called with only --buildtype/-Db_sanitize. git IS available here despite not being a declared nix devShell buildInput, because nix develop (unlike a sandboxed nix build) layers onto the ambient host PATH rather than replacing it — this project's own scripts already run from inside a real git checkout.
• Implications: xce-build.sh must be the one to compute commit id (git rev-parse --short HEAD, -dirty suffixed via git diff --quiet HEAD -- / git status --porcelain) and build time (date -u), independently of the nix-package path, and pass both via meson options. This is the fix that actually closes the incident's gap — the nix-package path already has a working (commit-id-only) mechanism.

Message/value plumbing per toolchain

• Context: three different build tool invocations (meson C custom target-free preprocessor define, Go's go_build shell script + Makefile, Python's meson-build.sh custom_target) need the same two values.
• Findings: src/meson.build:30 already threads commit_id project-wide via add_project_arguments() for C. src/tools/go_build (invoked by both apiserver/meson.build's and xcecli/meson.build's custom_target()) already threads other env vars (e.g. CGO flags) into its go build invocation — same mechanism can carry new XCE_GIT_COMMIT/XCE_BUILD_TIME env vars for -ldflags -X. src/xceadmin/meson.build's custom_target already passes several positional args to meson-build.sh; two more (or two new env vars) can carry the same values for a small generated Python module written before the pip install step.
• Implications: meson itself is the single source of truth for one build invocation — it resolves the two new options (commit_id, now actually SET by xce-build.sh; a new build_time) once, and every language-specific build step (C define, Go ldflags, Python generated module) reads from that same resolved value instead of independently recomputing it. Avoids the 3 toolchains ever disagreeing within one build.

Dirty-tree representation: separate flag vs. suffix convention

• Context: Requirement 1 AC3 requires distinguishing an uncommitted-changes build from an exact commit.
• Findings: both git's own --dirty flag and this project's existing Nix self.dirtyShortRev already use a suffix convention (<hash>-dirty) rather than a separate boolean field.
• Implications: reuse that convention — commit_id's own string value carries -dirty when applicable, no third meson option/env var needed. Matches the Build-vs-Adopt lens (reuse an established convention already present twice in this codebase) and the Simplification lens (one string carries both facts).

Design Decisions

Decision: One resolved value pair, fed from two independent computation sites

• Context: production (nix build) and dev-loop (xce-build.sh) are architecturally different environments (fully sandboxed vs. ambient-PATH devShell) that already compute commit_id differently for exactly that reason.
• Alternatives Considered:
  1. Make xce-build.sh also avoid shelling out to git, deriving everything from Nix like the packages do — rejected, xce-build.sh doesn't invoke Nix package builds at all, it drives meson directly inside nix develop.
  2. Add a git-invoking run_command() inside meson.build itself, used by both paths — rejected, breaks the sandboxed nix build path where git is unavailable by design.
• Selected Approach: extend the existing split — xce-build.sh computes via git+date and passes -Dcommit_id=/-Dbuild_time=; the nix packages (xce.nix, xcecli.nix) compute commit_id via self.shortRev/self.dirtyShortRev (unchanged, eval-time) and build_time via a preConfigure build-phase date -u hook appending to mesonFlagsArray (build-time, not eval-time), passed the same -Dbuild_time= way.
• Rationale: matches the codebase's own existing precedent exactly; zero new dependencies; keeps the sandboxed build's reproducibility model intact.
• Trade-offs: two computation sites to keep in sync (same risk the existing commit_id-only mechanism already carries and has lived with).
• Follow-up: verify during implementation that a plain meson setup with neither xce-build.sh nor a nix package (e.g. meson setup build run by hand) degrades to the "unknown" fallback (Requirement 1 AC4), not a build failure.
• Correction (task 2.2, round 1 → round 2): the nix-side value was originally attempted via builtins.currentTime (eval-time) as noted above — this broke Nix's flake pure-eval mode entirely (nix build .#xcecore_debug, the documented no-flag command, failed with error: attribute 'currentTime' missing, requiring --impure) and produced a raw epoch-seconds string rather than UTC ISO-8601. Caught by task-local review (REJECTED, round 1) before merge; corrected to the preConfigure/date -u build-phase approach described in "Selected Approach" above, verified to restore pure-eval and produce a correctly-formatted value (round 2, APPROVED). Recorded here so this decision's rationale stays accurate for anyone reading only research.md.

Decision: Worker gets a hand-rolled --version case, not a switch to argp

• Context: worker uses getopt_long(), not argp; controller/scheduler's free --version comes from argp's own auto-generated handling.
• Alternatives Considered: migrate worker's arg parsing to argp for consistency — rejected as disproportionate scope (a full parser migration to fix a missing flag).
• Selected Approach: add a {"version", no_argument, ...} long option and a case, printing the same MK_VERSION-style string then exiting 0 — mirrors argp's own output shape without adopting argp.
• Rationale: minimal, consistent output, zero risk to worker's existing option handling.

Risks & Mitigations

• Two independent computation sites (dev-loop vs. nix-package) could drift in format — mitigate by having both resolve to the exact same two meson option names and the exact same downstream consumption code, so only the value-sourcing differs, never the shape.
• go_build/meson-build.sh are shared shell scripts already carrying other env-var-driven behavior (CGO flags, schema sync) — new env vars must be additive/optional so a caller that doesn't set them still builds (falls back to "unknown"), not required.

References

• src/meson.build:30, src/meson.options — existing commit_id mechanism.
• src/flake.nix:15-18, src/nixSupport/xce.nix:109, xcecli.nix:30 — existing Nix-side commit id derivation.
• src/tools/xce-build.sh:70-98 — dev-loop meson invocation, current gap.
• src/tools/go_build, src/frontend/apiserver/meson.build:53-62, src/frontend/xcecli/meson.build:9-17 — Go build plumbing.
• src/xceadmin/meson.build:27-47, src/xceadmin/meson-build.sh — Python build plumbing.