diff --git a/.kiro/specs/build-time-versioning/design.md b/.kiro/specs/build-time-versioning/design.md new file mode 100644 index 00000000..d45682af --- /dev/null +++ b/.kiro/specs/build-time-versioning/design.md @@ -0,0 +1,241 @@ +# Design Document + +## Overview +**Purpose**: This feature gives every XCE component (controller, scheduler, worker, apiserver, xcecli, xceadmin) a build-time identity — the git commit it was built from and when it was built — visible from the running artifact itself, via a `--version` flag and (for the four long-running daemons) a startup log line. + +**Users**: Operators and developers diagnosing a running deployment (the motivating incident: a stale scheduler binary kept running a known-fixed crash, and confirming that required manual `ls -la` mtime checks plus `git merge-base --is-ancestor`). + +**Impact**: Extends the existing, partial C-side mechanism (controller/scheduler already have a commit-id-only `--version` via `argp`, but `xce-build.sh` never actually sets it, so it is empty on every local build today) to carry a build timestamp too, fills the worker/Go/Python gaps, and fixes the `xce-build.sh` gap that made the motivating incident possible in the first place. + +### Goals +- Every component's build artifact carries a git commit identifier (dirty-suffixed when applicable) and a build timestamp. +- `--version` on every component prints both and exits. +- Controller, scheduler, worker, and apiserver additionally log both at INFO level on startup. +- `xce-build.sh` (the dev-loop build path implicated in the motivating incident) actually sets these values — today it does not. +- The values are byte-for-byte identical between `--version` output and the startup log line for the same artifact. + +### Non-Goals +- Automatically detecting or alerting that a *running* binary is stale relative to current source (out of scope per requirements' Boundary Context — this feature only makes the comparison possible). +- Version compatibility negotiation between components running different versions. +- Retrofitting version info into already-built/deployed binaries. +- Migrating worker's `getopt_long()` argument parsing to `argp` (a much larger, unrelated change) — worker gets a hand-added `--version` case instead. + +## Boundary Commitments + +### This Spec Owns +- The two build-time values themselves (commit identifier, build timestamp) and how each of the six components' build steps receive them. +- `--version` output and (for the four daemons) the startup log line, for all six components. +- `xce-build.sh`'s computation of both values for the dev-loop build path. +- The nix packages' (`xce.nix`, `xcecli.nix`) computation of both values for the production build path. + +### Out of Boundary +- Any runtime mechanism that compares a running binary's version against "current" source (no such comparison exists or is planned). +- `xceadmin`'s own semantic package version (`pyproject.toml`'s `version = "0.1.0"`) — untouched; build metadata is additive, displayed alongside it, not a replacement for it. +- Worker's broader argument-parsing architecture. + +### Allowed Dependencies +- The existing `commit_id` meson option and `VERSION_COMMIT_ID`/`MK_VERSION` C mechanism (`src/meson.build`, `src/meson.options`, `src/include/utils.h`) — extended, not replaced. +- The existing Nix flake commit-id derivation (`self.shortRev`/`self.dirtyShortRev`, `src/flake.nix`) — extended with a new build-time value, same wiring style. +- `src/tools/go_build`'s existing env-var-driven `go build` invocation pattern (already used for CGO flags). +- `src/xceadmin/meson.build`'s existing `custom_target` → `meson-build.sh` argument-passing pattern. +- Go's standard `-ldflags -X` package-variable injection; Cobra's built-in `Version` field (xcecli already uses Cobra); Click's built-in `version_option` (xceadmin already uses Click). + +### Revalidation Triggers +- The `commit_id` meson option is renamed, removed, or its consumption in `src/include/utils.h`'s `MK_VERSION`/`STR` macros changes shape. +- `xce-build.sh`'s `MESON_OPTS` construction is refactored away from its current per-profile array pattern. +- `src/tools/go_build`'s final `go build` invocation line changes. +- `src/xceadmin/meson-build.sh`'s step ordering changes (the generated build-info module must land before the `pip install` step that packages it). + +## Architecture + +### Existing Architecture Analysis +- Controller and scheduler already parse `--version`/`-V` for free via GNU `argp`'s `argp_program_version` mechanism (`MK_VERSION("xce v0.0.1-", STR(VERSION_COMMIT_ID))`, both `main.c`s) — this feature extends that same macro rather than adding a parallel one. +- `VERSION_COMMIT_ID` already flows into every C component uniformly via `add_project_arguments()` at the top-level `src/meson.build` — the new build-timestamp define follows the identical path, added once, inherited everywhere. +- Worker's `getopt_long()`-based parsing has no equivalent auto-version behavior and needs an explicit new case. +- `git` is deliberately absent from the nix devShell's declared buildInputs (`src/nixSupport/xce.nix`) — the *sandboxed* `nix build` path already avoids shelling out to git for exactly this reason (uses Nix's own `self.shortRev` introspection instead). The *dev-loop* `xce-build.sh` path runs inside `nix develop`, which layers onto the ambient host PATH rather than replacing it, so git IS usable there — and needs to be used there, since `xce-build.sh` today never sets `commit_id` at all (the actual gap behind the motivating incident). + +### Architecture Pattern & Boundary Map +Two independent **computation sites** (respecting the existing sandboxed-vs-ambient split) feed one **resolved value pair** into meson, which every language-specific **consumption site** reads from — never recomputes independently. + +```mermaid +graph TB + DevScript[xce-build.sh git plus date] + NixPkg[xce.nix xcecli.nix self.shortRev plus builtins.currentTime] + MesonOpts[meson commit_id and build_time options] + CDefine[C VERSION_COMMIT_ID and VERSION_BUILD_TIME defines] + GoLdflags[go_build ldflags -X injection] + PyGen[meson-build.sh generated build info module] + Controller[controller and scheduler argp version] + Worker[worker getopt_long version case] + Apiserver[apiserver flag version] + Xcecli[xcecli cobra version] + Xceadmin[xceadmin click version_option] + + DevScript --> MesonOpts + NixPkg --> MesonOpts + MesonOpts --> CDefine + MesonOpts --> GoLdflags + MesonOpts --> PyGen + CDefine --> Controller + CDefine --> Worker + GoLdflags --> Apiserver + GoLdflags --> Xcecli + PyGen --> Xceadmin +``` + +**Architecture Integration**: +- Selected pattern: single resolved-value-pair fan-out, matching the existing `commit_id` mechanism's own shape. +- Domain/feature boundaries: computation (2 sites, environment-specific) is fully separated from consumption (6 sites, one per component) — a consumption site never knows or cares which computation site produced its value. +- Existing patterns preserved: `argp` version macro (C), Cobra `Version` field and Click `version_option` (already the idiomatic mechanism in each respective Go/Python tool), `go_build`'s env-var plumbing, `meson-build.sh`'s positional-arg plumbing. +- New components rationale: `sim_prescan_session_count`-style small, single-purpose additions only where a gap genuinely exists (worker's version case, Go's ldflags wiring, Python's generated module) — no new abstraction layer introduced. +- Steering compliance: no new external dependency (`tech.md`); wire/build-format changes are free pre-release (no back-compat burden); TDD required for every new behavior. + +### Technology Stack + +| Layer | Choice / Version | Role in Feature | Notes | +|-------|------------------|-----------------|-------| +| C build (controller/scheduler/worker) | Meson preprocessor defines (existing mechanism) | Carries `commit_id` (existing) + new `build_time` into every C translation unit | `src/meson.build:30` | +| C CLI (controller/scheduler) | GNU `argp` (existing) | `--version`/`-V` auto-handling via `argp_program_version` | Extend `MK_VERSION` macro | +| C CLI (worker) | `getopt_long()` (existing) | New explicit `--version` case | No parser migration | +| Go build | `-ldflags -X` (standard Go idiom, new to this repo) | Injects `main.GitCommit`/`main.BuildTime` package vars | Via `go_build` script + `Makefile` | +| Go CLI (apiserver) | `flag` package (existing) | New `-version` flag | Matches existing single-flag style | +| Go CLI (xcecli) | Cobra (existing) | `rootCmd.Version` field (built-in `--version`) | Zero new parsing code | +| Python build | Meson `custom_target` (existing) | New generated build-info module written before `pip install` | `meson-build.sh` | +| Python CLI (xceadmin) | Click (existing) | `click.version_option()` (built-in `--version`) | Zero new parsing code | +| Nix packaging | Nix flake (existing) | `self.shortRev`/`self.dirtyShortRev` (existing) + new `builtins.currentTime`-derived value | `xce.nix`, `xcecli.nix` | + +## File Structure Plan + +### Modified Files +- `src/meson.options` — add a `build_time` string option (default `''`), alongside the existing `commit_id`. +- `src/meson.build` — add `-DVERSION_BUILD_TIME=` to the existing `add_project_arguments()` call (line ~30). +- `src/include/utils.h` — extend `MK_VERSION`/introduce a companion macro so the version string includes the build time, not just the commit id. +- `src/controller/main.c`, `src/scheduler/src/main.c` — update `argp_program_version` construction to include build time; add the startup INFO log line (each already has one startup INFO call site to extend/follow). +- `src/worker/src/main.c` — add a `--version` long option + case (prints the same shape, exits 0); add the startup INFO log line (follows its own existing startup INFO call site). +- `src/tools/xce-build.sh` — compute `commit_id` (git, `-dirty` suffixed) and `build_time` (UTC `date`) and pass both via new `-Dcommit_id=`/`-Dbuild_time=` entries in `MESON_OPTS`, falling back to `"unknown"` if `git` fails (e.g. not a git checkout). +- `src/flake.nix`, `src/nixSupport/xce.nix`, `src/nixSupport/xcecli.nix` (exact filenames per existing commit_id wiring) — add a `builtins.currentTime`-derived build-time value passed the same way `commit_id` already is. +- `src/tools/go_build` — read `XCE_GIT_COMMIT`/`XCE_BUILD_TIME` env vars (empty-safe) and append `-ldflags "-X main.GitCommit=... -X main.BuildTime=..."` to the final `go build` invocation. +- `src/frontend/apiserver/meson.build`, `src/frontend/xcecli/meson.build` — pass the two new env vars into their `custom_target()` calls (mirrors existing env var passing). +- `src/frontend/Makefile` — add the same `-ldflags` construction (via `git`/`date` directly, dev-loop path) to `build-server`/`build-cli` targets. +- `src/frontend/apiserver/cmd/main.go` — declare `var (GitCommit, BuildTime string)`; add a `-version` flag; add/extend the startup `logger.Info(...)` line. +- `src/frontend/xcecli/cmd/main.go` — declare the same two vars; set `rootCmd.Version` (Cobra); extend its own startup `logger.Info(...)` line. +- `src/xceadmin/meson.build` — pass two new positional args (or env vars) to `meson-build.sh`. +- `src/xceadmin/meson-build.sh` — write a small generated Python module (e.g. `_build_info.py`) into the mirrored source tree, before the `pip install` step, containing the two values (falls back to `"unknown"` if not provided). +- `src/xceadmin/src/xceadmin/cli.py` — read the generated module (import guarded for the case it's genuinely absent, e.g. a non-meson editable install) and wire `click.version_option()` on the top-level group. + +### New Files +- `src/xceadmin/src/xceadmin/_build_info.py` — **generated at build time only** (not checked in); a committed fallback stub (all fields `"unknown"`) ships in-tree so `pip install -e .` (the documented fast-iteration path in `build-deployment.md`) still imports cleanly without having run the meson build. + +> No new C/Go/Python source files beyond the one generated-with-fallback-stub Python module above — every other change extends an existing file's existing responsibility (arg parsing gains a flag, a build script gains an option). + +## Requirements Traceability + +| Requirement | Summary | Components | Interfaces | Flows | +|-------------|---------|------------|------------|-------| +| 1.1, 1.2 | Every artifact carries commit id + build time | All 6 consumption sites + both computation sites | `MK_VERSION` macro, Go ldflags vars, Python `_build_info` module | Computation → meson → consumption | +| 1.3 | Dirty tree marked | `xce-build.sh`'s git computation | commit_id string carries `-dirty` suffix | n/a | +| 1.4 | Unknown when undeterminable | `xce-build.sh`, `meson-build.sh`, `go_build` | fallback to literal `"unknown"` | n/a | +| 2.1 | `--version` prints and exits | argp (controller/scheduler), worker's new case, apiserver's `-version`, Cobra `Version`, Click `version_option` | six CLI entry points | n/a | +| 2.2 | `--version` and startup log agree | shared macro/var/module per component | same source value read twice, never recomputed | n/a | +| 3.1, 3.2 | Startup log line at INFO | controller/scheduler/worker/apiserver startup sequences | existing first-INFO-line call sites, extended | Startup → log | +| 4.1, 4.2 | Build tooling confirms embedding | `xce-build.sh`, nix packages | non-empty resolved value check before/after `meson setup`/build | Build → verify | +| 4.3 | Accurate under every C build profile | `src/meson.build`'s project-wide `add_project_arguments()` | applies uniformly to asan/debug/release | n/a | +| 5.1, 5.2 | Values reflect actual tree/build event | `xce-build.sh` (git+date at invocation time), nix (`self.shortRev`+`builtins.currentTime` at evaluation time) | no hardcoded/stale values | n/a | + +## Components and Interfaces + +### C Daemons (controller, scheduler, worker) + +#### Version macro +| Field | Detail | +|-------|--------| +| Intent | Carry commit id + build time into every C daemon's `--version` output and startup log | +| Requirements | 1.1, 1.2, 2.1, 2.2, 3.1, 3.2, 4.3 | + +**Responsibilities & Constraints** +- `MK_VERSION` (`src/include/utils.h`) extended to compose `"xce v0.0.1- built "` (or equivalent) from the two new preprocessor defines — a single macro, both consumers (controller/scheduler's `argp_program_version`, worker's hand-added case) call it, guaranteeing `--version` and the startup log line render from the identical string. +- Controller/scheduler: their existing single startup INFO line (`main.c:365`/`377`+`381` respectively, per discovery) gets this version string appended or logged as an immediately-preceding line — never a second, independently-formatted string. +- Worker: new `{"version", no_argument, NULL, OPT_VERSION}` long option in its existing `long_opts[]`/`getopt_long()` switch (`main.c:358-364`), printing the same macro's output and exiting 0 before any other startup work begins. + +**Contracts**: Service [ ] / API [ ] / Event [ ] / Batch [ ] / State [ ] + +**Implementation Notes** +- Integration: extends `src/include/utils.h`'s existing `STR`/`MK_VERSION` macros — no new header. +- Validation: a build with `commit_id`/`build_time` unset must still compile and print `"unknown"` fields, never a blank/garbled string. +- Risks: worker's `getopt_long()` short-option string (`"n:l:vh"`) must gain a way to reach the new long-only option — use a long-option-only constant (no short equivalent), matching `--peers`'s own existing precedent in the same `long_opts[]` array. + +### Go Binaries (apiserver, xcecli) + +#### ldflags version injection +| Field | Detail | +|-------|--------| +| Intent | Inject build-time values into each Go binary at link time, exposed via each CLI's native version mechanism | +| Requirements | 1.1, 1.2, 2.1, 2.2, 3.1, 3.2, 4.1, 4.2 | + +**Responsibilities & Constraints** +- `src/tools/go_build` (consumed by both `apiserver/meson.build` and `xcecli/meson.build`'s `custom_target()`) reads `XCE_GIT_COMMIT`/`XCE_BUILD_TIME` from its environment (empty-safe — falls back to `"unknown"` when unset, e.g. a bare `go build` invoked by hand) and appends `-ldflags "-X main.GitCommit=$v1 -X main.BuildTime=$v2"` to its existing `go build` invocation. +- `src/frontend/Makefile`'s `build-server`/`build-cli` targets compute the same two values directly (git/date, dev-loop path, no meson involved) and pass the identical `-ldflags` shape — same output format as the meson path, different computation site (mirrors the C-side split). +- `apiserver/cmd/main.go`: package-level `var (GitCommit = "unknown"; BuildTime = "unknown")`, a new `-version` bool flag (checked first, before `utils.InitLogger()` — prints and exits, no logging side effects), and the existing `logger.Info("********** Initialize Logger done!")` call site extended to also log both values (same convention, `"********** done!"` is untouched; the version fields ride as structured zap fields on that same line or an adjacent one). +- `xcecli/cmd/main.go`: same two vars; `rootCmd.Version = fmt.Sprintf(...)` set once at init (Cobra then handles `--version` natively — zero new flag-parsing code); its own existing `logger.Info("********** Initialize Logger on client side done!")` extended the same way. + +**Contracts**: Service [ ] / API [ ] / Event [ ] / Batch [ ] / State [ ] + +**Implementation Notes** +- Integration: `go_build`'s env-var pattern already exists for CGO flags — this reuses it, not a new plumbing mechanism. +- Validation: `--version`/`-version` output and the startup log's values must be the exact same two package vars, read twice, never two separate computations. +- Risks: none identified — `-ldflags -X` only accepts string vars, both fields are strings by construction. + +### Python (xceadmin) + +#### Generated build-info module +| Field | Detail | +|-------|--------| +| Intent | Package build-time values into the wheel so `xceadmin --version` reflects the actual build, not just `pyproject.toml`'s static semantic version | +| Requirements | 1.1, 1.2, 2.1, 4.1, 4.2 | + +**Responsibilities & Constraints** +- `meson-build.sh` writes `_build_info.py` (containing `GIT_COMMIT`/`BUILD_TIME` string constants, sourced from two new args/env vars, `"unknown"` fallback) into the mirrored source tree's `xceadmin/` package directory, *before* the `pip install` step (so it packages into both the dev venv install and the final wheel) — same ordering discipline the existing schema-sync step already follows (also runs before `pip install`). +- `cli.py`'s top-level `@click.group()` gains `@click.version_option(version=f"{pkg_version} ({_build_info.GIT_COMMIT}, {_build_info.BUILD_TIME})")`, importing `_build_info` defensively (a committed fallback stub with `"unknown"` fields ships in-tree so a non-meson `pip install -e .` — the documented fast-iteration path — still imports cleanly). +- No startup log line (Requirement 3 scopes that to the four long-running daemons only; xceadmin is a one-shot CLI). + +**Contracts**: Service [ ] / API [ ] / Event [ ] / Batch [ ] / State [ ] + +**Implementation Notes** +- Integration: extends `meson-build.sh`'s existing custom_target sequence at exactly one new step, same file-write style already used for schema sync. +- Validation: `pip install -e .` (no meson) must still import cleanly against the committed fallback stub — this is the one file in this feature that must exist in two states (checked-in fallback vs. build-time-overwritten real values) and both must be valid Python. +- Risks: the generated module must not be checked in with real values (would go stale immediately) — only the `"unknown"`-only fallback stub is committed; `.gitignore` (or equivalent) must exclude the real generated file if `meson-build.sh` ever writes it into the actual source tree rather than only the mirrored build-tree copy (verify this during implementation — the mirroring step at `meson-build.sh` lines 32-46 should already keep it out of the real source tree, matching how the schema sync step works today). + +### Build Orchestration + +#### xce-build.sh value computation +| Field | Detail | +|-------|--------| +| Intent | Close the actual gap behind the motivating incident — the dev-loop build path must set real values, not leave them empty | +| Requirements | 4.1, 4.2, 4.3, 5.1, 5.2 | + +**Responsibilities & Constraints** +- Computes `commit_id` via `git rev-parse --short HEAD`, appending `-dirty` when `git diff --quiet HEAD --` (or `git status --porcelain`) reports changes; falls back to `"unknown"` if `git` itself fails (e.g. run against an extracted source tarball with no `.git`). +- Computes `build_time` via `date -u +%Y-%m-%dT%H:%M:%SZ`. +- Appends `-Dcommit_id="$commit_id" -Dbuild_time="$build_time"` to `MESON_OPTS` (`xce-build.sh:70-72`) for every profile, before the existing `meson setup`/`meson setup --reconfigure` calls. + +**Contracts**: Service [ ] / API [ ] / Event [ ] / Batch [ ] / State [ ] + +**Implementation Notes** +- Integration: a small addition right where `MESON_OPTS` is already constructed — no restructuring of the profile dispatch. +- Validation: the exact scenario from the motivating incident — build via `/xce-build scheduler debug`, run `--version`, confirm it shows a real (non-empty, non-"unknown") commit id matching `git rev-parse --short HEAD` at build time. +- Risks: none — purely additive to an existing array. + +## Testing Strategy + +### Unit Tests +- C: a new scheduler/controller/worker test (or a shared `lib` test if `MK_VERSION`'s logic is factored into a testable helper) asserting the version string contains both the commit id and build time defines when set, and `"unknown"` fallbacks when unset. +- Go: a test in each `cmd` package (or a shared helper) asserting `-version`/`--version` output contains both `GitCommit` and `BuildTime` values. +- Python: a `pytest` case (xceadmin's existing suite, run by `meson-build.sh`) asserting `xceadmin --version` output contains the generated build-info fields, and that the committed fallback stub is itself valid, importable Python. + +### Integration Tests +- `xce-build.sh`-driven build (any profile) followed by running `--version` on the resulting controller/scheduler/worker binaries, asserting a real (non-`"unknown"`) commit id is present — this is the regression test for the motivating incident itself. +- A `go_build`-driven meson build of apiserver/xcecli, followed by `-version`, asserting the same. +- `xceadmin`'s existing meson-driven build+test+wheel cycle, followed by installing the wheel and running `xceadmin --version`. + +### E2E / Consistency +- For at least one C daemon and one Go binary: start it, capture its startup log line, separately run `--version`, and assert the two report byte-identical commit id and build time (Requirement 2.2). diff --git a/.kiro/specs/build-time-versioning/requirements.md b/.kiro/specs/build-time-versioning/requirements.md new file mode 100644 index 00000000..a09db79c --- /dev/null +++ b/.kiro/specs/build-time-versioning/requirements.md @@ -0,0 +1,52 @@ +# Requirements Document + +## Project Description (Input) +Build-time versioning: every component (controller, scheduler, worker, apiserver, xcecli, xceadmin) must embed and display its source git commit id and build timestamp, shown both via a --version CLI flag and a startup log line (INFO level, matching existing daemon startup log conventions). xce-build.sh should also ensure/verify this info is correctly embedded for each component it builds. No existing infrastructure for this exists today (confirmed via investigation: no version.h/version.c, no meson version defines, no Go ldflags -X injection, no --version handling in any daemon, no dynamic pyproject versioning) -- this is new, cross-cutting work spanning three toolchains: C/meson (controller/scheduler/worker, linking libxce), Go (apiserver/xcecli, via ldflags), and Python (xceadmin, via pyproject/meson custom_target). Motivating incident: a live production-like deployment kept running a stale debug-profile scheduler binary that lacked a just-landed heap-corruption fix, and diagnosing that required manual `ls -la` mtime checks plus `git merge-base --is-ancestor` -- this feature should make that kind of staleness immediately visible from the running binary/log itself, no forensic git commands needed. + +## Introduction +This feature adds build-time version identification to every XCE component so operators can determine, directly from a running binary or its log output, exactly which source commit and build time produced it — without resorting to manual filesystem or git forensics. It is motivated by an incident where a stale scheduler binary silently kept running a known-fixed heap-corruption bug because there was no way to tell, from the binary or its logs, which commit it was actually built from. + +## Boundary Context (Optional) +- **In scope**: embedding and displaying a git commit identifier and build timestamp for controller, scheduler, worker, apiserver, xcecli, and xceadmin; exposing this via a `--version` flag and, for long-running daemons, a startup log line; the build tooling confirming this information was correctly embedded for each component it builds. +- **Out of scope**: automatically detecting or alerting that a running/deployed binary is stale relative to the latest source; retroactively updating the version information of an already-running process; version negotiation or compatibility checks between components running different versions. +- **Adjacent expectations**: operators are expected to compare the displayed commit identifier against the source repository themselves (for example, via `git`) to judge staleness; this feature makes that comparison possible, it does not automate it. + +## Requirements + +### Requirement 1: Version Information Content +**Objective:** As an operator diagnosing a running deployment, I want each component to know which exact source commit and build time produced it, so that I can determine whether a binary reflects a given fix without inspecting file timestamps or git history by hand. + +#### Acceptance Criteria +1. The system shall associate a git commit identifier with every built artifact of controller, scheduler, worker, apiserver, xcecli, and xceadmin. +2. The system shall associate a build timestamp, in absolute UTC form, with every built artifact of controller, scheduler, worker, apiserver, xcecli, and xceadmin. +3. If the source tree contains uncommitted changes at build time, then the system shall mark the associated version information as modified, so an operator does not mistake it for an exact committed state. +4. If the git commit identifier cannot be determined at build time (for example, a source tree without git history), then the system shall record the version information as unknown rather than fail the build. + +### Requirement 2: Version Display via CLI +**Objective:** As an operator or script, I want to query a component's build identity directly from its binary, so that I can verify a deployed artifact without needing separate log access. + +#### Acceptance Criteria +1. When a `--version` flag is passed to the controller, scheduler, worker, apiserver, xcecli, or xceadmin command, the invoked component shall print its associated commit identifier and build timestamp and exit without performing its normal function. +2. The system shall present the same commit identifier and build timestamp values through the `--version` flag and through the startup log line (Requirement 3) for the same build artifact, so the two never disagree. + +### Requirement 3: Version Display at Startup +**Objective:** As an operator watching a daemon's log output, I want to see its build identity as soon as it starts, so that I can confirm which commit is actually running without a separate query step. + +#### Acceptance Criteria +1. When the controller, scheduler, worker, or apiserver daemon starts, the daemon shall log its commit identifier and build timestamp at INFO level, following the daemon's existing startup log format. +2. While a controller, scheduler, worker, or apiserver daemon is running, an operator shall be able to determine its build identity by reading its log without sending it any request. + +### Requirement 4: Build-Time Embedding Verification +**Objective:** As a developer running the project's build tooling, I want confirmation that version information was actually embedded correctly, so that I do not unknowingly ship or deploy a binary with missing or incorrect build identity. + +#### Acceptance Criteria +1. When the build tooling builds a component covered by this feature, the build tooling shall confirm that a commit identifier and build timestamp were embedded in the resulting artifact for that component. +2. If the build tooling cannot confirm that version information was embedded for a component it built, then the build tooling shall report this as a build problem rather than silently succeeding. +3. The system shall embed accurate version information under every supported build profile of controller, scheduler, and worker, not only a subset of profiles. + +### Requirement 5: Accuracy of Embedded Version Information +**Objective:** As an operator relying on displayed version information to make a deployment decision, I want that information to always reflect the actual source tree and build event that produced the running binary, so that I can trust it during an incident. + +#### Acceptance Criteria +1. The system shall derive the embedded commit identifier from the actual state of the source tree at build time, not from a value set independently of the build. +2. The system shall derive the embedded build timestamp from the actual time the build artifact was produced, not from the source commit's own timestamp. diff --git a/.kiro/specs/build-time-versioning/research.md b/.kiro/specs/build-time-versioning/research.md new file mode 100644 index 00000000..497f626f --- /dev/null +++ b/.kiro/specs/build-time-versioning/research.md @@ -0,0 +1,56 @@ +# Research & Design Decisions + +## Summary +- **Feature**: `build-time-versioning` +- **Discovery Scope**: Extension (existing, partial C-side infrastructure; zero Go/Python infrastructure) +- **Key Findings**: + - Controller and scheduler already have a **working `--version`/`-V` flag** (GNU `argp` auto-generates it from `argp_program_version = MK_VERSION("xce v0.0.1-", STR(VERSION_COMMIT_ID))`, both `main.c`s) and a project-wide `VERSION_COMMIT_ID` preprocessor define wired through meson (`src/meson.build:30`, `src/meson.options`'s `commit_id` option) — but it carries **no build timestamp**, and **`xce-build.sh` never sets `-Dcommit_id=`**, so every locally-built binary (the entire `/xce-build` dev loop, including the one that produced the stale binary in the motivating incident) gets an **empty** commit id today. + - Worker uses plain `getopt_long()`, not `argp` — it has **no `--version` at all**. + - `git` is deliberately **not** a nix devShell buildInput (`src/nixSupport/xce.nix`) — the existing `commit_id` mechanism avoids shelling out to git entirely; the two production nix packages (`xce.nix:109`, `xcecli.nix:30`) derive it from the flake's own `self.shortRev`/`self.dirtyShortRev` introspection instead. Any new mechanism must respect this: no `run_command`/`custom_target` in `meson.build` may shell out to `git`. + - Go (apiserver, xcecli) and Python (xceadmin) have **zero** version infrastructure — no `-ldflags` injection anywhere in `src/tools/go_build` or `src/frontend/Makefile`, no `click.version_option`, no dynamic `pyproject.toml` version. + +## Research Log + +### Where does `xce-build.sh` (the actual dev-loop path) get its build metadata from? +- **Context**: the motivating incident happened via `/xce-build scheduler debug`, not `nix build` — need to know which computation path is actually load-bearing for fixing it. +- **Findings**: `xce-build.sh`'s `MESON_OPTS` array (set per-profile at `xce-build.sh:70-72`) never includes `-Dcommit_id=` or anything build-time related; `meson setup` is called with only `--buildtype`/`-Db_sanitize`. `git` IS available here despite not being a declared nix devShell buildInput, because `nix develop` (unlike a sandboxed `nix build`) layers onto the ambient host PATH rather than replacing it — this project's own scripts already run from inside a real git checkout. +- **Implications**: `xce-build.sh` must be the one to compute commit id (`git rev-parse --short HEAD`, `-dirty` suffixed via `git diff --quiet HEAD --` / `git status --porcelain`) and build time (`date -u`), independently of the nix-package path, and pass both via meson options. This is the fix that actually closes the incident's gap — the nix-package path already has a working (commit-id-only) mechanism. + +### Message/value plumbing per toolchain +- **Context**: three different build tool invocations (meson C custom target-free preprocessor define, Go's `go_build` shell script + `Makefile`, Python's `meson-build.sh` custom_target) need the same two values. +- **Findings**: `src/meson.build:30` already threads `commit_id` project-wide via `add_project_arguments()` for C. `src/tools/go_build` (invoked by both `apiserver/meson.build`'s and `xcecli/meson.build`'s `custom_target()`) already threads other env vars (e.g. CGO flags) into its `go build` invocation — same mechanism can carry new `XCE_GIT_COMMIT`/`XCE_BUILD_TIME` env vars for `-ldflags -X`. `src/xceadmin/meson.build`'s `custom_target` already passes several positional args to `meson-build.sh`; two more (or two new env vars) can carry the same values for a small generated Python module written before the `pip install` step. +- **Implications**: meson itself is the single source of truth for one build invocation — it resolves the two new options (`commit_id`, now actually SET by `xce-build.sh`; a new `build_time`) once, and every language-specific build step (C define, Go ldflags, Python generated module) reads from that same resolved value instead of independently recomputing it. Avoids the 3 toolchains ever disagreeing within one build. + +### Dirty-tree representation: separate flag vs. suffix convention +- **Context**: Requirement 1 AC3 requires distinguishing an uncommitted-changes build from an exact commit. +- **Findings**: both git's own `--dirty` flag and this project's existing Nix `self.dirtyShortRev` already use a suffix convention (`-dirty`) rather than a separate boolean field. +- **Implications**: reuse that convention — `commit_id`'s own string value carries `-dirty` when applicable, no third meson option/env var needed. Matches the Build-vs-Adopt lens (reuse an established convention already present twice in this codebase) and the Simplification lens (one string carries both facts). + +## Design Decisions + +### Decision: One resolved value pair, fed from two independent computation sites +- **Context**: production (`nix build`) and dev-loop (`xce-build.sh`) are architecturally different environments (fully sandboxed vs. ambient-PATH devShell) that already compute `commit_id` differently for exactly that reason. +- **Alternatives Considered**: + 1. Make `xce-build.sh` also avoid shelling out to git, deriving everything from Nix like the packages do — rejected, `xce-build.sh` doesn't invoke Nix package builds at all, it drives `meson` directly inside `nix develop`. + 2. Add a git-invoking `run_command()` inside `meson.build` itself, used by both paths — rejected, breaks the sandboxed `nix build` path where git is unavailable by design. +- **Selected Approach**: extend the existing split — `xce-build.sh` computes via `git`+`date` and passes `-Dcommit_id=`/`-Dbuild_time=`; the nix packages (`xce.nix`, `xcecli.nix`) compute via `self.shortRev`/`self.dirtyShortRev` (unchanged) + a new `builtins.currentTime`-derived value, passed the same way. +- **Rationale**: matches the codebase's own existing precedent exactly; zero new dependencies; keeps the sandboxed build's reproducibility model intact. +- **Trade-offs**: two computation sites to keep in sync (same risk the existing `commit_id`-only mechanism already carries and has lived with). +- **Follow-up**: verify during implementation that a plain `meson setup` with neither `xce-build.sh` nor a nix package (e.g. `meson setup build` run by hand) degrades to the "unknown" fallback (Requirement 1 AC4), not a build failure. + +### Decision: Worker gets a hand-rolled `--version` case, not a switch to argp +- **Context**: worker uses `getopt_long()`, not `argp`; controller/scheduler's free `--version` comes from argp's own auto-generated handling. +- **Alternatives Considered**: migrate worker's arg parsing to argp for consistency — rejected as disproportionate scope (a full parser migration to fix a missing flag). +- **Selected Approach**: add a `{"version", no_argument, ...}` long option and a `case`, printing the same `MK_VERSION`-style string then exiting 0 — mirrors argp's own output shape without adopting argp. +- **Rationale**: minimal, consistent output, zero risk to worker's existing option handling. + +## Risks & Mitigations +- Two independent computation sites (dev-loop vs. nix-package) could drift in format — mitigate by having both resolve to the exact same two meson option names and the exact same downstream consumption code, so only the *value-sourcing* differs, never the *shape*. +- `go_build`/`meson-build.sh` are shared shell scripts already carrying other env-var-driven behavior (CGO flags, schema sync) — new env vars must be additive/optional so a caller that doesn't set them still builds (falls back to "unknown"), not required. + +## References +- `src/meson.build:30`, `src/meson.options` — existing `commit_id` mechanism. +- `src/flake.nix:15-18`, `src/nixSupport/xce.nix:109`, `xcecli.nix:30` — existing Nix-side commit id derivation. +- `src/tools/xce-build.sh:70-98` — dev-loop meson invocation, current gap. +- `src/tools/go_build`, `src/frontend/apiserver/meson.build:53-62`, `src/frontend/xcecli/meson.build:9-17` — Go build plumbing. +- `src/xceadmin/meson.build:27-47`, `src/xceadmin/meson-build.sh` — Python build plumbing. diff --git a/.kiro/specs/build-time-versioning/spec.json b/.kiro/specs/build-time-versioning/spec.json new file mode 100644 index 00000000..51069ef0 --- /dev/null +++ b/.kiro/specs/build-time-versioning/spec.json @@ -0,0 +1,22 @@ +{ + "feature_name": "build-time-versioning", + "created_at": "2026-08-20T07:02:00Z", + "updated_at": "2026-08-20T09:20:00Z", + "language": "en", + "phase": "tasks-generated", + "approvals": { + "requirements": { + "generated": true, + "approved": true + }, + "design": { + "generated": true, + "approved": true + }, + "tasks": { + "generated": true, + "approved": true + } + }, + "ready_for_implementation": true +}