lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs(dirty-content-fingerprint): add spec (requirements, design, tasks)

Browse source 
Extends the already-shipped build-time-versioning feature so --version
output can distinguish different uncommitted-change states via a short
content-based fingerprint, instead of just a boolean -dirty flag.

Empirically confirmed during design (live probe against src/flake.nix,
reverted) that Nix's self.narHash is reachable at eval-time and content-
sensitive, resolving the spec's one open technical question for the
production build path without requiring git inside the sandbox.

Caught and corrected a design flaw before finalizing: an earlier approach
would have retired commit_id's -dirty suffix at its computation site,
which traces through to silently break the startup log's existing
modified-tree signal (same underlying value). Resolved by leaving
commit_id completely untouched everywhere and doing the -dirty-stripping
only inside --version's composition, only when a fingerprint is available
-- which also satisfies Requirement 5.2's fallback for free.

Independent task-graph sanity review (kiro-spec-tasks Step 3.5) returned
NEEDS_FIXES on 3 minor items; all fixed and self-verified before approval.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-08-21 03:45:02 +00:00
parent 084a5a95c3
commit d3c583b339
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
5 changed files with 514 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

237
.kiro/specs/dirty-content-fingerprint/design.md Normal file
View file

@ -0,0 +1,237 @@
# Design Document
## Overview
**Purpose**: This feature gives every XCE component's `--version` output the ability to distinguish two builds made from the same commit but with different uncommitted changes — replacing the existing plain `-dirty` boolean suffix (which cannot tell two dirty builds apart) with a short, content-derived fingerprint.
**Users**: Developers and operators iterating on a dev-loop build (`xce-build.sh`) who rebuild repeatedly while editing, and need to tell which specific edit produced the binary in front of them, without falling back to manual file-timestamp or diff inspection.
**Impact**: Extends the already-shipped `build-time-versioning` feature's commit_id/build_time mechanism with a third, optional value. `commit_id`, `build_time`, and the startup-log output are all left completely untouched — same computation, same value, same call sites. Only the `--version`-composition call site changes: when a fingerprint is available, it strips `commit_id`'s existing `-dirty` suffix (if present) and appends the fingerprint instead, producing a clean `<commit_id>-<hash>` shape without redundant markers; when no fingerprint is available, `--version` falls back to showing `commit_id` exactly as computed today (see `research.md`'s "Leave commit_id's computation fully untouched" decision).
### Goals
- A dirty build's `--version` output distinguishes itself from a different dirty build of the same commit, whenever their uncommitted content differs.
- Two builds with identical uncommitted content produce the identical fingerprint, reproducibly.
- A clean build's `--version` output is completely unchanged (just the commit identifier, as today).
- All six components (controller, scheduler, worker, apiserver, xcecli, xceadmin) get this capability.
- The daemon startup log line is untouched by this feature.
### Non-Goals
- Showing which files changed or their diff content anywhere in `--version` output (fingerprint only, not a diff viewer).
- Extending the startup log line to include the fingerprint (explicitly out of scope per Requirement 3).
- Closing the Nix production path's untracked-file blind spot (documented, accepted limitation — see `research.md`).
- Any change to `commit_id`'s computation, value, or its existing `-dirty` suffix, or to `build_time`'s computation — genuinely unaffected by this feature everywhere. The `-dirty`-stripping needed for a clean `--version` format is display-only logic inside the `--version`-composition call, not a change to `commit_id` itself.
## Boundary Commitments
### This Spec Owns
- Computation of a new `dirty_fingerprint` value at both existing computation sites (`xce-build.sh`, `flake.nix`).
- Threading that value through the existing single-resolved-value-pair meson-option mechanism as a third option.
- The `--version`-only composition logic that combines `commit_id` and `dirty_fingerprint` for display (including the `-dirty`-stripping rule, applied only at this one call site — see `research.md`).
### Out of Boundary
- The startup log line's content or format (unchanged; still shows only `commit_id`/`build_time`, per the original feature, via its completely untouched composition call site).
- The original feature's `commit_id`/`build_time` computation, values, or the `-dirty` suffix `commit_id` itself carries — genuinely unmodified everywhere, per the corrected design decision in `research.md`.
- Any mechanism for an operator to retrieve or view the actual uncommitted diff/file list (out of scope per requirements' Boundary Context).
- Making the Nix path's fingerprint reflect unstaged untracked files (documented limitation, not solved here).
### Allowed Dependencies
- The original `build-time-versioning` feature's meson option mechanism (`src/meson.options`, `src/meson.build`'s `add_project_arguments()`), extended with one new option, not replaced.
- `xce-build.sh`'s existing `commit_id`/`build_time` computation block and `MESON_OPTS` construction (task 2.1 of the original feature).
- `xce.nix`/`xcecli.nix`'s existing `preConfigure` build-phase hook (task 2.2 of the original feature).
- Each component's existing version-composition function/mechanism (`xce_compose_version()` for C, the Go ldflags vars, `_build_info`/`cli.py` for Python) — extended with one new optional field, not replaced.
### Revalidation Triggers
- `xce_compose_version()`'s signature or call sites change shape (would affect how the new field is threaded through).
- The original feature's `commit_id`/`build_time` meson options are renamed or restructured.
- `xce-build.sh`'s `MESON_OPTS` construction pattern changes.
- `self.narHash`'s semantics change in a future Nix version (unlikely, but the design's Nix-path assumption depends on it).
## Architecture
### Existing Architecture Analysis
- The original feature established: two computation sites (`xce-build.sh` for dev-loop, `xce.nix`/`xcecli.nix` for production) each resolve `commit_id`/`build_time` into meson options; six consumption sites each read those options once and never recompute. This feature adds a third value to that exact same shape.
- Each of the six components already has exactly one place where its version string is composed for `--version` and, separately (for the four daemons), one place where the startup log's version fields are logged — both currently reading the identical `commit_id`/`build_time` values. This feature's `--version`-only scope means the `--version` composition call site gains the new field while the startup-log composition call site does not — the first deliberate divergence between those two call sites in this feature family.
### Architecture Pattern & Boundary Map
Same two computation sites, one additional resolved value, fed only into each component's `--version` composition call — never into its startup-log call.
```mermaid
graph TB
DevScript[xce-build.sh git diff plus untracked files sha256]
NixPkg[flake.nix eval time self.narHash truncated]
MesonOpt[meson dirty_fingerprint option]
VersionCompose[per-component --version composition]
LogCompose[per-component startup log composition unchanged]
DevScript --> MesonOpt
NixPkg --> MesonOpt
MesonOpt --> VersionCompose
MesonOpt -.not consumed.-> LogCompose
```
**Architecture Integration**:
- Selected pattern: extend the existing single-resolved-value-pair fan-out with a third, optional value that only one of each component's two existing consumption call sites reads.
- Domain/feature boundaries: computation (2 sites, unchanged locations) stays separated from consumption; the new divergence is which consumption call site (per component) reads the new value, not a new computation or plumbing layer.
- Existing patterns preserved: meson option → C define / Go ldflags / Python generated module, identical to the original feature.
- New components rationale: no new components — every change extends an existing computation site or an existing composition call site with one additional field/argument.
- Steering compliance: no new external dependency; TDD required for every new behavior (per `testing.md`).
### Technology Stack
| Layer | Choice / Version | Role in Feature | Notes |
|-------|------------------|-----------------|-------|
| Dev-loop computation | `git diff HEAD` + `git ls-files --others --exclude-standard`, `sha256sum` (coreutils, already used by `xce-build.sh`) | Computes the fingerprint from tracked-modification diffs and untracked-file content | `xce-build.sh` |
| Nix production computation | `self.narHash` (Nix flake evaluator, eval-time) | Content hash of the dirty tree's tracked-file state, truncated the same way | `xce.nix`, `xcecli.nix`; empirically confirmed reachable and content-sensitive (see `research.md`) |
| C build | Meson preprocessor define (existing mechanism, one new define) | Carries `dirty_fingerprint` into `xce_compose_version()`'s `--version`-composition call only | `src/meson.build` |
| Go build | `-ldflags -X` (existing mechanism, one new var) | Injects `main.DirtyFingerprint`, read only by each binary's `--version`/`-version` handling | `go_build`, Makefile |
| Python build | Meson `custom_target` (existing mechanism, one new field) | `_build_info.py` gains a third field, read only by `cli.py`'s `click.version_option()` | `meson-build.sh` |
## File Structure Plan
### Modified Files
- `src/meson.options` — add a `dirty_fingerprint` string option (default `''`), sibling to the existing `commit_id`/`build_time`.
- `src/meson.build` — add `-DVERSION_DIRTY_FINGERPRINT=` to the existing `add_project_arguments()` call.
- `src/include/utils.h`, `src/lib/utils.c` — extend `xce_compose_version()` (or add a sibling composition function used only by `--version` call sites) to optionally append `-<dirty_fingerprint>` when non-empty; leave the existing signature's startup-log call sites unaffected (they simply don't pass the new field, or call the original unchanged function).
- `src/controller/main.c`, `src/scheduler/src/main.c`, `src/controller/testsuite/test.c`, `src/worker/src/main.c` — `--version`/version-flag handling call sites gain the new field; startup-log call sites are explicitly left unchanged (one-line, clearly-commented divergence — see `research.md`'s Design Decision).
- `src/tools/xce-build.sh` — compute `dirty_fingerprint` (git diff + untracked file content, sha256 truncated, empty when clean) and pass it via a new `-Ddirty_fingerprint=` entry in `MESON_OPTS`, alongside the existing `commit_id`/`build_time` entries.
- `src/flake.nix` — the existing eval-time `let` block gains `myDirtyFingerprint` (derived from `self.narHash`, truncated, empty on a clean tree), computed alongside `myCommitId` and passed into `nixSupport/default.nix` as a new argument, the same way `myCommitId` already is.
- `src/nixSupport/default.nix` — accepts and forwards `myDirtyFingerprint` into each `callPackage ./xce.nix`/`./xcecli.nix` call via `inherit`, alongside the existing `myCommitId`.
- `src/nixSupport/xce.nix`, `src/nixSupport/xcecli.nix` — accept the new `myDirtyFingerprint` argument and append `"-Ddirty_fingerprint=" + myDirtyFingerprint` to the existing `mesonFlags` list (eval-time, alongside `-Dcommit_id=`) — not through the `preConfigure`/`mesonFlagsArray` mechanism, which stays reserved for `build_time`.
- `src/tools/go_build` — read `XCE_DIRTY_FINGERPRINT` (empty-safe) and append it to the existing `-ldflags -X` construction as `main.DirtyFingerprint`.
- `src/frontend/apiserver/meson.build`, `src/frontend/xcecli/meson.build` — pass the new env var into `go_build`'s env, same pattern as `XCE_GIT_COMMIT`/`XCE_BUILD_TIME`.
- `src/frontend/Makefile` — compute and pass the same `-ldflags -X main.DirtyFingerprint=...` for `build-server`/`build-cli`'s dev-loop path.
- `src/frontend/apiserver/cmd/main.go` — declare `var DirtyFingerprint = ""`; `-version` flag's composed string appends it when non-empty; startup log's `logger.Info(...)` call is left unchanged (no new zap field).
- `src/frontend/xcecli/cmd/main.go` — declare the same var; the version string passed to `router.Execute(cm, version)` (which sets `rootCmd.Version`) appends it when non-empty.
- `src/xceadmin/meson.build` — pass one new positional arg (or env var) to `meson-build.sh`.
- `src/xceadmin/meson-build.sh` — `_build_info.py` gains a third field, `DIRTY_FINGERPRINT` (empty-string fallback, matching the existing per-field `unknown`-vs-empty convention used for the other two fields — see Implementation Notes).
- `src/xceadmin/src/xceadmin/_build_info.py` (committed fallback stub) — gains `DIRTY_FINGERPRINT = ""` alongside the existing two `"unknown"` fields.
- `src/xceadmin/src/xceadmin/cli.py` — `_compose_version()` appends the fingerprint when non-empty.
> No new files. Every change extends an existing computation site, an existing meson option chain, or an existing composition call site.
## Requirements Traceability
| Requirement | Summary | Components | Interfaces | Flows |
|-------------|---------|------------|------------|-------|
| 1.1, 1.2, 1.3 | Content-derived indicator present when dirty, deterministic, distinguishes different uncommitted content | `xce-build.sh` (git diff + untracked files), nix `self.narHash` | `dirty_fingerprint` meson option, per-component `--version` composition | Computation → meson → `--version` composition |
| 1.4 | Excludes ignored files | `xce-build.sh`'s `git ls-files --others --exclude-standard` (respects `.gitignore` natively); nix's `self.narHash` (only ever sees files the flake's git input would include, which already excludes ignored files) | n/a | n/a |
| 2.1 | Clean tree shows commit id alone | All 6 `--version` composition call sites | `dirty_fingerprint` empty string → no suffix appended | n/a |
| 3.1, 3.2 | `--version`-only scope; startup log unchanged | Each daemon's two separate composition call sites | Startup-log call site never receives `dirty_fingerprint` | n/a |
| 4.1 | Applies to all 6 components | All 6 consumption sites | Same as original feature's Requirement 1.1/1.2 fan-out, one new field | n/a |
| 5.1, 5.2 | Graceful degradation when content can't be determined | Every `--version` composition call site (falls back to `commit_id` as-is when `dirty_fingerprint` is empty) | `dirty_fingerprint` empty on computation failure; `commit_id`'s own pre-existing, untouched `-dirty` suffix supplies the "still indicate modified" signal for free | n/a |
## Components and Interfaces
### C Daemons (controller, scheduler, worker)
#### Version composition, `--version`-only extension
| Field | Detail |
|-------|--------|
| Intent | Append a content-derived fingerprint to `--version`'s commit id when the tree is dirty, without touching the startup log's version fields |
| Requirements | 1.1, 1.2, 1.3, 1.4, 2.1, 3.1, 3.2, 4.1 |
**Responsibilities & Constraints**
- `commit_id` and its existing `-dirty` suffix logic (from the original `build-time-versioning` feature) are **not modified** — identical computation, identical value, at both computation sites.
- `xce_compose_version()` (or a new sibling function reusing its internals) gains an optional `dirty_fingerprint` parameter, consumed **only by the `--version` call path**: when `dirty_fingerprint` is non-empty, the composition strips a trailing `-dirty` from `commit_id` (if present) and appends `-<dirty_fingerprint>` instead, producing `<commit_id>-<hash>`; when `dirty_fingerprint` is empty, the composition uses `commit_id` completely as-is — which already correctly shows either the bare hash (clean tree) or `<hash>-dirty` (tree is dirty but the fingerprint computation failed, Requirement 5.2's fallback, satisfied by the pre-existing mechanism with no new sentinel value needed).
- Each `main.c`'s `--version`/`argp_program_version` call site passes the new field; the corresponding startup-log INFO call site is **not touched at all** — it keeps calling the original, unmodified composition path exactly as the original feature shipped it, so it never even references `dirty_fingerprint`.
- Worker's hand-added `--version` case (from the original feature) gets the same treatment as controller/scheduler.
**Contracts**: Service [ ] / API [ ] / Event [ ] / Batch [ ] / State [ ]
**Implementation Notes**
- Integration: extends `xce_compose_version()` — check during implementation whether adding a parameter to the existing function is cleaner than adding a second function; either is acceptable as long as the startup-log call site's behavior is provably unchanged (byte-for-byte) by a regression test.
- Validation: (1) a build with `dirty_fingerprint` empty and a clean tree must produce `--version` output byte-identical to the original feature's already-shipped clean-tree behavior (`<commit_id>` alone); (2) a build with `dirty_fingerprint` empty and a dirty tree (fingerprint computation failed) must produce `--version` output byte-identical to the original feature's already-shipped dirty-tree behavior (`<commit_id>-dirty`) — Requirement 5.2, satisfied without new code beyond "don't touch commit_id when dirty_fingerprint is empty"; (3) a build with a real, non-empty `dirty_fingerprint` must produce `<bare-commit-hash>-<dirty_fingerprint>`, with the `-dirty` suffix stripped, not doubled.
- Risks: (1) accidentally threading the new field into the startup-log call site would silently violate Requirement 3.2 — must be caught by a test asserting the startup log line is unchanged from the original feature's already-shipped format; (2) the `-dirty`-stripping logic must anchor to the literal trailing suffix (not a blind substring removal) — covered by a dedicated unit test.
### Go Binaries (apiserver, xcecli)
#### ldflags fingerprint injection, `--version`/`-version`-only
| Field | Detail |
|-------|--------|
| Intent | Same `--version`-only extension for the Go toolchain |
| Requirements | 1.1, 1.2, 1.3, 1.4, 2.1, 3.1, 3.2, 4.1 |
**Responsibilities & Constraints**
- `go_build` reads `XCE_DIRTY_FINGERPRINT` (empty-safe) and appends `-X main.DirtyFingerprint=$v` to its existing `-ldflags` construction. `GitCommit` itself (and whatever `-dirty` suffix it already carries) is completely untouched.
- `apiserver/cmd/main.go`: the `-version` flag's composition applies the same strip-and-append rule as the C daemons — if `DirtyFingerprint` is non-empty, strip a trailing `-dirty` from `GitCommit` and append `-DirtyFingerprint`; if empty, print `GitCommit` as-is (already correctly showing `-dirty` or not). The existing `logger.Info("********** Initialize Logger done!")` structured-field call is left completely unchanged (no new zap field, `GitCommit` passed exactly as before) — this is the explicit Requirement 3.2 boundary for this component.
- `xcecli/cmd/main.go`: the version string passed into `router.Execute(cm, version)` applies the same strip-and-append rule; xcecli has no startup log (already established as out of scope by the original feature), so there is no divergence to manage here.
**Contracts**: Service [ ] / API [ ] / Event [ ] / Batch [ ] / State [ ]
**Implementation Notes**
- Integration: reuses `go_build`'s existing env-var pattern, one more variable.
- Validation: apiserver's startup log must be provably unchanged (byte-for-byte against the original feature's shipped format) even when `DirtyFingerprint` is non-empty.
- Risks: none identified beyond the same one already flagged for C — accidental leakage of the new field into the startup-log call site.
### Python (xceadmin)
#### Generated build-info module, third field
| Field | Detail |
|-------|--------|
| Intent | Same `--version`-only extension for xceadmin, which has no startup log at all (one-shot CLI, already out of scope per the original feature) |
| Requirements | 1.1, 1.2, 1.3, 1.4, 2.1, 4.1 |
**Responsibilities & Constraints**
- `meson-build.sh` writes `DIRTY_FINGERPRINT` (empty-string fallback) into `_build_info.py` alongside the existing `GIT_COMMIT`/`BUILD_TIME` fields. `GIT_COMMIT` itself (which already carries a `-dirty` suffix on a dirty tree, inherited from the shared `commit_id` meson option) is completely untouched.
- The committed fallback stub (`src/xceadmin/src/xceadmin/_build_info.py`) gains `DIRTY_FINGERPRINT = ""` — an editable install (`pip install -e .`) shows no fingerprint suffix, consistent with the stub's existing all-`unknown`/empty convention.
- `cli.py`'s `_compose_version()` applies the same strip-and-append rule as the other components: if `DIRTY_FINGERPRINT` is non-empty, strip a trailing `-dirty` from `GIT_COMMIT` and append `-DIRTY_FINGERPRINT`; if empty, use `GIT_COMMIT` as-is.
**Contracts**: Service [ ] / API [ ] / Event [ ] / Batch [ ] / State [ ]
**Implementation Notes**
- Integration: one more field in the same generated-module mechanism the original feature already built (task 5.1/5.2).
- Validation: editable install must still show no fingerprint (empty field, no import error); wheel install (built dirty) must show a real, non-empty fingerprint.
- Risks: none identified — xceadmin already has no startup log to accidentally leak into.
### Build Orchestration
#### Dev-loop fingerprint computation
| Field | Detail |
|-------|--------|
| Intent | Compute a content-based fingerprint of the uncommitted tree, with zero working-tree mutation |
| Requirements | 1.1, 1.2, 1.3, 1.4, 5.1, 5.2 |
**Responsibilities & Constraints**
- `commit_id` and its existing `-dirty` suffix (`xce-build.sh` task 2.1 of the original feature) are **not touched** by this computation — this task only adds a new, independent `dirty_fingerprint` computation alongside it.
- Computes `dirty_fingerprint` as one of two states: **empty** when the tree is clean, *or* when the tree is dirty but the hash computation itself fails (in the latter case, `commit_id`'s own pre-existing, untouched `-dirty` suffix already signals "modified" once `--version` falls back to using it as-is — Requirement 5.2, satisfied with no new sentinel value); a **real hash** (`sha256sum` truncated to a short hex prefix) over the concatenation of `git diff HEAD`'s output plus the raw content of each file `git ls-files --others --exclude-standard` lists, in stable sorted order, when the tree is dirty and the computation succeeds.
**Contracts**: Service [ ] / API [ ] / Event [ ] / Batch [ ] / State [ ]
**Implementation Notes**
- Integration: added to the same `xce-build.sh` block that already computes `commit_id`/`build_time` (original feature's task 2.1), no restructuring.
- Validation: the exact scenario this feature exists for — edit a file, build, note the fingerprint; edit further, rebuild, confirm a different fingerprint; revert to the first edit's exact content, rebuild, confirm the fingerprint matches the first build's again.
- Risks: `git diff HEAD` output size for a very large uncommitted change set is unbounded in principle — acceptable for this codebase's realistic dev-loop usage (source-code-sized diffs), not a concern requiring truncation of the *input*, only the *output* hash.
#### Nix production fingerprint computation
| Field | Detail |
|-------|--------|
| Intent | Same capability for the Nix production path, within its sandboxed, no-git constraint |
| Requirements | 1.1, 1.2, 1.3, 5.1 |
**Responsibilities & Constraints**
- `myCommitId` and its existing `self.dirtyShortRev`-derived `-dirty` suffix (original feature) are **not touched** by this computation.
- `myDirtyFingerprint` is derived from `self.narHash` at **eval time**, in `flake.nix`'s existing `let` block, alongside `myCommitId` — **not** via a `preConfigure` build-phase hook. Unlike `build_time` (which needed a build-phase hook specifically because `builtins.currentTime` breaks pure-eval), `self.narHash` is a legitimate flake self-introspection field with no such trap — empirically confirmed reachable and stable under plain `nix build`/`nix eval` (no `--impure`) in `research.md`. It is passed into `xce.nix`/`xcecli.nix` (via `nixSupport/default.nix`) as a new function argument, mirroring `myCommitId`'s existing parameter, and appended to each derivation's `mesonFlags` list directly (`"-Ddirty_fingerprint=" + myDirtyFingerprint`), the exact same way `-Dcommit_id=` already is — not through `mesonFlagsArray`/`preConfigure`, which stays reserved for genuinely build-phase-only values like `build_time`.
- Resolves to the same two states as the dev-loop path: **empty** when the tree is clean, checked the same way `myCommitId` already branches on `hasAttr "shortRev"` (a clean tree only has `self.shortRev`, not `self.dirtyShortRev`); otherwise a **real hash**, truncated to match the dev-loop path's format.
- **Known, accepted limitation** (see `research.md`): does not reflect brand-new untracked files unless staged. Must be documented as a code comment at this `let`-binding, not silently left as a surprise.
**Contracts**: Service [ ] / API [ ] / Event [ ] / Batch [ ] / State [ ]
**Implementation Notes**
- Integration: extends `flake.nix`'s existing eval-time `myCommitId` computation with one sibling value, passed through to `xce.nix`/`xcecli.nix` as one more `mesonFlags` entry — no new build-phase mechanism, no change to the existing `build_time` `preConfigure` hook.
- Validation: `nix build .#xcecore_debug` (no `--impure`, the documented canonical command) must still succeed and produce a binary whose `--version` fingerprint changes when a tracked file in the flake's source is edited and the flake is re-evaluated.
- Risks: low — `research.md` already empirically confirmed `self.narHash` is reachable and content-sensitive without `--impure`; the implementer should still re-confirm live as part of RED-phase validation rather than merely trusting this document, since the earlier `build_time`/`builtins.currentTime` mistake in the original feature came from exactly that kind of unverified assumption.
## Testing Strategy
### Unit Tests
- C: `test_utils.c` cases asserting `xce_compose_version()`'s `--version`-path composition: (1) `dirty_fingerprint` non-empty + `commit_id` ending in `-dirty` → strips the suffix, appends the fingerprint (`<hash>-<fingerprint>`, not `<hash>-dirty-<fingerprint>`); (2) `dirty_fingerprint` empty + clean `commit_id` → `<hash>` alone, unchanged; (3) `dirty_fingerprint` empty + `commit_id` ending in `-dirty` (fingerprint computation failed, tree still dirty) → `<hash>-dirty`, Requirement 5.2's fallback, produced by simply not touching `commit_id` — no special-casing needed in the composition function beyond the empty check.
- Shell: a unit-level check of `xce-build.sh`'s fingerprint-computation logic confirms it emits a real hash when the tree is dirty and the computation succeeds, and confirms it emits empty (not a fabricated value) both when the tree is clean and when the hash-computation step is made to fail (simulated) on a dirty tree — the latter case relying on `commit_id`'s own separate, untouched `-dirty` computation to still convey "modified" once composed.
- Go: a test asserting `-version`/`--version` output includes `DirtyFingerprint` when set, and that apiserver's startup-log-composition path is unaffected by its presence.
- Python: a `pytest` case asserting `xceadmin --version` includes the fingerprint field when `_build_info.DIRTY_FINGERPRINT` is non-empty, and omits any suffix when empty (both the committed-stub and generated-module cases, mirroring the original feature's task 5.2 test pattern).
### Integration Tests
- `xce-build.sh`-driven dev-loop build (any C daemon): edit a tracked file, build, capture `--version`'s fingerprint; edit again with different content, rebuild, confirm the fingerprint differs; revert to the first edit's exact content, rebuild, confirm the fingerprint matches the first capture exactly. This is the direct regression test for Requirement 1's core capability.
- Same daemon, confirm the startup log line is untouched (byte-identical to the original `build-time-versioning` feature's already-shipped format) regardless of whether the tree is dirty.
- A clean-tree build (`git stash` the dev tree, or run in a freshly-cloned worktree) confirms `--version` shows the commit id alone, no suffix — the direct regression test for Requirement 2.
### E2E / Consistency
- For the Nix path: confirm `nix build .#xcecore_debug` (no `--impure`) succeeds and the resulting binary's `--version` fingerprint changes when a tracked source file is edited before evaluation — the direct regression test for the `self.narHash`-based mechanism actually working end-to-end, not just at the `nix eval` probe level this design's research was based on.

56
.kiro/specs/dirty-content-fingerprint/requirements.md Normal file
View file

@ -0,0 +1,56 @@
# Requirements Document
## Project Description (Input)
Dirty-content fingerprint for --version output: extends the build-time-versioning feature (already shipped) so the version string can distinguish between different uncommitted-change states at build time, not just a boolean "-dirty" flag. Today, xce-build.sh computes commit_id as `git rev-parse --short HEAD` with a plain "-dirty" suffix when the tree has uncommitted changes -- this means two different builds made from the same dirty commit (edit file, rebuild, edit further, rebuild again) are indistinguishable from their version string alone, since the suffix never changes as long as *some* uncommitted change exists.
New behavior: when the source tree is clean, --version continues to show just `<commit_id>` (no change). When the tree is dirty, --version shows `<commit_id>-<hash>` where `<hash>` is a short (e.g. 8 hex char) content-based digest computed from exactly what `git add -A && git commit` would capture -- i.e. `git diff HEAD`'s content for modified/staged tracked files, plus the full content of untracked files. Any actual content change (even to the same set of files) must change the hash; an unchanged uncommitted state must produce the same hash across repeated builds. This is a *stricter* version of the same requirement the original feature already implements ("mark version info as modified when uncommitted changes exist"), not a new requirement.
**Superseded during design** (kept here for input-history fidelity, not as authoritative intent -- see `research.md`'s "Leave commit_id's computation fully untouched" decision for the resolved approach): an earlier framing of this input proposed that "the literal '-dirty' string is retired in favor of the hash's mere presence." That approach was rejected after design tracing revealed it would silently break the startup log's existing modified-tree signal (the startup log reads the same `commit_id` value `--version` does). The resolved design instead leaves `commit_id`'s `-dirty` suffix completely untouched everywhere, and strips it only inside the `--version`-composition call, only when a fingerprint is actually available.
Scope: --version output only, for all six components (controller, scheduler, worker, apiserver, xcecli, xceadmin) -- the existing startup log line (controller/scheduler/worker/apiserver) is explicitly OUT of scope for this enhancement and keeps its current behavior/format unchanged; this is a deliberate, scoped exception to the original feature's guarantee that `--version` and the startup log never disagree, analogous to how xcecli's startup log was already excluded in the original spec for being a one-shot CLI.
Open technical question requiring investigation during design (light discovery, extension feature): the project's dev-loop build path (xce-build.sh) has real `git` available and can straightforwardly compute this hash via `git diff HEAD` + untracked file contents. The production Nix build path (xce.nix/xcecli.nix) runs in a sandboxed derivation without `git` on PATH by design (this repo deliberately keeps git out of the nix devShell buildInputs) and currently gets its "-dirty" marker for free from Nix's own eval-time `self.dirtyShortRev` introspection, which is NOT a content hash (it can't distinguish two different dirty states either). Design must investigate whether Nix's flake evaluator exposes an equivalent eval-time content hash (e.g. `self.narHash`) that could serve this purpose without shelling out to git inside the sandbox -- verify this empirically rather than assuming. If no such eval-time-safe content hash exists for the sandboxed path, document that as a real constraint and decide a reasonable fallback -- do not invent a mechanism that would require adding git as a nix sandbox dependency, since that was an explicit prior architectural decision in the original spec's research.
Hash algorithm/length is an implementation detail for design to decide (short, stable, no external dependency beyond what's already available in each toolchain/environment).
## Boundary Context (Optional)
- **In scope**: computing and displaying, in `--version` output only, a content-derived indicator that distinguishes different uncommitted-change states, for controller, scheduler, worker, apiserver, xcecli, and xceadmin; graceful degradation when that indicator cannot be determined in a given build environment.
- **Out of scope**: any change to the startup log format of controller, scheduler, worker, or apiserver (unchanged by this feature); the underlying commit identifier / build timestamp mechanism already shipped by the build-time-versioning feature (extended, not replaced); exposing the names of changed files or diff content anywhere in `--version` output (this feature only makes two builds comparable for content-equality, it does not reveal what changed); a tool for comparing or explaining the difference between two builds; retrofitting this indicator into already-built/deployed binaries.
- **Adjacent expectations**: this feature depends on the already-shipped build-time-versioning feature's commit identifier and build timestamp display; it is additive to that mechanism. An operator wanting to know exactly what changed between two builds is still expected to inspect the source tree themselves (e.g. via `git diff`); this feature only tells them whether two builds' uncommitted content was identical or different, it does not show the content itself.
## Requirements
### Requirement 1: Content-Based Distinguishability of Uncommitted State
**Objective:** As an operator comparing two builds made from the same base commit at different points during active development, I want their version identifiers to differ whenever the uncommitted content differs, so that I can tell apart two builds that a plain "modified" marker would otherwise show as identical.
#### Acceptance Criteria
1. When the source tree contains uncommitted changes at build time, the system shall include, alongside the build's commit identifier, a content-derived indicator reflecting the actual content of those uncommitted changes.
2. If two builds are produced from the same commit but with different uncommitted content, then the system shall produce different content-derived indicators for the two builds.
3. If two builds are produced from the same commit with identical uncommitted content, then the system shall produce the identical content-derived indicator for both builds.
4. The system shall derive the content-derived indicator only from files that would be included by committing all current changes (tracked-file modifications and untracked files), excluding any file the source tree's ignore rules exclude from commits.
### Requirement 2: Clean-Tree Behavior Unchanged
**Objective:** As an operator inspecting a build produced from an exact, unmodified commit, I want its version identifier to remain exactly the commit identifier with no additional marker, so a clean build stays as concise as it is today and is never confused with a modified one.
#### Acceptance Criteria
1. While the source tree contains no uncommitted changes at build time, the system shall present the version identifier as the commit identifier alone, with no content-derived indicator or other modified-state marker appended.
### Requirement 3: Scope — Version Query Surface Only
**Objective:** As an operator, I want the content-derived indicator available when I explicitly query a component's version, without changing the routine startup log output I already rely on.
#### Acceptance Criteria
1. When an operator queries a component's version via its version-query command, the system shall include the content-derived indicator described in Requirement 1 for that build.
2. The system shall present the commit identifier and build timestamp in the startup log of controller, scheduler, worker, and apiserver without the content-derived indicator, leaving the existing startup log format unchanged.
### Requirement 4: Applies to Every Version-Displaying Component
**Objective:** As an operator, I want this distinguishing capability available consistently across every component that already displays version information, so I do not need to remember which components support it.
#### Acceptance Criteria
1. The system shall provide the content-derived indicator described in Requirement 1 for controller, scheduler, worker, apiserver, xcecli, and xceadmin.
### Requirement 5: Graceful Degradation When Content Cannot Be Determined
**Objective:** As a developer running the build tooling in an environment where the uncommitted content cannot be reliably inspected, I want the build to still succeed and clearly indicate the limitation, so that I never see a fabricated or misleading indicator.
#### Acceptance Criteria
1. If the build tooling cannot determine the content of uncommitted changes at build time, then the system shall omit the content-derived indicator rather than fabricate one.
2. If the build tooling cannot determine the content of uncommitted changes at build time while the source tree does have uncommitted changes, then the system shall still indicate that the version information reflects a modified tree, so the operator is not misled into believing the build is clean.

79
.kiro/specs/dirty-content-fingerprint/research.md Normal file
View file

@ -0,0 +1,79 @@
# Research & Design Decisions
## Summary
- **Feature**: `dirty-content-fingerprint`
- **Discovery Scope**: Extension (builds directly on the shipped `build-time-versioning` feature's commit_id/build_time mechanism)
- **Key Findings**:
- Nix's flake evaluator exposes `self.narHash` at eval-time, reachable from inside `xce.nix`/`xcecli.nix` exactly like the already-used `self.shortRev`/`self.dirtyShortRev` — empirically confirmed content-sensitive (different tracked-file content produces a different hash; identical content reproduces the identical hash) and requires no `git` on the sandboxed build's PATH, respecting the same architectural constraint the original feature's research already established.
- `self.narHash` has a real, empirically-confirmed blind spot: it reflects modifications to already-**tracked** files, but does **not** pick up brand-new **untracked** files unless they are `git add`-staged first. This differs from the dev-loop path's planned `git diff HEAD` + untracked-file-content approach, which naturally includes unstaged untracked files.
- The dev-loop path (`xce-build.sh`) has real `git` available (established precedent from the original feature's task 2.1) and can compute an untracked-file-inclusive fingerprint directly via `git diff HEAD` plus `git ls-files --others --exclude-standard` (the exact enumeration `git add -A` would use, already respecting `.gitignore`), with zero working-tree mutation.
- `commit_id`'s existing computation (and its `-dirty` suffix) must stay completely untouched by this feature -- an earlier design draft that retired `-dirty` at the computation site was caught and corrected after tracing its blast radius onto the startup log, which reads the same `commit_id` value and would have silently lost its only modified-tree signal. The `-dirty`-stripping needed for a clean `--version` format instead happens only inside the `--version`-composition call, only when a fingerprint is actually available (see Design Decisions).
## Research Log
### Does Nix expose an eval-time content hash of the dirty tree, usable inside the sandboxed build without `git` on PATH?
- **Context**: The original build-time-versioning feature's nix path (`xce.nix`/`xcecli.nix`) gets its `-dirty` marker for free from `self.dirtyShortRev`, which is a boolean-ish suffix, not a content hash — it can't distinguish two different dirty states from each other, the same limitation this feature exists to fix for the dev-loop path. Needed to determine, empirically, whether an equivalent content-sensitive value exists for the nix path without shelling out to git inside the sandbox (which would violate the original feature's explicit "no git as a nix sandbox dependency" architectural decision).
- **Sources Consulted**: Live empirical testing against this repo's actual `src/flake.nix`, via `nix flake metadata ./src --json` and a temporary `debugNarHash = self.narHash;` output added to and then reverted from `src/flake.nix`.
- **Findings**:
1. `nix flake metadata ./src --json`'s `locked.narHash` field is present even for a dirty git-type flake input, and is reachable from inside the flake's own expressions as `self.narHash` (confirmed via `nix eval ./src#debugNarHash --raw`, temporarily added then reverted).
2. Editing a tracked file in the flake's directory (`src/flake.nix` itself, via a temporary comment line) changed `narHash` from `sha256-50twgW+...` to `sha256-FEEP3A...`; a second, differently-worded edit changed it again to `sha256-CdD8g1...`; reverting the file to its exact original committed content (via `git checkout --`) reproduced the exact original `narHash` (modulo unrelated background tree changes between captures -- see caveat below). This confirms `narHash` is a genuine, deterministic content hash of tracked-file state, not merely a "some change exists" boolean.
3. Creating a new, unstaged, untracked file (`src/README.md`) inside the flake's directory did **not** change `narHash` at all -- even after further editing that same untracked file's content. Only after `git add`-staging it (still uncommitted) did `narHash` change. This is a real, confirmed blind spot: unstaged untracked files are invisible to Nix's dirty-tree evaluation.
4. Caveat on finding #2's revert check: the final "revert to baseline" capture did not exactly match the very first baseline capture, because unrelated `.wolf/*` bookkeeping files (tracked, modified by this session's OpenWolf hook automation between captures) also feed into the same repo-wide `narHash`. This is expected -- `narHash` reflects the *entire* dirty tree's tracked-file state, not just the file under test -- and does not undermine the core finding (content-determinism), it only means two captures separated by real intervening changes elsewhere in the tree will legitimately differ, exactly as intended.
- **Implications**: `self.narHash` is a viable, eval-time-safe basis for a nix-path content fingerprint, with one documented, accepted limitation: it will not reflect a build made dirty *only* by adding new untracked files that were never staged. Given the nix production path is not the path implicated in the original motivating incident (that was `xce-build.sh scheduler debug`), and given a workaround requiring `git add` or `git` on the sandbox's PATH was already ruled out by the original feature's architectural decision, this asymmetry is accepted and documented rather than engineered around.
### Untracked-file enumeration for the dev-loop path, without mutating the working tree
- **Context**: The dev-loop path (`xce-build.sh`) needs to include untracked files' content in its fingerprint (unlike the nix path, which has the staging blind spot above) while making zero changes to the developer's working tree (no `git add`, no `git stash`).
- **Findings**: `git ls-files --others --exclude-standard` lists exactly the untracked files that are not excluded by `.gitignore`/`.git/info/exclude` -- the same set `git add -A` would pick up -- without touching the index or working tree. Combined with `git diff HEAD` (which already covers all tracked modifications, staged or not, relative to `HEAD`), concatenating `git diff HEAD`'s output with the raw content of each file `git ls-files --others --exclude-standard` lists (in a stable, sorted order) and hashing the result gives a fingerprint matching Requirement 1's AC4 exactly ("only from files that would be included by committing all current changes... excluding ignored files").
- **Implications**: No new git subcommands beyond what's already used elsewhere in this codebase (`git diff`, `git status --porcelain` equivalents); no working-tree mutation; deterministic given a fixed input order.
## Architecture Pattern Evaluation
| Option | Description | Strengths | Risks / Limitations | Notes |
|--------|-------------|-----------|---------------------|-------|
| Single shared meson option (matches original feature's pattern) | `xce-build.sh`/nix each compute their own fingerprint value, both feed a new `dirty_fingerprint` meson option, consumed once by C/Go/Python exactly like `commit_id`/`build_time` already are | Reuses the exact, already-proven single-resolved-value-pair architecture; zero new plumbing pattern to learn | Two computation sites, asymmetric capability (nix misses unstaged untracked files) | Selected -- see Decision below |
| Compute fingerprint independently per consumption site | Each of the 6 components computes its own fingerprint at its own build step | No new meson option | Directly reintroduces the exact multi-computation-site drift risk the original feature's architecture was designed to avoid | Rejected |
## Design Decisions
### Decision: Extend the existing single-resolved-value-pair architecture with a third value
- **Context**: The original feature already established "one resolved value pair (commit_id, build_time), two computation sites, six consumption sites reading the same values, never recomputing." This feature adds a third value (the content fingerprint) that is conceptually parallel to `commit_id` (both describe "what source produced this build") but is only ever non-empty when the tree is dirty.
- **Alternatives Considered**:
1. Fold the fingerprint directly into `commit_id`'s own string (e.g., `xce-build.sh` composes `<hash>-<content_hash>` and passes that whole string as `commit_id`) -- rejected: conflates two independently-meaningful values into one opaque string, makes Requirement 2 (clean tree = commit id alone, byte-identical to today) awkward to guarantee at the composition site instead of trivially true by construction.
2. A wholly separate new meson option (`dirty_fingerprint`), computed alongside `commit_id`/`build_time` by the same two computation sites, consumed only by the composition step that already builds the final version string for `--version` (not by the startup-log composition path) -- selected.
- **Selected Approach**: Add `dirty_fingerprint` as a new meson string option (default `''`), computed by `xce-build.sh` (git diff + untracked file content, sha256 truncated) and by `xce.nix`/`xcecli.nix` (`self.narHash`, truncated the same way, empty when the tree is clean since there is nothing to distinguish). The existing C `xce_compose_version()`/Go ldflags/Python `_build_info` consumption sites gain a fourth field, used only by the `--version`-composition call site of each component, not by the startup-log composition call site -- this is the concrete mechanism satisfying Requirement 3's "--version only, startup log unchanged" boundary, and requires each of the 6 components' `main`/`cli.py` to call the composition function twice with different arguments (once for `--version`'s output, once for the startup log's, as today) rather than sharing one fully-identical string as the original feature's Requirement 2.2 did for `commit_id`/`build_time`.
- **Rationale**: Keeps `commit_id` exactly as-is (Requirement 2 falls out for free -- a clean tree's `dirty_fingerprint` is simply empty, so nothing is appended), reuses the proven two-computation-sites/single-value-pair shape, and cleanly isolates the `--version`-only scope to a single new field rather than a startup-log/version-string fork.
- **Trade-offs**: The two composition call sites (startup log vs. `--version`) per component, previously guaranteed identical by construction (Requirement 2.2 of the original feature), now diverge by design for this one new field -- acceptable and explicitly scoped by this feature's Requirement 3, but must be clearly commented in each component's source so a future change doesn't "fix" the divergence by accident.
- **Follow-up**: Verify during implementation that omitting `dirty_fingerprint` from the startup-log composition call is a one-line, obviously-intentional diff (not something that looks like an oversight) in each of controller/scheduler/worker/apiserver's `main.c`/`main.go`.
### Decision: Leave commit_id's computation fully untouched; strip its -dirty suffix only inside --version's composition, only when a fingerprint is available
- **Context**: The already-shipped `build-time-versioning` feature's `commit_id` computation unconditionally appends a literal `-dirty` suffix when the tree has uncommitted changes (`xce-build.sh` task 2.1, `flake.nix`'s `self.dirtyShortRev`), and **both** the `--version` output and the startup log read that same `commit_id` value today. An earlier draft of this decision proposed retiring `-dirty` from `commit_id`'s own computation entirely -- but tracing that change's full blast radius revealed a real regression: since the startup log reads the identical `commit_id` value, removing `-dirty` at the computation site would silently strip the startup log's only modified-tree signal too, even though this feature's Requirement 3 explicitly requires the startup log to be left unchanged. That draft under-scoped its own blast radius by only considering the `--version` surface.
- **Alternatives Considered**:
1. Retire `-dirty` from `commit_id`'s computation entirely (both computation sites) -- rejected after tracing the blast radius: silently regresses the startup log's existing modified-tree signal, a real behavior change to a surface this feature must leave untouched.
2. Leave `commit_id`'s `-dirty` suffix untouched everywhere, append `dirty_fingerprint` after it unconditionally when non-empty -- rejected: produces the redundant `<hash>-dirty-<fingerprint>` shape when a fingerprint is available, not the plain `<commit_id>-<hash>` shape the requirements were written around (the user's own stated preference during requirements gathering: "just commit id - hash of changed file contents, if no file has been changed just commit id is enough").
3. Leave `commit_id`'s computation **completely unmodified** everywhere (both computation sites, and therefore the startup log too); perform the `-dirty`-stripping **only inside the `--version`-composition call**, and **only when `dirty_fingerprint` is non-empty** -- **selected**.
- **Selected Approach**: `commit_id` is never touched by this feature -- identical computation, identical value, at both computation sites, for both consumption surfaces. `dirty_fingerprint` is computed independently (empty when clean or when the tree is dirty but the hash computation fails; a real content hash when the tree is dirty and computable). The `--version`-only composition call applies this rule: if `dirty_fingerprint` is non-empty, strip a trailing `-dirty` from `commit_id` (if present) and append `-<dirty_fingerprint>` instead; if `dirty_fingerprint` is empty, use `commit_id` completely as-is -- which, for a genuinely dirty tree where only the fingerprint computation failed, still correctly shows `<hash>-dirty` (Requirement 5.2's fallback, satisfied automatically by the pre-existing, unmodified `-dirty` mechanism, with zero new fallback marker/sentinel value needed).
- **Rationale**: Zero risk to the startup log (its composition call path never changes, doesn't even know `dirty_fingerprint` exists) and zero risk to any of the original feature's already-shipped tests that assert `commit_id`'s `-dirty` behavior (none of that behavior changes) -- Requirement 5.2 falls out for free from a mechanism that already exists, rather than needing a new literal sentinel string. Strictly simpler than the rejected alternatives.
- **Trade-offs**: The stripping logic lives in exactly one place (the `--version` composition function), so it must be implemented carefully (string-suffix check, not a blind substring removal, in case a commit hash itself could theoretically end in characters resembling "dirty" -- extremely unlikely with hex hashes, but the implementation should anchor the check to the literal `-dirty` suffix position, not just presence anywhere in the string).
- **Follow-up**: None of the original feature's already-shipped tests need updating, since `commit_id`'s behavior is provably unchanged -- implementation should add a regression assertion confirming this explicitly (e.g., `test-build-version-regression.sh`'s existing `-dirty` assertions continue to pass completely unmodified).
### Decision: Nix path accepts the untracked-file blind spot rather than working around it
- **Context**: Empirically confirmed (see Research Log) that `self.narHash` does not reflect new untracked files unless staged, while the dev-loop path's `git diff` + `git ls-files --others --exclude-standard` approach does.
- **Alternatives Considered**:
1. Have the nix `preConfigure` build-phase hook shell out to `git add -A --dry-run`-style enumeration or otherwise touch git state to force-include untracked files -- rejected: `git` is deliberately not a nix devShell/build dependency in this repo (original feature's explicit architectural decision), and any workaround here would reintroduce exactly that dependency.
2. Accept the asymmetry, document it clearly, and let the nix path's fingerprint simply not change for untracked-only dirty deltas -- selected.
- **Selected Approach**: `flake.nix`'s existing eval-time `let` block (which already computes `myCommitId` via `self.shortRev`/`self.dirtyShortRev`) also computes `myDirtyFingerprint` from `self.narHash` (truncated), passed into `xce.nix`/`xcecli.nix` the same way `myCommitId` already is -- eval-time, via `mesonFlags`, not through the build-phase `preConfigure`/`mesonFlagsArray` mechanism reserved for `build_time`. No git-plumbing is added to the nix path.
- **Rationale**: The motivating incident for the entire `build-time-versioning` feature family was a dev-loop (`xce-build.sh`) build; the nix production path was never implicated and is not this feature's primary target. A silent, undocumented gap would be a defect; a clearly documented, deliberately scoped one is an acceptable trade-off consistent with the original feature's own precedent (research.md's "two independent computation sites... risk mitigated by identical shape, not identical computation").
- **Trade-offs**: An operator diagnosing two nix-built dirty artifacts that differ *only* by an unstaged untracked file addition will see identical `dirty_fingerprint` values despite the builds genuinely differing -- must be documented in design.md and in the nix path's own code comments, mirroring how the original feature documented its own `builtins.currentTime` pure-eval correction inline.
- **Follow-up**: If this gap ever proves operationally significant (e.g., a real incident traced to it), a future spec could revisit having `xce-build.sh` itself pre-stage untracked files in a way that doesn't require nix to shell out to git -- out of scope here.
## Risks & Mitigations
- The `--version`-only `-dirty`-stripping logic (see Decision above) must anchor its check to the literal trailing `-dirty` suffix, not a blind substring removal, to avoid ever mis-stripping a commit hash that coincidentally contains similar characters -- mitigated by keeping the strip logic in exactly one place (the shared composition function) with a dedicated unit test for the boundary case.
- Two computation sites for the same new value (`xce-build.sh` git-based, nix `self.narHash`-based) could, in principle, produce different fingerprints for what a human would call "the same change" -- mitigated the same way the original feature's `commit_id`/`build_time` already are: identical *shape* (both are short hex digests, both empty on a clean tree), not identical *computation*, and each is only ever compared against a build from the *same* computation site in practice (an operator compares two dev-loop builds against each other, or two nix builds against each other, not across the two).
- The nix path's untracked-file blind spot (documented above) could mislead an operator into thinking two nix-built dirty artifacts are content-identical when they differ only by an added-but-unstaged file -- mitigated by explicit documentation in design.md, a code comment at the `myDirtyFingerprint` eval-time `let`-binding in `flake.nix`, and (per Requirement 5) the general graceful-degradation framing that this indicator is a best-effort distinguishing aid, not a cryptographic guarantee of the full working-tree state.
- Computing `git diff HEAD` plus untracked-file content on every dev-loop build adds a small amount of build-time cost -- mitigated by scope: this repo's working trees are source-code sized (not data-file sized), and the original feature's own `xce-build.sh` already runs `git rev-parse`/`git status --porcelain` on every invocation with no observed performance concern.
## References
- `src/flake.nix:12-32` -- existing `myCommitId` eval-time computation this feature's nix-path decision extends, and the empirical `debugNarHash` probe site (added and reverted during this research).
- `src/flake.nix:12-32` -- existing eval-time `myCommitId` computation (original feature) that this feature's `myDirtyFingerprint` extends with the same mechanism, not the sibling `preConfigure` build-phase `date -u` hook in `src/nixSupport/xce.nix`/`xcecli.nix` (original feature's task 2.2), which stays reserved for `build_time` and is otherwise untouched.
- `src/tools/xce-build.sh:80-119` -- existing `commit_id`/`build_time` computation block (original feature's task 2.1) that this feature's dev-loop fingerprint computation extends.
- `.kiro/specs/build-time-versioning/research.md` -- the original feature's own research log, in particular its "One resolved value pair, fed from two independent computation sites" decision, which this feature's architecture directly continues.

22
.kiro/specs/dirty-content-fingerprint/spec.json Normal file
View file

@ -0,0 +1,22 @@
{
"feature_name": "dirty-content-fingerprint",
"created_at": "2026-08-21T02:44:43Z",
"updated_at": "2026-08-21T03:40:00Z",
"language": "en",
"phase": "tasks-generated",
"approvals": {
"requirements": {
"generated": true,
"approved": true
},
"design": {
"generated": true,
"approved": true
},
"tasks": {
"generated": true,
"approved": true
}
},
"ready_for_implementation": true
}

120
.kiro/specs/dirty-content-fingerprint/tasks.md Normal file
View file

@ -0,0 +1,120 @@
# Implementation Plan
- [ ] 1. Foundation: dirty_fingerprint meson option and C-wide define
- Add a `dirty_fingerprint` meson option (string, default `''`) alongside the existing `commit_id`/`build_time` options in `src/meson.options`
- Extend `src/meson.build`'s existing `add_project_arguments()` call to also define `VERSION_DIRTY_FINGERPRINT` for every C component, mirroring `VERSION_COMMIT_ID`/`VERSION_BUILD_TIME`
- Observable: a plain `meson setup` with the option unset still configures successfully (the define resolves to empty string, no build failure)
- _Requirements: 1.1, 5.1_
- [ ] 2. Foundation: compute real fingerprint values at both build entry points
- [ ] 2.1 (P) xce-build.sh computes the dev-loop fingerprint, independently of commit_id
- `commit_id`'s existing computation (including its `-dirty` suffix) is **not touched** -- this task only adds a new, independent computation alongside it
- Compute `dirty_fingerprint` as: empty when the tree is clean, or when the tree is dirty but the hash computation itself fails; a short hex digest (sha256, truncated) over the concatenation of `git diff HEAD`'s output (tracked modifications, staged or unstaged) and the raw content of every file listed by `git ls-files --others --exclude-standard` (untracked, non-ignored files, in a stable sorted order) when the tree is dirty and the hash computation succeeds; zero working-tree mutation in all cases
- Append `-Ddirty_fingerprint="$fp"` to `MESON_OPTS` for every build profile, alongside the existing `-Dcommit_id=`/`-Dbuild_time=` entries
- Observable: editing a tracked file, running `/xce-build scheduler debug`, and checking `--version` shows a fingerprint suffix; editing again with different content and rebuilding shows a different fingerprint; reverting the file to its first edit's exact content and rebuilding reproduces the first fingerprint exactly; creating a new file matched by `.gitignore` and rebuilding leaves the fingerprint unchanged from the pre-creation value (Requirement 1.4's exclusion, exercised live, not just structurally implied by `git ls-files --others --exclude-standard`); the original feature's own `test-build-version-regression.sh` (which asserts a literal `-dirty` substring in `commit_id`'s output) continues to pass completely unmodified, confirming `commit_id` was never touched
- _Requirements: 1.1, 1.2, 1.3, 1.4, 5.1_
- _Boundary: xce-build.sh_
- [ ] 2.2 (P) Nix flake computes the fingerprint via self.narHash at eval time, independently of myCommitId
- `myCommitId`'s existing computation (including its `self.dirtyShortRev`-derived `-dirty` suffix) is **not touched** -- this task only adds a new, independent value alongside it
- `src/flake.nix`'s existing eval-time `let` block gains `myDirtyFingerprint`, derived from `self.narHash` (truncated to match the dev-loop path's format), computed the same way `myCommitId` already is -- **not** via a `preConfigure` build-phase hook, since `self.narHash` (unlike `build_time`'s `builtins.currentTime`) is a legitimate flake self-introspection field with no pure-eval trap; resolves to empty on a clean tree, or a real hash otherwise
- `src/nixSupport/default.nix` forwards `myDirtyFingerprint` into each `callPackage` call, alongside the existing `myCommitId`
- `src/nixSupport/xce.nix`, `src/nixSupport/xcecli.nix` accept the new argument and append `"-Ddirty_fingerprint=" + myDirtyFingerprint` to the existing eval-time `mesonFlags` list
- Observable: `nix build .#xcecore_debug` (no `--impure`, the documented canonical command) still succeeds; editing a tracked file in the flake's source and re-evaluating changes the resulting binary's `--version` fingerprint
- _Requirements: 1.1, 1.2, 1.3, 5.1_
- _Boundary: Nix packaging_
- [ ] 3. Core: C daemon --version fingerprint display
- [ ] 3.1 Extend the shared version-composition mechanism with an optional fingerprint field, without touching commit_id
- `commit_id`'s own computation and its existing `-dirty` suffix are **not modified** -- this task only changes how the `--version` composition function *displays* it
- `xce_compose_version()` (or a sibling function reusing its internals) gains an optional `dirty_fingerprint` parameter, consumed only by the `--version` call path: when non-empty, the composition strips a trailing `-dirty` from `commit_id` (if present) and appends `-<dirty_fingerprint>` instead, producing `<commit_id>-<hash>`; when empty, the composition uses `commit_id` completely as-is (already correctly showing the bare hash on a clean tree, or `<hash>-dirty` on a dirty tree where the fingerprint computation failed -- Requirement 5.2, satisfied for free)
- The existing signature/call sites used by startup-log composition are left completely unaffected -- no new parameter reaches them, and `commit_id` is passed to them exactly as before
- Observable: new unit tests in `test_utils.c` confirm all three cases (empty field + clean `commit_id` -> `<commit_id>` alone; empty field + `-dirty`-suffixed `commit_id` -> `<commit_id>-dirty` unchanged; non-empty field -> `-dirty` stripped and replaced with `-<hash>`) and confirm the startup-log-facing composition path's existing tests still pass completely unmodified
- _Requirements: 1.1, 1.2, 1.3, 1.4, 2.1, 3.1, 3.2, 5.2_
- _Boundary: utils.h, utils.c_
- [ ] 3.2 (P) Controller and scheduler: fingerprint in --version, startup log unaffected
- `argp_program_version`'s composition call (controller/main.c, scheduler/main.c, controller/testsuite/test.c) passes the new `dirty_fingerprint` value; the startup INFO log line's own composition call is left unchanged, with a code comment marking the divergence as deliberate
- Observable confirmed live: on a dirty tree, `xce_controller --version`/`xce_test --version`/`xce_scheduler --version` show the fingerprint suffix; the startup log line for both daemons is byte-identical to the original `build-time-versioning` feature's already-shipped format (with or without the tree being dirty)
- _Requirements: 1.1, 1.2, 1.3, 1.4, 2.1, 3.1, 3.2, 4.1_
- _Boundary: controller/main.c, scheduler/main.c, controller/testsuite/test.c_
- _Depends: 3.1_
- [ ] 3.3 (P) Worker: fingerprint in --version, startup log unaffected
- Worker's hand-added `--version` case composes with the new `dirty_fingerprint` value; its startup INFO log line's composition call is left unchanged
- Observable confirmed live: on a dirty tree, `xcecd --version` shows the fingerprint suffix; both startup log branches remain byte-identical to the original feature's already-shipped format
- _Requirements: 1.1, 1.2, 1.3, 1.4, 2.1, 3.1, 3.2, 4.1_
- _Boundary: worker/main.c_
- _Depends: 3.1_
- [ ] 4. Core: Go binary --version/-version fingerprint display
- [ ] 4.1 go_build injects the fingerprint via ldflags
- `src/tools/go_build` reads `XCE_DIRTY_FINGERPRINT` (empty-safe) and appends `-X main.DirtyFingerprint=...` to its existing `-ldflags` construction, falling back to empty when unset
- Observable: invoking `go_build` with the env var unset still produces a working binary whose injected `DirtyFingerprint` var reads empty (not a fabricated value)
- _Requirements: 1.1, 5.1_
- _Boundary: go_build script_
- [ ] 4.2 (P) apiserver and xcecli meson builds pass the resolved value through
- Both `apiserver/meson.build` and `xcecli/meson.build` pass `XCE_DIRTY_FINGERPRINT` (from `get_option('dirty_fingerprint')`) into `go_build`'s env, matching the existing `XCE_GIT_COMMIT`/`XCE_BUILD_TIME` pattern exactly (same quoting behavior)
- Observable: a real commit-state fingerprint reaches `go build`'s actual invocation and is embedded in both binaries' Go build-info metadata (`go version -m`), once tasks 4.4/4.5 declare the consuming var
- _Requirements: 1.1, 4.1_
- _Boundary: apiserver/meson.build, xcecli/meson.build_
- _Depends: 4.1_
- [ ] 4.3 (P) Makefile dev-loop path computes the same fingerprint, independently of GIT_COMMIT
- `src/frontend/Makefile`'s existing `GIT_COMMIT`/`GIT_STATUS` computation (original feature, task 4.3, including its own `-dirty` suffix) is **not touched**
- Compute a `DIRTY_FINGERPRINT` value directly (git diff + untracked files, dev-loop path, no meson involved) using the identical logic as task 2.1 (empty when clean or on computation failure, real hash otherwise), and pass it via the identical `-ldflags -X main.DirtyFingerprint=` shape
- Observable: `go version -m` on a Makefile-built binary shows the same fingerprint format/value as a meson-built binary from the identical tree state; `GIT_COMMIT`'s own `-dirty` behavior is unaffected
- _Requirements: 1.1, 1.2, 1.3, 1.4, 5.1_
- _Boundary: Makefile_
- _Depends: 4.1_
- [ ] 4.4 (P) apiserver: fingerprint in -version, startup log unaffected
- `DirtyFingerprint` package var (default `""`); `-version` flag's composition applies the same strip-and-append rule as the C daemons: if `DirtyFingerprint` is non-empty, strip a trailing `-dirty` from `GitCommit` and append `-DirtyFingerprint`; if empty, print `GitCommit` as-is
- `GitCommit`'s own computation and the existing `logger.Info("********** Initialize Logger done!")` structured-field call are left completely unchanged -- no new zap field, `GitCommit` logged exactly as before, byte-identical to the original feature's shipped format
- Observable confirmed live: on a dirty build, `-version` shows `<hash>-<fingerprint>` (not `<hash>-dirty-<fingerprint>`); the startup log's first line is unchanged from the original feature's already-shipped format regardless of tree state
- _Requirements: 1.1, 1.2, 1.3, 1.4, 2.1, 3.1, 3.2, 4.1_
- _Boundary: apiserver/cmd/main.go_
- _Depends: 4.1_
- [ ] 4.5 (P) xcecli: fingerprint in --version
- `DirtyFingerprint` var in `main.go`; the version string passed into `router.Execute(cm, version)` (which sets `rootCmd.Version`) applies the same strip-and-append rule
- Observable confirmed live: on a dirty build, `xcecli --version` shows `<hash>-<fingerprint>` and exits 0; a clean build shows the commit id alone, unchanged from the original feature; a dirty build where fingerprint computation failed shows `<hash>-dirty`, unchanged from the original feature
- _Requirements: 1.1, 1.2, 1.3, 1.4, 2.1, 4.1_
- _Boundary: xcecli/cmd/main.go_
- _Depends: 4.1_
- [ ] 5. Core: xceadmin fingerprint display
- [ ] 5.1 (P) meson-build.sh generates the fingerprint field in the build-info module
- `meson-build.sh`'s already-existing `_build_info.py` generation step (from the original feature) gains a third field, `DIRTY_FINGERPRINT` (empty-string fallback, independent of the other two fields' fallback), sourced from one new positional arg/env var wired through `xceadmin/meson.build`
- Observable confirmed live (full real build+test+wheel cycle via `/xce-build xceadmin`): on a dirty tree, the mirrored tree's `_build_info.py` and the packaged wheel both contain a real, non-empty `DIRTY_FINGERPRINT`; on a clean tree, it resolves to empty
- _Requirements: 1.1, 1.2, 1.3, 1.4, 5.1_
- _Boundary: xceadmin/meson.build, xceadmin/meson-build.sh_
- _Depends: 1_
- [ ] 5.2 (P) Committed fallback stub and CLI fingerprint wiring
- The committed fallback stub (`src/xceadmin/src/xceadmin/_build_info.py`) gains `DIRTY_FINGERPRINT = ""` alongside its existing two `"unknown"` fields
- `cli.py`'s `_compose_version()` applies the same strip-and-append rule as the other components: if `DIRTY_FINGERPRINT` is non-empty, strip a trailing `-dirty` from `GIT_COMMIT` and append `-DIRTY_FINGERPRINT`; if empty, use `GIT_COMMIT` as-is
- Observable confirmed live (both install paths): an editable install (`pip install -e .`) shows no fingerprint suffix (empty field, `GIT_COMMIT` unaffected, no import error); a wheel built from a dirty tree shows `<hash>-<fingerprint>`, not `<hash>-dirty-<fingerprint>`
- _Requirements: 1.1, 1.2, 1.3, 1.4, 2.1, 4.1_
- _Boundary: xceadmin/src/xceadmin_
- _Depends: 5.1_
- [ ] 6. Integration: cross-toolchain consistency and regression verification
- [ ] 6.1 Dev-loop distinguishability and clean-tree regression test
- New standalone script (following the `build-time-versioning` feature's `src/tools/test-*.sh` convention) drives a real `xce-build.sh scheduler` build cycle: build on a clean tree and confirm `--version` shows the commit id alone (no `-dirty` suffix, no fingerprint suffix); edit a tracked file, rebuild, confirm a fingerprint suffix appears and `-dirty` does not; edit again with different content, rebuild, confirm the fingerprint differs from the previous one; revert to the first edit's exact content, rebuild, confirm the fingerprint matches the first capture exactly
- Observable: the script exits non-zero and reports clearly if any of the four expected states (clean/no-suffix-at-all, dirty-A, dirty-B different from A, reverted-to-A matching A) does not hold, or if a literal `-dirty` substring ever reappears anywhere in `--version` output
- _Requirements: 1.1, 1.2, 1.3, 1.4, 2.1_
- _Depends: 2.1, 3.2_
- [ ] 6.2 Startup-log-unchanged consistency test
- For at least one C daemon (scheduler) and apiserver, on a dirty tree, capture the startup log's version line and separately run `--version`/`-version`; assert the startup log line is byte-identical to the original `build-time-versioning` feature's already-shipped format (no fingerprint present), while `--version`/`-version` does show the fingerprint
- Observable: a passing test proves the fingerprint never leaks into the startup log for either artifact, even while the tree is dirty
- _Requirements: 3.1, 3.2_
- _Depends: 3.2, 4.4_
- [ ] 6.3 Nix production path end-to-end fingerprint verification
- Confirm `nix build .#xcecore_debug` (no `--impure`) succeeds and produces a binary; edit a tracked file inside the flake's source, re-run the build, and confirm the resulting binary's `--version` fingerprint changed from the prior build's
- Observable: the fingerprint values from two builds of the same commit with different tracked-file edits are different, and no `--impure` flag or `git` invocation inside the sandbox was required
- _Requirements: 1.1, 1.2, 1.3, 5.1_
- _Depends: 2.2_