myxce/.kiro/specs/dirty-content-fingerprint/tasks.md

# Implementation Plan

- [x] 1. Foundation: dirty_fingerprint meson option and C-wide define
  - `dirty_fingerprint` meson option (string, default `''`) added in `src/meson.options`, byte-for-byte matching `commit_id`/`build_time`'s style/type/yield; `add_project_arguments()` extended with `-DVERSION_DIRTY_FINGERPRINT=`
  - Observable confirmed live (independently re-verified by review, fresh scratch build): a bare `meson setup` with zero `-D` options configures cleanly (73 targets); all 542 C compile commands contain the new define (empty string), matching the existing two defines' counts exactly; no consumption code touched yet
  - _Requirements: 1.1, 5.1_

- [x] 2. Foundation: compute real fingerprint values at both build entry points
- [x] 2.1 (P) xce-build.sh computes the dev-loop fingerprint, independently of commit_id
  - `dirty_fingerprint` computed via `git diff HEAD` + `git ls-files -z --others --exclude-standard | LC_ALL=C sort -z` content, `sha256sum` truncated to 8 hex chars; empty on a clean tree or on computation failure (3 independently-verified graceful-degradation paths: mktemp failure, sha256sum failure, git diff failure -- none abort the script despite `set -euo pipefail`); zero working-tree/index mutation (read-only git invocations only); `commit_id`'s own computation left byte-for-byte untouched
  - Observable confirmed live (independently re-verified by review, real builds with HEAD pinned to control for concurrent background commits): editing a tracked file and rebuilding produces a fingerprint; different content produces a different fingerprint; reverting to the first edit's exact bytes reproduces the first fingerprint exactly; a `.gitignore`-matched `*.bak` file leaves the fingerprint unchanged; `commit_id`'s `-dirty` suffix stayed intact across every build; every profile (asan/debug/release) carries the option; the original feature's unmodified `test-build-version-regression.sh` passes twice in full (asan+debug)
  - _Requirements: 1.1, 1.2, 1.3, 1.4, 5.1_
  - _Boundary: xce-build.sh_

- [x] 2.2 (P) Nix flake computes the fingerprint via self.narHash at eval time, independently of myCommitId
  - `myDirtyFingerprint` computed as `builtins.hashString "sha256" self.narHash` truncated to 8 hex chars (`self.narHash` is Nix's SRI base64 format, not hex -- rehashing was confirmed necessary, not optional), gated on the same `hasAttr "shortRev"`-style clean/dirty branch `myCommitId` already uses; threaded through `nixSupport/default.nix` into `xce.nix`/`xcecli.nix`'s eval-time `mesonFlags` list as `-Ddirty_fingerprint=`, alongside the untouched `-Dcommit_id=`; `build_time`'s separate `preConfigure`/`mesonFlagsArray` mechanism left completely undisturbed
  - Observable confirmed live (independently re-verified by review, including a full re-run of both `nix build ./src#xcecore_debug` and `./src#xcecli`, no `--impure`): `nix eval` resolves a real, non-empty, content-sensitive `-Ddirty_fingerprint=` value matching an independent Python cross-check; editing a tracked file changes the resolved value; `-Dcommit_id=` unaffected throughout; `nix log` confirms both values reach the real `mesonConfigurePhase` invocation. Full artifact completion is blocked in this sandbox by a pre-existing, unrelated network limitation (an autoscaler Go module fetch needing `goproxy.cn`, confirmed recurring identically on two disjoint Go module sets) -- not a regression from this task, and orthogonal to the eval/meson-configuration behavior this task's Observable actually requires
  - _Requirements: 1.1, 1.2, 1.3, 5.1_
  - _Boundary: Nix packaging_

- [x] 3. Core: C daemon --version fingerprint display
- [x] 3.1 Extend the shared version-composition mechanism with an optional fingerprint field, without touching commit_id
  - New sibling function `xce_compose_version_ex()` added alongside the completely untouched `xce_compose_version()` (zero diff lines in its body or its 3 pre-existing tests); anchors the `-dirty` strip via a true tail-slice `strcmp` (not `strstr`), so it cannot false-match a `-dirty` substring elsewhere; correctly appends `-<fingerprint>` with exactly one dash even when the commit field has no `-dirty` to strip; delegates via a genuine `return xce_compose_version(...)` call (not a re-implementation) when the fingerprint is empty/NULL, structurally guaranteeing byte-identical output, proven by a dedicated equivalence regression test
  - No `main.c` call sites were touched by this task (grep-confirmed all still call the original, untouched `xce_compose_version()`) -- that's tasks 3.2/3.3's job
  - Observable confirmed live (independently re-verified by review, full lib suite on both debug and ASAN profiles): 301 tests, 0 failures, 0 ignored on both profiles; all 3 required cases present with byte-exact assertions, plus 2 meaningful additional regression/defensive tests; buffer handling confirmed snprintf-bounded with no overflow risk
  - _Requirements: 1.1, 1.2, 1.3, 1.4, 2.1, 3.1, 3.2, 5.2_
  - _Boundary: utils.h, utils.c_

- [x] 3.2 (P) Controller and scheduler: fingerprint in --version, startup log unaffected
  - `argp_program_version` for controller/scheduler now composed via a genuinely separate buffer/call to `xce_compose_version_ex()` (reading `STR(VERSION_DIRTY_FINGERPRINT)`, matching the existing commit_id/build_time macro pattern); the startup-log call site's `xce_compose_version()` call, arguments, and buffer are byte-for-byte unchanged (independently verified by review via diff + grep); `controller/testsuite/test.c` (no startup log, confirmed via grep) got a direct in-place swap; all three files carry explicit code comments marking the divergence as deliberate
  - Observable confirmed live (independently re-verified by review, real builds + live daemon starts): `xce_controller --version`/`xce_test --version`/`xce_scheduler --version` all show `<commit>-<fingerprint>`, cross-checked against `compile_commands.json`; live controller and scheduler startup logs show the old `<commit>-dirty` format unchanged, fingerprint never present, build_time matching `--version` exactly
  - _Requirements: 1.1, 1.2, 1.3, 1.4, 2.1, 3.1, 3.2, 4.1_
  - _Boundary: controller/main.c, scheduler/main.c, controller/testsuite/test.c_
  - _Depends: 3.1_

- [x] 3.3 (P) Worker: fingerprint in --version, startup log unaffected
  - `OPT_VERSION`'s case now composes via a genuinely separate `version_display_buf`/`xce_compose_version_ex()` call, byte-for-byte mirroring task 3.2's already-approved controller/scheduler pattern (independently confirmed by review via direct diff comparison against commit `fd62e26`); both `XCECD_LOG` startup-log call sites (`--peers` and no-`--peers`) left completely untouched, same `xce_compose_version()` call, same `version_buf`
  - Observable confirmed live (independently re-verified by review, real build + live daemon runs in both branches): `xcecd --version` shows `<commit>-<fingerprint>` matching `compile_commands.json` exactly, `-dirty` correctly stripped not doubled; both startup log branches show the old fingerprint-less `<commit>-dirty` format unchanged
  - _Requirements: 1.1, 1.2, 1.3, 1.4, 2.1, 3.1, 3.2, 4.1_
  - _Boundary: worker/main.c_
  - _Depends: 3.1_

- [x] 4. Core: Go binary --version/-version fingerprint display
- [x] 4.1 go_build injects the fingerprint via ldflags
  - `go_build` reads `XCE_DIRTY_FINGERPRINT` (empty-safe) and appends `-X main.DirtyFingerprint=...` to the existing `-ldflags` construction; falls back to plain empty string (deliberately NOT `"unknown"`, unlike `GitCommit`/`BuildTime`), matching `dirty_fingerprint`'s convention everywhere else in this feature; the existing `GitCommit`/`BuildTime` `unknown`-fallback logic is completely untouched
  - Observable confirmed live (independently re-verified by review, real scratch Go builds traced via `bash -x`): fingerprint unset -> `-X main.DirtyFingerprint=` empty, binary confirms empty (not fabricated); fingerprint set -> flag carries the exact value; all three vars unset -> `GitCommit`/`BuildTime` still fall back to `unknown`, `DirtyFingerprint` stays empty, the two fallback conventions coexist without interference
  - _Requirements: 1.1, 5.1_
  - _Boundary: go_build script_

- [x] 4.2 (P) apiserver and xcecli meson builds pass the resolved value through
  - Both `apiserver/meson.build` and `xcecli/meson.build` add `'XCE_DIRTY_FINGERPRINT=' + get_option('dirty_fingerprint')`, unquoted, identically positioned to the existing `XCE_GIT_COMMIT`/`XCE_BUILD_TIME` entries
  - Observable confirmed live (independently re-verified by review with a distinct test value, fresh configure, and real `ninja` builds of both binaries): the value reaches `go build`'s real invocation and shows up in `go version -m`'s recorded `-ldflags` build setting for both apiserver and xcecli; matches the C-side `-DVERSION_DIRTY_FINGERPRINT=` byte-for-byte from the same configure; degrades to empty identically to the two pre-existing vars when omitted
  - _Requirements: 1.1, 4.1_
  - _Boundary: apiserver/meson.build, xcecli/meson.build_
  - _Depends: 4.1_

- [x] 4.3 (P) Makefile dev-loop path computes the same fingerprint, independently of GIT_COMMIT
  - `GIT_COMMIT`/`GIT_STATUS`/`BUILD_TIME` (and `GIT_COMMIT`'s own `-dirty` suffix) confirmed byte-for-byte untouched throughout two rounds of adversarial remediation
  - `DIRTY_FINGERPRINT` computed independently, mirroring task 2.1's algorithm exactly (cross-checked byte-identical on the real dirty tree); failure-detection required two remediation rounds to get right -- round 1 fixed a `{ git diff HEAD; ...; } >file` grouped command silently swallowing a `git diff HEAD` failure (only the last command's exit status was checked); round 2 found the SAME class of bug one step further down (`git ls-files | sort | xargs cat`'s pipe also swallowing an `ls-files` failure) and fixed it by pulling `git ls-files` out of the pipe entirely into its own directly-redirected, individually-checked step -- deliberately choosing this over a `set -o pipefail;` prefix (which empirically also worked) because this repo's own `build-deployment.md` documents a past incident of code silently relying on `/bin/sh` happening to be bash with no `SHELL:=bash` pin
  - One narrow, explicitly-disclosed residual gap remains and was assessed as acceptable: a `sort`-only failure (not `cat`) in the trailing `sort -z | xargs -0 -r cat --` pipe can still be masked -- independently confirmed present, low-risk, and accurately scoped, not grounds for further rejection
  - Observable confirmed live (independently re-verified across 3 review rounds, real builds + two independently-reproduced failure-injection wrappers with control tests proving each wrapper genuinely triggers the old bug): `make -n build-cli`'s fingerprint matches a standalone reference computation byte-for-byte; both the `git diff`-fails and `git ls-files`-fails scenarios correctly degrade to empty, not a fabricated hash; `GIT_COMMIT`'s `-dirty` behavior unaffected throughout
  - _Requirements: 1.1, 1.2, 1.3, 1.4, 5.1_
  - _Boundary: Makefile_
  - _Depends: 4.1_

- [x] 4.4 (P) apiserver: fingerprint in -version, startup log unaffected
  - `DirtyFingerprint` package var (default `""`) + `composeVersionDisplay()` helper doing the strip-and-append via a genuine anchored `strings.TrimSuffix` (not a blind substring replace, independently confirmed via a self-constructed mutation test); `GitCommit`'s declaration and the `logger.Info` startup-log call site (including its zap fields) confirmed byte-for-byte untouched
  - Observable confirmed live (independently re-verified by review, real build across all 5 scenarios + a live daemon-start log capture): `-version` shows `<hash>-<fingerprint>` never `<hash>-dirty-<fingerprint>`; the startup log shows the raw, un-fingerprinted `GitCommit` (still `-dirty`-suffixed when applicable) on the same binary whose `-version` output differs; `go.sum`/`go.mod` untouched
  - _Requirements: 1.1, 1.2, 1.3, 1.4, 2.1, 3.1, 3.2, 4.1_
  - _Boundary: apiserver/cmd/main.go_
  - _Depends: 4.1_

- [x] 4.5 (P) xcecli: fingerprint in --version
  - `DirtyFingerprint` var + `composeVersionDisplay()` helper, a verbatim logical duplicate of apiserver's just-approved implementation (task 4.4); applied at the sole consumption point (the version string fed into `router.Execute(cm, version)`); `router.go` confirmed untouched (independently re-verified by the parent controller after the review, not just the reviewer, given a mid-review incident where the reviewer's own tooling briefly discarded the uncommitted diff before catching and byte-for-byte restoring it)
  - Observable confirmed live (independently re-verified by review, real builds across all 3 scenarios + full xcecli module test suite): `--version`/`-v` shows `<hash>-<fingerprint>` on dirty+fingerprint, `<hash>` alone on clean, `<hash>-dirty` unchanged on dirty+degraded-fingerprint; all exit 0; no regressions in any other xcecli package
  - _Requirements: 1.1, 1.2, 1.3, 1.4, 2.1, 4.1_
  - _Boundary: xcecli/cmd/main.go_
  - _Depends: 4.1_

- [x] 5. Core: xceadmin fingerprint display
- [x] 5.1 (P) meson-build.sh generates the fingerprint field in the build-info module
  - `dirty_fingerprint` threaded through as an 8th positional arg (`$8`), correctly numbered and consistent between `meson.build` and `meson-build.sh`; `_build_info.py` gains `DIRTY_FINGERPRINT` with an independent empty-string fallback (not `"unknown"`); `GIT_COMMIT`/`BUILD_TIME`'s existing generation and fallback confirmed byte-for-byte untouched
  - Observable confirmed live (independently re-verified by review, full real build+test+wheel cycles across 3 scenarios): dirty tree -> real, non-empty `DIRTY_FINGERPRINT` present identically in the mirrored tree, the build/lib copy, an installed venv, and the packaged wheel, cross-checked byte-for-byte against that same build's `compile_commands.json`; explicit-empty fingerprint -> genuinely `""` while `GIT_COMMIT`/`BUILD_TIME` pass through real values; all-empty -> `GIT_COMMIT`/`BUILD_TIME` still fall back to `"unknown"` while `DIRTY_FINGERPRINT` stays `""`; 261 passed/3 skipped in all three scenarios
  - _Requirements: 1.1, 1.2, 1.3, 1.4, 5.1_
  - _Boundary: xceadmin/meson.build, xceadmin/meson-build.sh_
  - _Depends: 1_

- [x] 5.2 (P) Committed fallback stub and CLI fingerprint wiring
  - `_build_info.py`'s fallback stub gains `DIRTY_FINGERPRINT = ""`; `cli.py`'s new `_compose_version_display()` applies the strip-and-append rule via a genuine anchored `removesuffix("-dirty")` (independently confirmed via a second, differently-constructed mutation test); both fallback paths (`_build_info.py`'s module stub and `cli.py`'s separate `ImportError` fallback class) correctly carry the field
  - A pre-existing test's hard-coded literal-`GIT_COMMIT` assertion was updated to compute its expected value via the production `_compose_version_display()` function instead -- independently assessed by review as a legitimate strengthening (the old assertion's premise no longer holds once a fingerprint is present), not a weakened/gutted check; the 4 new dedicated unit tests independently guard the transform itself with hard-coded literal assertions
  - Observable confirmed live (independently re-verified by review, both install paths via a full real `/xce-build xceadmin` cycle): editable install shows `0.1.0 (unknown, unknown)`, no import error; wheel built from the dirty tree shows `<hash>-<fingerprint>`, never `<hash>-dirty-<fingerprint>`; full suite 267 passed/3 skipped (261 baseline + 6 new)
  - _Requirements: 1.1, 1.2, 1.3, 1.4, 2.1, 4.1_
  - _Boundary: xceadmin/src/xceadmin_
  - _Depends: 5.1_

- [x] 6. Integration: cross-toolchain consistency and regression verification
- [x] 6.1 Dev-loop distinguishability and clean-tree regression test
  - New script `src/tools/test-dirty-fingerprint-distinguishability.sh` uses a genuine isolated `git worktree add --detach` (unique `mktemp -u` path, `trap cleanup EXIT` registered before creation, safe no-op if creation never happened, `rm -rf`+`prune` fallback) to sidestep this actively-shared repo's ambient concurrent-editing noise -- all file edits confined strictly inside the isolated worktree, the real repo's own working tree never touched
  - Drives a real `xce-build.sh scheduler` cycle through the 4 required states via genuine builds; fails loudly with a clear message on any mismatch
  - Observable confirmed live (independently re-verified by review across two full live runs, including a deliberately sabotaged copy forcing edit-B to reproduce edit-A's content): clean/no-suffix, dirty-A, dirty-B-differs-from-A, reverted-to-A-matches-A all held with real fingerprints; sabotaged run correctly failed with a clear message and exit 1, proving genuine (not just synthetic) discriminating power; `git worktree list` and the real repo's working tree confirmed byte-identical to baseline before and after every run, both by implementer and reviewer
  - _Requirements: 1.1, 1.2, 1.3, 1.4, 2.1_
  - _Depends: 2.1, 3.2_

- [x] 6.2 Startup-log-unchanged consistency test
  - New script `src/tools/test-dirty-fingerprint-startup-log-unchanged.sh` reuses task 6.1's already-approved isolated-git-worktree technique; builds scheduler and apiserver for real on a genuinely dirty tree, starts both daemons, captures `--version`/`-version` (expect fingerprint) and each startup log (expect old fingerprint-free format)
  - Apiserver's injected fingerprint value is reused from scheduler's own extracted value (same dirty tree, same content) rather than independently recomputed -- independently assessed by review as legitimate, not tautological: apiserver has no independent fingerprint-computation code of its own in production (only ever receives an externally-injected value via `-ldflags`), the script still exercises apiserver's own real, unmocked `composeVersionDisplay()`/`logger.Info` code paths on a genuinely running compiled binary, and a round-trip `api_fp == sched_fp` check would catch a real strip-and-append bug
  - Observable confirmed live (independently re-verified by review, fresh full run + a separately-authored 4-case discrimination probe outside the script's own demo block): both daemons' `--version`/`-version` show real fingerprints while their startup logs show the unchanged, fingerprint-free old format; no leftover processes, worktrees, or build dirs after either run
  - _Requirements: 3.1, 3.2_
  - _Depends: 3.2, 4.4_

- [x] 6.3 Nix production path end-to-end fingerprint verification
  - New script `src/tools/test-dirty-fingerprint-nix-production-path.sh` (reusing task 6.1's isolated-worktree technique) edits `flake.nix`, runs `nix build ./src#xcecore_debug -L --keep-going` (no `--impure`) twice with different content, and cross-checks `nix eval`'s eval-time `myDirtyFingerprint` value against the real build log's `mesonConfigurePhase flags:` line for each
  - This sandbox's `nix build` dispatches by default to an unreachable-in-practice remote builder (`ssh://rshen@172.31.109.18`, confirmed genuinely expected behavior, not a misconfiguration) making a directly-runnable binary impractical to retrieve; the eval+build-log cross-check is not a weaker substitute -- independently confirmed structurally sound by review, since `self.narHash`'s value is computed at flake-evaluation time and baked as a literal string into the derivation before any build phase runs, so eval-time and build-time values cannot diverge by construction, not just by empirical coincidence. Forcing `--builders ''` (local-only) was investigated per a live suggestion and does reach real local `.c` compilation with a direct log match, but still can't produce a fully linked binary in this sandbox (a second, unrelated network dependency), confirming no stronger approach is currently available; recorded as a learning for future sessions
  - Observable confirmed live (independently re-verified by review, real `nix build` invocations including a live-witnessed remote-builder connection attempt and a genuine local-only run): two builds with different tracked-file edits produce different fingerprints in both eval and build-log evidence; reverting to the first edit's exact content reproduces the first fingerprint exactly; `--impure` never used; no `git` dependency in the build sandbox; `flake.nix`/`nixSupport/` and the worktree list confirmed byte-identical to baseline after every run
  - _Requirements: 1.1, 1.2, 1.3, 5.1_
  - _Depends: 2.2_

## Implementation Notes
- **Task 4.3**: a `{ cmd1; cmd2; } >file` or `cmd1 | cmd2 | cmd3` shell construct's exit status (as checked by `if ! ... ; then ...; fi`) only reflects the LAST command's exit status, not any earlier command's -- this silently swallows an earlier failure and can produce fabricated output from partial/incomplete data instead of the intended graceful-degradation path. Took two rejected review rounds to fully generalize (first fixed for `git diff HEAD`, then found again one step later for `git ls-files | sort | xargs`). When writing any multi-stage shell pipeline whose failure must be individually detectable, either enable `pipefail` explicitly (verify the actual invoking shell first -- `$(shell ...)` in Make does NOT inherit a Makefile's `SHELL`/`.SHELLFLAGS` settings) or restructure each stage to its own directly-redirected, individually-`$?`-checked step (the safer default absent a `SHELL:=bash`-style pin, per this repo's own documented `/bin/sh`-is-not-guaranteed-bash incident in `build-deployment.md`).