myxce/.claude/skills/buglog/bug-record.md

# Bug Record Standard

Each bug is one file, `.wolf/bugs/<id>.md`. `scripts/buglog.py` creates
records, changes their status and checks them against this standard; the
pre-commit hook refuses a commit whose staged records do not meet it. See
[Storage and enforcement](#storage-and-enforcement).

Every bug record is read by two audiences: a person deciding what to work on,
and an agent that has to reproduce, fix or verify the bug with no other
context. So a record uses **few fields**, and each field is plain Markdown
text that must satisfy the rules in its section below. The rules say what the
text must contain, how to collect it, and what does not count.

Write every field so that someone who has never seen the bug, on a different
day and a different machine, could act on it.

## Fields at a glance

| # | Field | When required | One-line rule |
|---|-------|---------------|---------------|
| 1 | **Summary** | always | The `# <id>: <summary>` title: one line, ≤ 100 characters, symptom + component. |
| 2 | **Description** | always | What happened vs. what should happen, with the exact error text and the impact. |
| 3 | **Environment** | always | OS, libc, toolchain, build profile, deployment, and the exact code version. |
| 4 | **Reproduction** | always | Steps someone else can run, how often it fails, and where the evidence is kept. |
| 5 | **Root cause** | status `fixed` or `partial` | The mechanism, in code terms, that produces the symptom. |
| 6 | **Fix** | status `fixed` or `partial` | What was changed and why this change removes the cause. |
| 7 | **Verification** | status `fixed` | Proof the fix works, and that the same test failed before the fix. |
| 8 | **References** | status `fixed` | Fix commits, related bugs, specs or docs. |

Machine fields stay short and live in the front matter, outside these
sections: `id`, `status`, `status_updated`, `status_note`, `component`,
`severity`, `timestamp`, `fix_commits`, and a few more. Their values are defined in `.claude/rules/buglog-status.md`
and in [Machine fields](#machine-fields).

---

## 1. Summary

**Must include**
- The observable symptom, not the suspected cause.
- The component or binary where it shows (`xce_controller`, `xcecd_virtual`,
  `xce_scheduler --sim`, `gitea-sync`, …).

**Rules**
- Written as the record's title, `# <id>: <summary>`; the summary itself is
  one line, ≤ 100 characters, no trailing period. It becomes the Gitea issue title.
- No status words (`OPEN`, `FIXED`, `NOT FIXED`, `PRIORITY: LOW`). Status lives in `status`.

**Not acceptable**: `Missing guard clause` · `Incorrect value in code` ·
`crash` · `bug in ha.c`.

**Good**: `xcecd_virtual leaks worker strings on every finished job (LeakSanitizer at exit)`

## 2. Description

**Must include**
- **Observed**: what happened, with the exact error text, log line, assertion
  or sanitizer headline in a code block, copied, not paraphrased.
- **Expected**: what should have happened instead.
- **Impact**: who or what is affected and how badly: crash, hang, data loss,
  wrong result, leak (and its growth rate), noise only.
- **How it was found**: user report, HA soak, ASAN run, audit, code review,
  unit test, or a spec task (name it).

**Include when relevant**
- The first bad state you can see, when the visible symptom comes much later.
- Which other bugs showed up in the same run.

**Rules**
- Keep it to the facts known at the time you log the bug. Later findings go
  in Root cause, or in a dated `UPDATE (YYYY-MM-DD):` line at the end of this
  section. Never rewrite history silently.
- No guesses stated as facts. Label a suspicion as `Suspected:`.

## 3. Environment

The goal: another person can tell whether their machine and build are
affected, and can recreate the same conditions.

**Must include**

- **OS**: distribution and version, and the kernel.
  - Kernel: `uname -srm`.
  - Distribution: `/etc/os-release` `PRETTY_NAME`. If that file is missing,
    say so and give what identifies the system instead. For example, this dev
    host has no `/etc/os-release`. It is a Docker container (`/.dockerenv`
    exists) on a NixOS kernel (`/proc/version` shows `#1-NixOS`).
  - Container or VM, when it applies.
- **libc**: name and version of the C library **the failing binary actually
  loads**, not the host's.
  - Get it from the binary:
    `readelf -l <binary> | grep 'program interpreter'` → `/nix/store/…-glibc-2.42-47/lib/ld-linux-x86-64.so.2`.
  - Why it matters: on this host `ldd --version` outside the Nix shell reports
    glibc **2.40**, but every XCE binary built in the dev shell runs on glibc
    **2.42**. The host value is the wrong answer.
- **Toolchain**: compiler and version (`gcc --version`) plus Meson, and Go or
  Python for those components, taken from the same shell that built the binary.
- **Build**: profile and directory: `asan` (`xce.build`), `debug`
  (`build.dbg`) or `release` (`build.release`); and the sanitizer options in
  force when the failure was seen (e.g. `ASAN_OPTIONS=detect_leaks=0`).
- **Code version**: the commit the failing binary was built from, and the
  branch (`git rev-parse --short HEAD` at build time; add `-dirty` if the tree
  had uncommitted changes). For a bug seen on a deployed cluster, the version
  of **each** component involved (controller, scheduler, worker, apiserver,
  xceadmin) when they differ.
- **Deployment**: topology (single controller, 3-node HA, multihost), worker
  kind (`xcecd_virtual` or real root `xcecd`), and host names when more than
  one machine is involved.

**Include when relevant**
- Config set (`src/configs` or a fixture directory) and any non-default values.
- CPU count and memory, for races, timeouts and load-dependent bugs.
- systemd and cgroup version, for worker process or accounting bugs.
- Anything else that ran on the machine at the same time (parallel builds,
  other soaks), for timing-sensitive failures.

**Not acceptable**: `Linux` · `local` · `latest code` · `ASAN build` with no
commit · the host's `ldd --version` for a Nix-built binary.

**Collect it with** (Nix-built binaries):

```bash
uname -srm; cat /etc/os-release 2>/dev/null | grep PRETTY_NAME || echo "no /etc/os-release"
[ -f /.dockerenv ] && echo "docker container"
readelf -l <binary> | grep -o '/nix/store/[^]]*ld-linux[^]]*'   # libc actually loaded
nix develop ./src --command bash -c 'gcc --version | head -1; meson --version'
git rev-parse --short HEAD; git diff --quiet || echo dirty
```

## 4. Reproduction

**Must include**
- **Steps**: numbered commands someone else can paste, starting from a clean
  checkout at the commit in Environment. Name the test binary, soak mode,
  replay file or script and its arguments.
- **Frequency**: `deterministic`, or measured as `N/M runs` with the date
  (e.g. `2/4 force-snapshot runs, 2026-09-26`). Never `sometimes`.
- **Evidence**: where the logs, cores, replay files or captures are kept. Use
  a path that survives the session: copy them out of `/tmp` and out of
  per-session scratch directories first. If they could not be kept, say so.

**Include when relevant**
- A smaller reproduction (unit test, single command) once you have one. Keep
  the original steps too.
- Conditions that stop it reproducing (e.g. passes with `--jobs 1`).

**When it does not reproduce**: record what was tried, how many runs, on
which commit and date; this is what `status cannot-reproduce --repro` stores.

## 5. Root cause *(fixed or partial)*

**Must include**
- The mechanism in code terms: the function(s) and file(s), the state or
  ordering that goes wrong, and why that produces the symptom in Description.
- Since when: the commit or change that introduced it when known
  (`git log -S`, `git bisect`), otherwise `pre-existing, origin unknown`.
- Scope: other call sites or components with the same defect, checked or
  not checked.

**Rules**
- Describe the cause, not the fix. `Added a NULL check` is a fix, not a cause.
- If part of the explanation is still unproven, mark that part `Unconfirmed:`.

## 6. Fix *(fixed or partial)*

**Must include**
- What changed, per file or function, in a few lines.
- Why this change removes the cause in Root cause.
- For `partial`: what is left and where it is tracked.

**Include when relevant**
- Alternatives that were rejected, and why.
- Behaviour changes, compatibility or migration notes (wire format, WAL,
  snapshot, config).

**Rules**
- A proposal that is not implemented yet is labelled `Proposed:` and needs
  status `open` or `investigating`, never `fixed`.

## 7. Verification *(fixed)*

**Must include**
- **Test added or changed**: the test name and file, and a statement that it
  **failed before the fix** (RED) and passes after (GREEN). If no automated
  test is possible, say why and describe the manual check.
- **Suites run**: which suites, on which build profile, with results
  (e.g. `worker suite 12/12, asan`).
- **Live check** for HA, persistence, scheduler or worker-process bugs: the
  soak or deploy run used, with its mode, rounds and result.

**Not acceptable**: `tests pass` · `verified` · `should be fixed now`.

## 8. References *(fixed)*

**Must include**
- Fix commit(s), short SHA, on a pushed branch. They live in the front-matter
  field `fix_commits`, set by `buglog.py status <id> fixed --commit <sha>`,
  which refuses a hash that is not a commit in this repository.

**Include when relevant**
- Related bugs, with the relationship: `regression of`, `split from`,
  `split into`, `same root cause as`, `duplicate of`.
- Spec, steering or design documents that changed or explain the design.
- The Gitea issue number (filled in by `gitea-sync`).

---

## Machine fields

Front matter: one `key: <JSON value>` per line (valid YAML), between `---` lines.

| Field | Values |
|-------|--------|
| `id` | `bug-<n>`; must match the file name. `buglog.py new` assigns it. |
| `status` | see `.claude/rules/buglog-status.md`; change it with `buglog.py status` |
| `status_updated`, `status_note` | set by `buglog.py status`; a note is required for every status except `fixed` |
| `component` | `controller` · `scheduler` · `worker` · `lib` · `harness` · `apiserver` · `xcecli` · `xceadmin` · `autoscaler` · `build` · `config` · `tooling` |
| `severity` | `data-loss` · `crash` · `hang` · `wrong-result` · `leak` · `security` · `noise` |
| `timestamp` | when the bug was first seen, ISO-8601 UTC, real time (no placeholder `T00:00:00Z`) |
| `last_seen`, `occurrences` | updated by `buglog.py recur` |
| `files`, `tags`, `related` | JSON lists; old status words are not tags (`fixed`, `partially-fixed`, `closed-by-…`) |
| `fix_commits` | short SHAs, required for `fixed` |
| `duplicate_of`, `repro_attempts` | required for `duplicate` / `cannot-reproduce` |
| `legacy` | `true` on records migrated from the old JSON log |
| `gitea_issue`, `gitea_synced_status`, `gitea_synced_hash` | written by `gitea-sync.py` only |

A value of `TODO…` counts as empty: `buglog.py new` writes the template with
`TODO` markers, and `check` lists each one still unfilled.

## Template

`buglog.py new` writes this, with Environment already filled in:

````markdown
---
id: "bug-1872"
status: "open"
status_updated: "2026-09-27"
status_note: "newly logged"
component: "scheduler"
severity: "hang"
timestamp: "2026-09-27T20:05:00Z"
files: []
tags: []
related: []
---

# bug-1872: <symptom in component, ≤ 100 chars>

## Description
**Observed:**
```
<exact error / log / sanitizer text>
```
**Expected:** <…>
**Impact:** <…>
**Found by:** <user report | soak | ASAN | audit | review | test | spec task …>

## Environment
- OS: <distribution + version | "no /etc/os-release" + what it is>; kernel <uname -srm>; <container/VM>
- libc: <name version, from the binary's interpreter>
- Toolchain: gcc <…>, meson <…>
- Build: <asan|debug|release> (<dir>), <sanitizer options>
- Code: <branch> @ <short sha>[-dirty]
- Deployment: <topology>, <worker kind>, <hosts>

## Reproduction
1. <command>
2. <command>
- Frequency: <deterministic | N/M runs, date>
- Evidence: <durable path | "not kept">

## Root cause
<mechanism; since when; scope>

## Fix
<what changed and why it removes the cause>

## Verification
- Test: <name> in <file>: RED before, GREEN after
- Suites: <suite results, profile>
- Live: <soak mode, rounds, result | n/a because …>

## References
- Related: <bug-id (relationship)>, …
- Docs: <path>, …
````

## Example

`.wolf/bugs/bug-1866.md` is a complete record written to this standard
(a worker leak: exact sanitizer text, Environment from `buglog.py env`, a
reproduction on the discovery commit, RED/GREEN verification, fix commit).

## Storage and enforcement

- **Records**: `.wolf/bugs/<id>.md`, one per bug. People and agents read and
  edit them directly; keep the `## <Section>` headings.
- **Index**: `.wolf/buglog.json` is generated from the records (for OpenWolf's
  hooks, `openwolf bug search` and `jq`). Never edit it; `buglog.py` rewrites
  it on every write, and `buglog.py index` after a hand edit.
- **Archive**: `.wolf/buglog-archive.json` keeps the 710 retired OpenWolf hook
  auto-detections (template messages such as "Missing guard clause", ids
  between bug-002 and bug-1672) verbatim. They are not bug reports and are
  not in the index.
- **Migrated records** (`legacy: true`, 2026-09-27) show
  `_Not recorded (migrated from buglog.json)._` where the old log had no such
  field. They are exempt from the reporting sections; changing one to
  `fixed` or `partial` still needs the fix-time sections.
- **Enforcement**:
  - `buglog.py status` refuses a status whose requirements the record does
    not meet, and writes nothing.
  - `buglog.py check <id>|--all` lists every unmet rule.
  - The pre-commit hook (`scripts/git-hooks/pre-commit`, installed per clone
    with `ln -sf ../../scripts/git-hooks/pre-commit .git/hooks/pre-commit`)
    runs `check --staged`: it refuses staged records that miss the standard,
    a legacy record hand-changed to fixed/partial without the fix-time
    sections, and a stale index.