feat(buglog): one Markdown record per bug, checked against a record standard
The buglog was one 27K-line JSON file whose fields covered only symptom,
root cause and fix, as free text: 2% of entries named the code version a
bug was found on, 12% the build profile, 36% a way to reproduce, and 58% of
the file was OpenWolf hook auto-detections ("Missing guard clause",
"Function not marked async" on .md files).
- .claude/skills/buglog/bug-record.md: the record standard. Eight
sections (Summary as the title, Description, Environment, Reproduction;
Root cause, Fix, Verification, References once fixed), each with the
content it must contain -- e.g. Environment needs the OS, the libc the
failing binary loads (from its ELF interpreter: glibc 2.42 in the nix
shell vs 2.40 on the host), toolchain, build profile and commit.
- .wolf/bugs/<id>.md: 500 records migrated from buglog.json; every field
verified to round-trip. They carry legacy: true and an explicit "not
recorded" where the old log had no such field; component derived from
file paths, severity from keywords (else unknown), fix_commits from
hashes in status_note/fix that git resolves (91 of 463 fixed).
- .wolf/buglog-archive.json: the 710 hook auto-detections, verbatim.
- .wolf/buglog.json is now an index generated from the records, keeping
the fields OpenWolf's hooks and `openwolf bug search` read.
- scripts/buglog.py rewritten: new (template + Environment collected from
the machine and --binary), check, status (refuses a status the record
doesn't support; fixed needs Root cause, Fix, Verification and
--commit hashes git resolves), note, recur, set, show, list, triage,
env, index. 29 unittest cases in scripts/tests/test_buglog.py.
- scripts/git-hooks/pre-commit runs `check --staged`: incomplete staged
records, a legacy record hand-changed to fixed without the fix-time
sections, a new record claiming legacy: true, or a stale index refuse
the commit.
- scripts/gitea-sync.py reads the records: issue body = the record's
sections plus linked fix commits (origin remote, GITEA_COMMIT_REPO to
override), and a body hash so an issue's text is refreshed when the
record changes.
- bug-1866 rewritten to the full standard as the worked example.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
27b7a84980
commit
d26ff8870d
GPG key ID:
80533536B0EB651C
512 changed files with 40724 additions and 17843 deletions
|
|
@ -1,48 +1,82 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Read and edit .wolf/buglog.json without re-serializing it.
|
||||
"""Read, write and check the XCE bug records: one Markdown file per bug.
|
||||
|
||||
Re-serializing the whole buglog (json.dumps / jq) rewrites ~1,200 unrelated
|
||||
lines because existing entries mix raw UTF-8 and \\uXXXX escapes, so every
|
||||
write here edits only the lines of the entry it touches. Every write is
|
||||
re-parsed before it is saved and saved atomically (tmp file + rename), the
|
||||
same way OpenWolf's own hooks write it.
|
||||
Each bug is `.wolf/bugs/<id>.md`: a front-matter block of short machine
|
||||
fields (one `key: <JSON value>` per line, which is also valid YAML), then
|
||||
`# <id>: <summary>` and the sections of the record standard
|
||||
(.claude/skills/buglog/bug-record.md). `check` enforces that standard.
|
||||
|
||||
The `status` values and when to use each are defined in
|
||||
.claude/rules/buglog-status.md.
|
||||
`.wolf/buglog.json` is an index generated from the records for OpenWolf's
|
||||
hooks and `jq` queries; every write here regenerates it, never edit it by
|
||||
hand. The retired hook-generated entries live in `.wolf/buglog-archive.json`.
|
||||
|
||||
Status values and when to use each: .claude/rules/buglog-status.md.
|
||||
|
||||
Commands:
|
||||
add --error MSG [--root-cause T] [--fix T] [--file F] [--tags a,b]
|
||||
[--related bug-1,bug-2] [--status S] [--note T]
|
||||
Append a new entry with the next free id; prints the id.
|
||||
status ID STATUS [--note T] [--duplicate-of ID] [--repro T]
|
||||
Set status + status_updated (today) + status_note. --note is
|
||||
required except for `fixed`; duplicate needs --duplicate-of,
|
||||
cannot-reproduce needs --repro.
|
||||
note ID TEXT Append an "UPDATE (date): TEXT" paragraph to `fix`.
|
||||
recur ID [--note T] The bug was seen again: bump occurrences/last_seen;
|
||||
cannot-reproduce -> investigating, fixed/wontfix -> open.
|
||||
set ID key=<json> ... Set arbitrary single-line fields.
|
||||
show ID Print one entry.
|
||||
new --summary S --component C --severity V [--binary B] [--tags a,b]
|
||||
[--files f,g] [--related bug-1,bug-2]
|
||||
Create a record from the template, Environment filled in.
|
||||
check [ID ...] [--all] [--staged] Check records against the standard.
|
||||
status ID STATUS [--note T] [--commit SHA]... [--duplicate-of ID] [--repro T]
|
||||
Change status; refused if the record does not meet what the
|
||||
new status requires.
|
||||
note ID TEXT [--section S] Append "UPDATE (date): TEXT" to a section.
|
||||
recur ID [--note T] Seen again: bump occurrences/last_seen;
|
||||
cannot-reproduce -> investigating, fixed/wontfix -> open.
|
||||
set ID key=<json> ... Set front-matter fields.
|
||||
show ID Print one record.
|
||||
list [--status S,S | --open] [--tag T] [--grep RE] [--since DATE]
|
||||
One line per matching entry.
|
||||
triage List entries with no (or an unknown) status.
|
||||
next-id Print the id `add` would use.
|
||||
triage Records with no (or an unknown) status.
|
||||
env [BINARY] Print an Environment section for this machine.
|
||||
index Regenerate .wolf/buglog.json.
|
||||
next-id Print the id `new` would use.
|
||||
|
||||
Importable: set_fields(path, bug_id, {key: value}), BuglogEditError.
|
||||
Importable: load, load_all, set_meta, check, build_index, BuglogEditError.
|
||||
"""
|
||||
import argparse
|
||||
import datetime
|
||||
import json
|
||||
import os
|
||||
import platform
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
DEFAULT_PATH = Path(__file__).resolve().parent.parent / ".wolf" / "buglog.json"
|
||||
|
||||
STATUSES = ["open", "partial", "investigating", "fixed",
|
||||
"cannot-reproduce", "wontfix", "duplicate"]
|
||||
OPEN_STATUSES = {"open", "partial", "investigating"}
|
||||
COMPONENTS = ["controller", "scheduler", "worker", "lib", "harness",
|
||||
"apiserver", "xcecli", "xceadmin", "autoscaler", "build",
|
||||
"config", "tooling"]
|
||||
SEVERITIES = ["data-loss", "crash", "hang", "wrong-result", "leak",
|
||||
"security", "noise"]
|
||||
SECTIONS = ["Description", "Environment", "Reproduction", "Root cause",
|
||||
"Fix", "Verification", "References", "Legacy notes"]
|
||||
META_ORDER = ["id", "status", "status_updated", "status_note", "component",
|
||||
"severity", "timestamp", "last_seen", "occurrences", "files",
|
||||
"tags", "related", "fix_commits", "duplicate_of",
|
||||
"repro_attempts", "legacy", "gitea_issue",
|
||||
"gitea_synced_status", "gitea_synced_hash"]
|
||||
ID_RE = re.compile(r"bug-\d+(?:-[a-z0-9-]+)?")
|
||||
# Text a migrated record carries where the old log had no such field.
|
||||
NOT_RECORDED = "_Not recorded (migrated from buglog.json)._"
|
||||
STATUS_WORDS = re.compile(r"\b(OPEN|FIXED|NOT FIXED|PRIORITY)\b")
|
||||
|
||||
ROOT = BUGS_DIR = INDEX_PATH = ARCHIVE_PATH = None
|
||||
|
||||
|
||||
def configure(root):
|
||||
"""Point the module at a repository root (tests use a temp dir)."""
|
||||
global ROOT, BUGS_DIR, INDEX_PATH, ARCHIVE_PATH
|
||||
ROOT = Path(root)
|
||||
BUGS_DIR = ROOT / ".wolf" / "bugs"
|
||||
INDEX_PATH = ROOT / ".wolf" / "buglog.json"
|
||||
ARCHIVE_PATH = ROOT / ".wolf" / "buglog-archive.json"
|
||||
|
||||
|
||||
configure(Path(__file__).resolve().parent.parent)
|
||||
|
||||
|
||||
class BuglogEditError(Exception):
|
||||
|
|
@ -57,178 +91,590 @@ def _now():
|
|||
return datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
|
||||
|
||||
def _load(path):
|
||||
return json.loads(Path(path).read_text(encoding="utf-8"))["bugs"]
|
||||
|
||||
|
||||
def _save(path, text):
|
||||
"""Refuse to write text that no longer parses; write atomically."""
|
||||
json.loads(text)
|
||||
path = Path(path)
|
||||
tmp = path.with_name(f"{path.name}.{os.getpid()}.tmp")
|
||||
tmp.write_text(text, encoding="utf-8")
|
||||
os.replace(tmp, path)
|
||||
|
||||
|
||||
def _entry_span(lines, bug_id):
|
||||
"""(start, end, indent) of the entry: start = its "id" line, end = its '}'."""
|
||||
id_re = re.compile(r'^(\s+)"id": ' + re.escape(json.dumps(bug_id)) + r",?$")
|
||||
starts = [i for i, ln in enumerate(lines) if id_re.match(ln)]
|
||||
if len(starts) != 1:
|
||||
raise BuglogEditError(f"{bug_id}: expected exactly one entry, found {len(starts)}")
|
||||
start = starts[0]
|
||||
indent = id_re.match(lines[start]).group(1)
|
||||
close_re = re.compile(r"^" + re.escape(indent[:-2]) + r"\},?$")
|
||||
end = next(i for i in range(start, len(lines)) if close_re.match(lines[i]))
|
||||
return start, end, indent
|
||||
|
||||
|
||||
def set_fields(path, bug_id, fields):
|
||||
"""Set each key in `fields` on entry `bug_id`, touching only its lines.
|
||||
|
||||
An existing single-line field is replaced in place; a new one is inserted
|
||||
right after the entry's "id" line. Multi-line (array/object) fields are
|
||||
refused -- edit those by hand.
|
||||
"""
|
||||
path = Path(path)
|
||||
lines = path.read_text(encoding="utf-8").split("\n")
|
||||
start, end, indent = _entry_span(lines, bug_id)
|
||||
|
||||
for key, value in fields.items():
|
||||
rendered = json.dumps(value, ensure_ascii=False)
|
||||
key_re = re.compile(r"^" + re.escape(indent) + re.escape(json.dumps(key)) + r": (.*?)(,?)$")
|
||||
hit = next((i for i in range(start, end) if key_re.match(lines[i])), None)
|
||||
if hit is not None:
|
||||
old, comma = key_re.match(lines[hit]).groups()
|
||||
if old in ("[", "{"):
|
||||
raise BuglogEditError(f"{bug_id}.{key}: multi-line value, edit by hand")
|
||||
lines[hit] = f"{indent}{json.dumps(key)}: {rendered}{comma}"
|
||||
else:
|
||||
lines.insert(start + 1, f"{indent}{json.dumps(key)}: {rendered},")
|
||||
end += 1
|
||||
|
||||
text = "\n".join(lines)
|
||||
entry = next(b for b in json.loads(text)["bugs"] if b.get("id") == bug_id)
|
||||
for key, value in fields.items():
|
||||
if entry[key] != value:
|
||||
raise BuglogEditError(f"{bug_id}.{key}: wrote {entry[key]!r}, wanted {value!r}")
|
||||
_save(path, text)
|
||||
|
||||
|
||||
def next_id(bugs):
|
||||
"""One past the highest numeric id. (OpenWolf's own hook uses len(bugs)+1,
|
||||
which collides with existing ids once any were renumbered or skipped.)"""
|
||||
nums = [int(m.group(1)) for b in bugs
|
||||
if (m := re.fullmatch(r"bug-(\d+)", str(b.get("id", ""))))]
|
||||
return f"bug-{max(nums, default=0) + 1:03d}"
|
||||
|
||||
|
||||
def append_entry(path, entry):
|
||||
"""Append `entry` as the last element of the `bugs` array."""
|
||||
path = Path(path)
|
||||
lines = path.read_text(encoding="utf-8").split("\n")
|
||||
close = max(i for i, ln in enumerate(lines) if ln == " ]")
|
||||
prev = close - 1
|
||||
if lines[prev] == " }":
|
||||
lines[prev] = " },"
|
||||
elif lines[prev] != " \"bugs\": [":
|
||||
raise BuglogEditError(f"unexpected line before bugs array end: {lines[prev]!r}")
|
||||
body = json.dumps(entry, indent=2, ensure_ascii=False).split("\n")
|
||||
lines[close:close] = [" " + ln for ln in body]
|
||||
text = "\n".join(lines)
|
||||
if json.loads(text)["bugs"][-1] != entry:
|
||||
raise BuglogEditError("appended entry did not round-trip")
|
||||
_save(path, text)
|
||||
|
||||
|
||||
def _split(value):
|
||||
return [v.strip() for v in (value or "").split(",") if v.strip()]
|
||||
|
||||
|
||||
def _text(value):
|
||||
if isinstance(value, list):
|
||||
return " || ".join(str(v) for v in value)
|
||||
return value or ""
|
||||
def _run(cmd, **kw):
|
||||
try:
|
||||
r = subprocess.run(cmd, capture_output=True, text=True, timeout=180, **kw)
|
||||
except (OSError, subprocess.TimeoutExpired):
|
||||
return None
|
||||
return r.stdout.strip() if r.returncode == 0 else None
|
||||
|
||||
|
||||
def _find(bugs, bug_id):
|
||||
for b in bugs:
|
||||
if b.get("id") == bug_id:
|
||||
return b
|
||||
raise BuglogEditError(f"{bug_id}: no such entry")
|
||||
# ── Record format ────────────────────────────────────────────────────────────
|
||||
|
||||
class Record:
|
||||
def __init__(self, meta, summary, sections, path=None):
|
||||
self.meta = meta
|
||||
self.summary = summary
|
||||
self.sections = sections
|
||||
self.path = path
|
||||
|
||||
@property
|
||||
def id(self):
|
||||
return self.meta.get("id")
|
||||
|
||||
|
||||
def _line(b):
|
||||
note = b.get("status_note") or _text(b.get("error_message"))
|
||||
return f"{b['id']:<10} {b.get('status') or '-':<16} {note[:110]}"
|
||||
def render_meta(meta):
|
||||
keys = [k for k in META_ORDER if k in meta] + \
|
||||
sorted(k for k in meta if k not in META_ORDER)
|
||||
lines = ["---"] + [f"{k}: {json.dumps(meta[k], ensure_ascii=False)}"
|
||||
for k in keys] + ["---"]
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def cmd_add(a, path):
|
||||
def render(rec):
|
||||
out = [render_meta(rec.meta), f"# {rec.id}: {rec.summary}", ""]
|
||||
for name in SECTIONS:
|
||||
if name in rec.sections:
|
||||
out += [f"## {name}", rec.sections[name].strip("\n"), ""]
|
||||
return "\n".join(out)
|
||||
|
||||
|
||||
_HEADER_RE = re.compile(r"^## (" + "|".join(map(re.escape, SECTIONS)) + r")\s*$")
|
||||
_ESCAPED_RE = re.compile(r"^\\(## (" + "|".join(map(re.escape, SECTIONS)) + r")\s*)$",
|
||||
re.M)
|
||||
|
||||
|
||||
def escape_text(text):
|
||||
"""Stop a line of free text from being read as a section header."""
|
||||
return re.sub(_HEADER_RE.pattern, r"\\\g<0>", text, flags=re.M)
|
||||
|
||||
|
||||
def unescape_text(text):
|
||||
return _ESCAPED_RE.sub(r"\1", text)
|
||||
|
||||
|
||||
def parse(text, path=None):
|
||||
lines = text.split("\n")
|
||||
if not lines or lines[0] != "---":
|
||||
raise BuglogEditError(f"{path}: no front matter")
|
||||
meta = {}
|
||||
i = 1
|
||||
while i < len(lines) and lines[i] != "---":
|
||||
key, sep, raw = lines[i].partition(": ")
|
||||
if not sep:
|
||||
raise BuglogEditError(f"{path}:{i + 1}: expected 'key: value'")
|
||||
try:
|
||||
meta[key] = json.loads(raw)
|
||||
except json.JSONDecodeError as e:
|
||||
raise BuglogEditError(f"{path}:{i + 1}: {key}: {e}") from None
|
||||
i += 1
|
||||
if i == len(lines):
|
||||
raise BuglogEditError(f"{path}: front matter not closed")
|
||||
|
||||
summary, sections, current, buf, fence = None, {}, None, [], None
|
||||
for ln in lines[i + 1:]:
|
||||
m = re.match(r"^\s*(`{3,}|~{3,})", ln)
|
||||
if m:
|
||||
mark = m.group(1)
|
||||
if fence is None:
|
||||
fence = mark
|
||||
elif mark[0] == fence[0] and len(mark) >= len(fence):
|
||||
fence = None
|
||||
if fence is None and not m:
|
||||
h = _HEADER_RE.match(ln)
|
||||
if h:
|
||||
if current:
|
||||
sections[current] = "\n".join(buf).strip("\n")
|
||||
current, buf = h.group(1), []
|
||||
continue
|
||||
if summary is None and current is None and ln.startswith("# "):
|
||||
title = ln[2:].strip()
|
||||
prefix = f"{meta.get('id')}: "
|
||||
summary = title[len(prefix):] if title.startswith(prefix) else title
|
||||
continue
|
||||
if current:
|
||||
buf.append(ln)
|
||||
if current:
|
||||
sections[current] = "\n".join(buf).strip("\n")
|
||||
return Record(meta, summary or "", sections, path)
|
||||
|
||||
|
||||
def record_path(bug_id):
|
||||
if not ID_RE.fullmatch(bug_id or ""):
|
||||
raise BuglogEditError(f"{bug_id!r}: not a bug id")
|
||||
return BUGS_DIR / f"{bug_id}.md"
|
||||
|
||||
|
||||
def load(bug_id):
|
||||
path = record_path(bug_id)
|
||||
if not path.exists():
|
||||
raise BuglogEditError(f"{bug_id}: no such record")
|
||||
return parse(path.read_text(encoding="utf-8"), path)
|
||||
|
||||
|
||||
def _id_key(bug_id):
|
||||
m = re.match(r"bug-(\d+)(.*)", bug_id)
|
||||
return (int(m.group(1)), m.group(2)) if m else (0, bug_id)
|
||||
|
||||
|
||||
def load_all():
|
||||
recs = [parse(p.read_text(encoding="utf-8"), p) for p in BUGS_DIR.glob("*.md")]
|
||||
return sorted(recs, key=lambda r: _id_key(r.id or ""))
|
||||
|
||||
|
||||
def _write_atomic(path, text):
|
||||
tmp = path.with_name(f".{path.name}.{os.getpid()}.tmp")
|
||||
tmp.write_text(text, encoding="utf-8")
|
||||
os.replace(tmp, path)
|
||||
|
||||
|
||||
def save(rec, reindex=True):
|
||||
"""Write a record; refuses text that would not parse back identically."""
|
||||
text = render(rec)
|
||||
again = parse(text, rec.path)
|
||||
if again.meta != rec.meta or again.summary != rec.summary or \
|
||||
again.sections != {k: v.strip("\n") for k, v in rec.sections.items()}:
|
||||
raise BuglogEditError(f"{rec.id}: record would not round-trip")
|
||||
path = record_path(rec.id)
|
||||
_write_atomic(path, text)
|
||||
if reindex:
|
||||
build_index()
|
||||
|
||||
|
||||
def set_meta(bug_id, fields, reindex=True):
|
||||
"""Set front-matter fields only; the body bytes are left untouched."""
|
||||
path = record_path(bug_id)
|
||||
text = path.read_text(encoding="utf-8")
|
||||
rec = parse(text, path)
|
||||
rec.meta.update(fields)
|
||||
head_end = text.index("\n---\n", 3) + len("\n---\n")
|
||||
new = render_meta(rec.meta) + text[head_end:]
|
||||
if parse(new, path).meta != rec.meta:
|
||||
raise BuglogEditError(f"{bug_id}: front matter would not round-trip")
|
||||
_write_atomic(path, new)
|
||||
if reindex:
|
||||
build_index()
|
||||
|
||||
|
||||
# Kept for scripts/gitea-sync.py and older callers.
|
||||
def set_fields(_path_ignored, bug_id, fields):
|
||||
set_meta(bug_id, fields)
|
||||
|
||||
|
||||
# ── Index ────────────────────────────────────────────────────────────────────
|
||||
|
||||
def index_entry(rec):
|
||||
s, m = rec.sections, rec.meta
|
||||
unesc = lambda k: unescape_text(s.get(k, "")) # noqa: E731
|
||||
fix = unesc("Fix")
|
||||
if s.get("Verification") and s["Verification"] != NOT_RECORDED:
|
||||
fix += "\n\nVerification:\n" + unesc("Verification")
|
||||
entry = {
|
||||
"id": rec.id,
|
||||
"status": m.get("status"),
|
||||
"status_updated": m.get("status_updated"),
|
||||
"status_note": m.get("status_note", ""),
|
||||
"component": m.get("component"),
|
||||
"severity": m.get("severity"),
|
||||
"timestamp": m.get("timestamp"),
|
||||
"last_seen": m.get("last_seen", m.get("timestamp")),
|
||||
"occurrences": m.get("occurrences", 1),
|
||||
"error_message": (rec.summary + "\n\n" + unesc("Description")).strip(),
|
||||
"file": ", ".join(m.get("files") or []),
|
||||
"root_cause": unesc("Root cause"),
|
||||
"fix": fix.strip(),
|
||||
"tags": m.get("tags") or [],
|
||||
"related_bugs": m.get("related") or [],
|
||||
"fix_commits": m.get("fix_commits") or [],
|
||||
"record": str(record_path(rec.id).relative_to(ROOT)),
|
||||
}
|
||||
for k in ("duplicate_of", "repro_attempts", "gitea_issue", "gitea_synced_status"):
|
||||
if k in m:
|
||||
entry[k] = m[k]
|
||||
return entry
|
||||
|
||||
|
||||
def build_index():
|
||||
data = {
|
||||
"version": 1,
|
||||
"generated": "by scripts/buglog.py from .wolf/bugs/*.md -- do not edit",
|
||||
"bugs": [index_entry(r) for r in load_all()],
|
||||
}
|
||||
_write_atomic(INDEX_PATH, json.dumps(data, indent=2, ensure_ascii=False) + "\n")
|
||||
|
||||
|
||||
def next_id():
|
||||
ids = [p.stem for p in BUGS_DIR.glob("*.md")]
|
||||
if ARCHIVE_PATH.exists():
|
||||
ids += [b.get("id", "") for b in
|
||||
json.loads(ARCHIVE_PATH.read_text(encoding="utf-8")).get("bugs", [])]
|
||||
nums = [int(m.group(1)) for i in ids if (m := re.match(r"bug-(\d+)", i))]
|
||||
return f"bug-{max(nums, default=0) + 1:03d}"
|
||||
|
||||
|
||||
# ── Checks against the record standard ──────────────────────────────────────
|
||||
|
||||
def _empty(value):
|
||||
v = (value or "").strip()
|
||||
return not v or v.upper().startswith("TODO") or v == NOT_RECORDED
|
||||
|
||||
|
||||
def _item(section, label):
|
||||
"""Value of a `- Label: value` list item, or None."""
|
||||
m = re.search(r"^\s*[-*] " + re.escape(label) + r":[ \t]*(.*)$", section or "", re.M)
|
||||
return m.group(1).strip() if m else None
|
||||
|
||||
|
||||
def _bold(section, label):
|
||||
"""Text after `**Label:**` up to the next bold label."""
|
||||
m = re.search(r"\*\*" + re.escape(label) + r":\*\*(.*?)(?=^\*\*[\w ]+:\*\*|\Z)",
|
||||
section or "", re.S | re.M)
|
||||
return m.group(1).strip() if m else None
|
||||
|
||||
|
||||
def bad_commits(shas):
|
||||
bad = []
|
||||
for sha in shas:
|
||||
if not re.fullmatch(r"[0-9a-f]{7,40}", str(sha)) or \
|
||||
_run(["git", "-C", str(ROOT), "cat-file", "-e", f"{sha}^{{commit}}"]) is None:
|
||||
bad.append(sha)
|
||||
return bad
|
||||
|
||||
|
||||
def check(rec, target_status=None):
|
||||
"""Problems with `rec` under the record standard, as a list of strings.
|
||||
|
||||
`target_status` checks the record as if it were moved to that status:
|
||||
that is how `status` refuses an incomplete change. Legacy (migrated)
|
||||
records are exempt from the reporting sections, but a status change to
|
||||
fixed/partial still needs the fix-time sections.
|
||||
"""
|
||||
m, s, P = rec.meta, rec.sections, []
|
||||
status = target_status or m.get("status")
|
||||
legacy = bool(m.get("legacy"))
|
||||
strict_fix = not legacy or target_status is not None
|
||||
|
||||
if not ID_RE.fullmatch(str(m.get("id", ""))):
|
||||
P.append(f"id {m.get('id')!r} is not a bug id")
|
||||
elif rec.path and Path(rec.path).stem != m["id"]:
|
||||
P.append(f"id {m['id']} does not match file name {Path(rec.path).name}")
|
||||
if status not in STATUSES:
|
||||
P.append(f"status {status!r} is not one of {', '.join(STATUSES)}")
|
||||
if status not in ("fixed", None) and _empty(m.get("status_note")):
|
||||
P.append(f"status {status} needs a status_note (what remains / why / what was seen)")
|
||||
if status == "duplicate" and not m.get("duplicate_of"):
|
||||
P.append("status duplicate needs duplicate_of")
|
||||
if status == "cannot-reproduce" and not m.get("repro_attempts"):
|
||||
P.append("status cannot-reproduce needs repro_attempts (runs tried, date)")
|
||||
|
||||
if not legacy:
|
||||
if m.get("component") not in COMPONENTS:
|
||||
P.append(f"component {m.get('component')!r} is not one of {', '.join(COMPONENTS)}")
|
||||
if m.get("severity") not in SEVERITIES:
|
||||
P.append(f"severity {m.get('severity')!r} is not one of {', '.join(SEVERITIES)}")
|
||||
if not re.fullmatch(r"\d{4}-\d\d-\d\dT\d\d:\d\d:\d\d(\.\d+)?Z", str(m.get("timestamp", ""))):
|
||||
P.append("timestamp must be ISO-8601 UTC, e.g. 2026-09-27T14:05:00Z")
|
||||
if _empty(rec.summary):
|
||||
P.append("Summary (the # title) is empty")
|
||||
elif len(rec.summary) > 100:
|
||||
P.append(f"Summary is {len(rec.summary)} chars, max 100")
|
||||
elif STATUS_WORDS.search(rec.summary):
|
||||
P.append("Summary must not carry status words (OPEN/FIXED/PRIORITY)")
|
||||
|
||||
d = s.get("Description")
|
||||
for label in ("Observed", "Expected", "Impact", "Found by"):
|
||||
if _empty(_bold(d, label)):
|
||||
P.append(f"Description: **{label}:** is missing or empty")
|
||||
|
||||
e = s.get("Environment")
|
||||
for label, rx, why in (
|
||||
("OS", r"\d+\.\d+", "distribution/version and kernel version"),
|
||||
("libc", r"\d+\.\d+", "a version, from the failing binary's interpreter"),
|
||||
("Toolchain", r"\d+\.\d+", "compiler and build tool versions"),
|
||||
("Build", r"\b(asan|debug|release)\b", "the build profile: asan, debug or release"),
|
||||
("Code", r"\b[0-9a-f]{7,40}\b", "the commit the binary was built from"),
|
||||
("Deployment", r"\S", "topology, worker kind, hosts")):
|
||||
v = _item(e, label)
|
||||
if _empty(v):
|
||||
P.append(f"Environment: - {label}: is missing or empty ({why})")
|
||||
elif not re.search(rx, v):
|
||||
P.append(f"Environment: - {label}: needs {why}")
|
||||
|
||||
r = s.get("Reproduction") or ""
|
||||
steps = re.search(r"^\s*\d+\.\s+(?!TODO)\S", r, re.M) or "```" in r
|
||||
if not steps:
|
||||
P.append("Reproduction: needs numbered steps (1. <command>) or a code block")
|
||||
freq = _item(r, "Frequency")
|
||||
if _empty(freq):
|
||||
P.append("Reproduction: - Frequency: is missing (deterministic, or N/M runs with the date)")
|
||||
elif not re.search(r"deterministic|\d+\s*/\s*\d+", freq, re.I):
|
||||
P.append("Reproduction: - Frequency: must be 'deterministic' or 'N/M runs, <date>'")
|
||||
if _empty(_item(r, "Evidence")):
|
||||
P.append("Reproduction: - Evidence: is missing (a durable path, or 'not kept')")
|
||||
|
||||
if strict_fix and status in ("fixed", "partial"):
|
||||
for name in ("Root cause", "Fix"):
|
||||
if _empty(s.get(name)):
|
||||
P.append(f"{name}: required for status {status}")
|
||||
if strict_fix and status == "fixed":
|
||||
v = s.get("Verification")
|
||||
if _empty(v):
|
||||
P.append("Verification: required for status fixed")
|
||||
else:
|
||||
for label in ("Test", "Suites"):
|
||||
if _empty(_item(v, label)):
|
||||
P.append(f"Verification: - {label}: is missing or empty")
|
||||
commits = m.get("fix_commits") or []
|
||||
if not commits:
|
||||
P.append("fix_commits: status fixed needs the fix commit(s) (status ... --commit <sha>)")
|
||||
for sha in bad_commits(commits):
|
||||
P.append(f"fix_commits: {sha} is not a commit in this repository")
|
||||
return P
|
||||
|
||||
|
||||
# ── Environment collection ───────────────────────────────────────────────────
|
||||
|
||||
def _os_line():
|
||||
kernel = f"{platform.system()} {platform.release()} {platform.machine()}"
|
||||
distro = None
|
||||
for f in ("/etc/os-release", "/usr/lib/os-release"):
|
||||
if os.path.exists(f):
|
||||
m = re.search(r'^PRETTY_NAME="?([^"\n]*)', Path(f).read_text(), re.M)
|
||||
distro = m.group(1) if m else None
|
||||
break
|
||||
parts = [distro or "no /etc/os-release"]
|
||||
if os.path.exists("/.dockerenv"):
|
||||
parts.append("Docker container")
|
||||
try:
|
||||
if "NixOS" in Path("/proc/version").read_text():
|
||||
parts.append("NixOS kernel")
|
||||
except OSError:
|
||||
pass
|
||||
return "; ".join(parts + [f"kernel {kernel}"])
|
||||
|
||||
|
||||
def _libc_line(binary):
|
||||
if binary:
|
||||
out = _run(["readelf", "-l", str(binary)]) or ""
|
||||
m = re.search(r"interpreter: ([^\]]+)\]", out)
|
||||
if m:
|
||||
v = re.search(r"glibc-(\d+\.\d+[\w.-]*)", m.group(1))
|
||||
name = f"glibc {v.group(1)}" if v else "unknown libc"
|
||||
return f"{name} (interpreter {m.group(1)})"
|
||||
out = _run(["ldd", "--version"]) or ""
|
||||
v = re.search(r"(\d+\.\d+)", out.split("\n")[0]) if out else None
|
||||
host = f"glibc {v.group(1)}" if v else "unknown"
|
||||
return f"TODO: pass the failing binary (`buglog.py env <binary>`); host ldd reports {host}"
|
||||
|
||||
|
||||
def _toolchain_line():
|
||||
probe = "gcc --version | head -1; meson --version"
|
||||
if os.environ.get("IN_NIX_SHELL") or not shutil.which("nix"):
|
||||
out = _run(["bash", "-c", probe])
|
||||
else:
|
||||
out = _run(["nix", "develop", str(ROOT / "src"), "--command", "bash", "-c", probe],
|
||||
cwd=str(ROOT))
|
||||
if not out:
|
||||
return "TODO: gcc and meson versions from the shell that built the binary"
|
||||
gcc, _, meson = out.partition("\n")
|
||||
return f"{gcc.strip()}, meson {meson.strip()}"
|
||||
|
||||
|
||||
def _build_line(binary):
|
||||
profile = None
|
||||
if binary:
|
||||
for d, p in (("xce.build", "asan"), ("build.dbg", "debug"), ("build.release", "release")):
|
||||
if d in str(binary):
|
||||
profile = f"{p} ({d})"
|
||||
line = profile or "TODO: asan (xce.build) | debug (build.dbg) | release (build.release)"
|
||||
if os.environ.get("ASAN_OPTIONS"):
|
||||
line += f", ASAN_OPTIONS={os.environ['ASAN_OPTIONS']}"
|
||||
return line
|
||||
|
||||
|
||||
def _code_line():
|
||||
sha = _run(["git", "-C", str(ROOT), "rev-parse", "--short=8", "HEAD"])
|
||||
if not sha:
|
||||
return "TODO: branch @ commit"
|
||||
branch = _run(["git", "-C", str(ROOT), "rev-parse", "--abbrev-ref", "HEAD"]) or "?"
|
||||
dirty = _run(["git", "-C", str(ROOT), "status", "--porcelain", "--untracked-files=no"])
|
||||
return f"{branch} @ {sha}" + ("-dirty" if dirty else "")
|
||||
|
||||
|
||||
def collect_env(binary=None):
|
||||
return "\n".join([
|
||||
f"- OS: {_os_line()}",
|
||||
f"- libc: {_libc_line(binary)}",
|
||||
f"- Toolchain: {_toolchain_line()}",
|
||||
f"- Build: {_build_line(binary)}",
|
||||
f"- Code: {_code_line()}",
|
||||
"- Deployment: TODO: topology (single / 3-node HA / multihost), worker kind, hosts",
|
||||
])
|
||||
|
||||
|
||||
TEMPLATE = {
|
||||
"Description": "**Observed:**\nTODO: the exact error, log line or sanitizer text, in a code block\n\n"
|
||||
"**Expected:** TODO\n**Impact:** TODO\n"
|
||||
"**Found by:** TODO: user report | soak | ASAN | audit | review | test | spec task",
|
||||
"Reproduction": "1. TODO: commands someone else can paste\n"
|
||||
"- Frequency: TODO: deterministic | N/M runs, <date>\n"
|
||||
"- Evidence: TODO: durable path to logs/cores/replays, or 'not kept'",
|
||||
"Root cause": "TODO: required for fixed/partial",
|
||||
"Fix": "TODO: required for fixed/partial",
|
||||
"Verification": "- Test: TODO: name + file; RED before the fix, GREEN after\n"
|
||||
"- Suites: TODO: suite results and build profile\n"
|
||||
"- Live: TODO: soak/deploy run, or n/a and why",
|
||||
"References": "- Related: none",
|
||||
}
|
||||
|
||||
|
||||
# ── Commands ─────────────────────────────────────────────────────────────────
|
||||
|
||||
def _line(r):
|
||||
note = r.meta.get("status_note") or r.summary
|
||||
return f"{r.id:<10} {r.meta.get('status') or '-':<16} {note[:110]}"
|
||||
|
||||
|
||||
def _fail(problems, what):
|
||||
sys.exit(f"{what}:\n" + "\n".join(f" - {p}" for p in problems)
|
||||
+ "\nStandard: .claude/skills/buglog/bug-record.md")
|
||||
|
||||
|
||||
def cmd_new(a):
|
||||
if a.component not in COMPONENTS:
|
||||
sys.exit(f"--component must be one of {', '.join(COMPONENTS)}")
|
||||
if a.severity not in SEVERITIES:
|
||||
sys.exit(f"--severity must be one of {', '.join(SEVERITIES)}")
|
||||
bid, now = next_id(), _now()
|
||||
meta = {"id": bid, "status": "open", "status_updated": _today(),
|
||||
"status_note": "newly logged", "component": a.component,
|
||||
"severity": a.severity, "timestamp": now, "last_seen": now,
|
||||
"occurrences": 1, "files": _split(a.files), "tags": _split(a.tags),
|
||||
"related": _split(a.related)}
|
||||
sections = dict(TEMPLATE)
|
||||
sections["Environment"] = collect_env(a.binary)
|
||||
rec = Record(meta, a.summary, {k: sections[k] for k in SECTIONS if k in sections})
|
||||
save(rec)
|
||||
print(record_path(bid).relative_to(ROOT))
|
||||
probs = check(load(bid))
|
||||
if probs:
|
||||
print(f"{len(probs)} item(s) still to fill in; run `buglog.py check {bid}` when done.")
|
||||
|
||||
|
||||
def _staged(pathspec, diff_filter="ACMD"):
|
||||
out = _run(["git", "-C", str(ROOT), "diff", "--cached", "--name-only",
|
||||
f"--diff-filter={diff_filter}", "--", pathspec]) or ""
|
||||
return [p for p in out.split("\n") if p]
|
||||
|
||||
|
||||
def _staged_records():
|
||||
return [Path(p).stem for p in _staged(".wolf/bugs/", "ACM") if p.endswith(".md")]
|
||||
|
||||
|
||||
_HEAD_INDEX = None
|
||||
|
||||
|
||||
def _head_status(bug_id):
|
||||
"""(found, status) of the bug at HEAD: its record, else its entry in
|
||||
HEAD's buglog.json (how the migration commit is recognised)."""
|
||||
global _HEAD_INDEX
|
||||
text = _run(["git", "-C", str(ROOT), "show", f"HEAD:.wolf/bugs/{bug_id}.md"])
|
||||
if text:
|
||||
try:
|
||||
return True, parse(text).meta.get("status")
|
||||
except BuglogEditError:
|
||||
return True, None
|
||||
if _HEAD_INDEX is None:
|
||||
raw = _run(["git", "-C", str(ROOT), "show", "HEAD:.wolf/buglog.json"])
|
||||
try:
|
||||
_HEAD_INDEX = {b.get("id"): b.get("status")
|
||||
for b in json.loads(raw).get("bugs", [])} if raw else {}
|
||||
except (json.JSONDecodeError, AttributeError):
|
||||
_HEAD_INDEX = {}
|
||||
if bug_id in _HEAD_INDEX:
|
||||
return True, _HEAD_INDEX[bug_id]
|
||||
return False, None
|
||||
|
||||
|
||||
def cmd_check(a):
|
||||
global _HEAD_INDEX
|
||||
_HEAD_INDEX = None # read HEAD afresh on every check
|
||||
ids = a.ids or []
|
||||
if a.staged:
|
||||
ids = _staged_records()
|
||||
elif a.all or not ids:
|
||||
ids = [r.id for r in load_all()]
|
||||
failed = 0
|
||||
for bid in ids:
|
||||
rec = load(bid)
|
||||
target, extra = None, []
|
||||
if a.staged and rec.meta.get("legacy"):
|
||||
found, head = _head_status(bid)
|
||||
if not found:
|
||||
extra.append("a new record cannot be legacy: true (only migrated records are)")
|
||||
elif head != rec.meta.get("status"):
|
||||
# a legacy record whose status changed in this commit must
|
||||
# meet what the new status requires
|
||||
target = rec.meta.get("status")
|
||||
probs = extra + check(rec, target_status=target)
|
||||
if probs:
|
||||
failed += 1
|
||||
print(f"{bid}:")
|
||||
for p in probs:
|
||||
print(f" - {p}")
|
||||
if a.staged and (_staged(".wolf/bugs/") or _staged(".wolf/buglog.json")):
|
||||
expect = json.dumps({"version": 1,
|
||||
"generated": "by scripts/buglog.py from .wolf/bugs/*.md -- do not edit",
|
||||
"bugs": [index_entry(r) for r in load_all()]},
|
||||
indent=2, ensure_ascii=False) + "\n"
|
||||
if not INDEX_PATH.exists() or INDEX_PATH.read_text(encoding="utf-8") != expect:
|
||||
failed += 1
|
||||
print(".wolf/buglog.json is stale: run `python3 scripts/buglog.py index` and stage it")
|
||||
if failed:
|
||||
sys.exit(f"{failed} record(s) do not meet .claude/skills/buglog/bug-record.md")
|
||||
if not a.staged:
|
||||
print(f"{len(ids)} record(s) OK")
|
||||
|
||||
|
||||
def cmd_status(a):
|
||||
if a.status not in STATUSES:
|
||||
sys.exit(f"unknown status {a.status!r}; one of {', '.join(STATUSES)}")
|
||||
bugs = _load(path)
|
||||
bid = next_id(bugs)
|
||||
now = _now()
|
||||
entry = {"id": bid, "status": a.status, "status_updated": _today()}
|
||||
if a.note:
|
||||
entry["status_note"] = a.note
|
||||
entry.update({
|
||||
"timestamp": now,
|
||||
"error_message": a.error,
|
||||
"file": a.file or "",
|
||||
"root_cause": a.root_cause or "",
|
||||
"fix": a.fix or "",
|
||||
"tags": _split(a.tags),
|
||||
"related_bugs": _split(a.related),
|
||||
"occurrences": 1,
|
||||
"last_seen": now,
|
||||
})
|
||||
append_entry(path, entry)
|
||||
print(bid)
|
||||
|
||||
|
||||
def cmd_status(a, path):
|
||||
if a.status not in STATUSES:
|
||||
sys.exit(f"unknown status {a.status!r}; one of {', '.join(STATUSES)}")
|
||||
if a.status == "duplicate" and not a.duplicate_of:
|
||||
sys.exit("duplicate needs --duplicate-of")
|
||||
if a.status == "cannot-reproduce" and not a.repro:
|
||||
sys.exit("cannot-reproduce needs --repro (what was tried, e.g. '0/40 runs, 2026-08-01')")
|
||||
if a.note is None and a.status != "fixed":
|
||||
sys.exit(f"{a.status} needs --note (what remains / why / what was seen)")
|
||||
# Always (re)write the note so a stale one never outlives its status.
|
||||
fields = {"status": a.status, "status_updated": _today(),
|
||||
"status_note": a.note or ""}
|
||||
rec = load(a.id)
|
||||
rec.meta.update({"status": a.status, "status_updated": _today(),
|
||||
"status_note": a.note or ""})
|
||||
if a.commit:
|
||||
rec.meta["fix_commits"] = list(dict.fromkeys((rec.meta.get("fix_commits") or [])
|
||||
+ a.commit))
|
||||
if a.duplicate_of:
|
||||
fields["duplicate_of"] = a.duplicate_of
|
||||
rec.meta["duplicate_of"] = a.duplicate_of
|
||||
if a.repro:
|
||||
fields["repro_attempts"] = a.repro
|
||||
set_fields(path, a.id, fields)
|
||||
print(_line(_find(_load(path), a.id)))
|
||||
rec.meta["repro_attempts"] = a.repro
|
||||
probs = check(rec, target_status=a.status)
|
||||
if probs:
|
||||
_fail(probs, f"{a.id}: not changed to {a.status}")
|
||||
set_meta(a.id, {k: rec.meta[k] for k in rec.meta})
|
||||
print(_line(load(a.id)))
|
||||
|
||||
|
||||
def cmd_note(a, path):
|
||||
fix = _find(_load(path), a.id).get("fix") or ""
|
||||
if isinstance(fix, list):
|
||||
raise BuglogEditError(f"{a.id}.fix is a list, edit by hand")
|
||||
para = f"UPDATE ({_today()}): {a.text}"
|
||||
set_fields(path, a.id, {"fix": f"{fix}\n\n{para}" if fix else para})
|
||||
def cmd_note(a):
|
||||
if a.section not in SECTIONS:
|
||||
sys.exit(f"--section must be one of {', '.join(SECTIONS)}")
|
||||
rec = load(a.id)
|
||||
old = rec.sections.get(a.section, "")
|
||||
old = "" if _empty(old) else old
|
||||
para = f"UPDATE ({_today()}): {escape_text(a.text)}"
|
||||
rec.sections[a.section] = f"{old}\n\n{para}" if old else para
|
||||
save(rec)
|
||||
|
||||
|
||||
def cmd_recur(a, path):
|
||||
b = _find(_load(path), a.id)
|
||||
fields = {"occurrences": int(b.get("occurrences") or 1) + 1, "last_seen": _now()}
|
||||
def cmd_recur(a):
|
||||
rec = load(a.id)
|
||||
fields = {"occurrences": int(rec.meta.get("occurrences") or 1) + 1, "last_seen": _now()}
|
||||
reopen = {"cannot-reproduce": "investigating", "fixed": "open", "wontfix": "open"}
|
||||
if b.get("status") in reopen:
|
||||
fields.update(status=reopen[b["status"]], status_updated=_today())
|
||||
fields["status_note"] = a.note or f"recurred {_today()} (was {b['status']})"
|
||||
if rec.meta.get("status") in reopen:
|
||||
fields.update(status=reopen[rec.meta["status"]], status_updated=_today())
|
||||
fields["status_note"] = a.note or f"recurred {_today()} (was {rec.meta['status']})"
|
||||
elif a.note:
|
||||
fields["status_note"] = a.note
|
||||
set_fields(path, a.id, fields)
|
||||
print(_line(_find(_load(path), a.id)))
|
||||
set_meta(a.id, fields)
|
||||
print(_line(load(a.id)))
|
||||
|
||||
|
||||
def cmd_set(a, path):
|
||||
def cmd_set(a):
|
||||
fields = {}
|
||||
for kv in a.pairs:
|
||||
key, sep, raw = kv.partition("=")
|
||||
|
|
@ -238,55 +684,53 @@ def cmd_set(a, path):
|
|||
fields[key] = json.loads(raw)
|
||||
except json.JSONDecodeError:
|
||||
fields[key] = raw
|
||||
set_fields(path, a.id, fields)
|
||||
set_meta(a.id, fields)
|
||||
|
||||
|
||||
def cmd_show(a, path):
|
||||
print(json.dumps(_find(_load(path), a.id), indent=2, ensure_ascii=False))
|
||||
|
||||
|
||||
def cmd_list(a, path):
|
||||
def cmd_list(a):
|
||||
want = OPEN_STATUSES if a.open else set(_split(a.status)) or None
|
||||
rx = re.compile(a.grep, re.I) if a.grep else None
|
||||
for b in _load(path):
|
||||
if want is not None and b.get("status") not in want:
|
||||
for r in load_all():
|
||||
m = r.meta
|
||||
if want is not None and m.get("status") not in want:
|
||||
continue
|
||||
if a.tag and a.tag not in (b.get("tags") or []):
|
||||
if a.tag and a.tag not in (m.get("tags") or []):
|
||||
continue
|
||||
if a.since and max(b.get("last_seen") or "", b.get("status_updated") or "") < a.since:
|
||||
if a.since and max(m.get("last_seen") or "", m.get("status_updated") or "") < a.since:
|
||||
continue
|
||||
if rx and not rx.search(json.dumps(b, ensure_ascii=False)):
|
||||
if rx and not rx.search(render(r)):
|
||||
continue
|
||||
print(_line(b))
|
||||
|
||||
|
||||
def cmd_triage(a, path):
|
||||
for b in _load(path):
|
||||
if b.get("status") not in STATUSES:
|
||||
print(_line(b))
|
||||
print(_line(r))
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
p = argparse.ArgumentParser(description=__doc__.split("\n")[0],
|
||||
epilog="Status values: " + ", ".join(STATUSES))
|
||||
p.add_argument("--path", default=str(DEFAULT_PATH), help=argparse.SUPPRESS)
|
||||
sub = p.add_subparsers(dest="cmd", required=True)
|
||||
|
||||
s = sub.add_parser("add")
|
||||
s.add_argument("--error", required=True)
|
||||
s.add_argument("--root-cause")
|
||||
s.add_argument("--fix")
|
||||
s.add_argument("--file")
|
||||
s = sub.add_parser("new")
|
||||
s.add_argument("--summary", required=True)
|
||||
s.add_argument("--component", required=True, help=", ".join(COMPONENTS))
|
||||
s.add_argument("--severity", required=True, help=", ".join(SEVERITIES))
|
||||
s.add_argument("--binary", help="the failing binary, for its libc and build profile")
|
||||
s.add_argument("--tags")
|
||||
s.add_argument("--files")
|
||||
s.add_argument("--related")
|
||||
s.add_argument("--status", default="open")
|
||||
s.add_argument("--note")
|
||||
s.set_defaults(fn=cmd_add)
|
||||
s.set_defaults(fn=cmd_new)
|
||||
|
||||
s = sub.add_parser("check")
|
||||
s.add_argument("ids", nargs="*")
|
||||
s.add_argument("--all", action="store_true")
|
||||
s.add_argument("--staged", action="store_true",
|
||||
help="records staged for commit (the pre-commit hook)")
|
||||
s.set_defaults(fn=cmd_check)
|
||||
|
||||
s = sub.add_parser("status")
|
||||
s.add_argument("id")
|
||||
s.add_argument("status")
|
||||
s.add_argument("--note")
|
||||
s.add_argument("--commit", action="append", default=[],
|
||||
help="fix commit (repeatable); required for fixed")
|
||||
s.add_argument("--duplicate-of")
|
||||
s.add_argument("--repro")
|
||||
s.set_defaults(fn=cmd_status)
|
||||
|
|
@ -294,6 +738,7 @@ def main(argv=None):
|
|||
s = sub.add_parser("note")
|
||||
s.add_argument("id")
|
||||
s.add_argument("text")
|
||||
s.add_argument("--section", default="Description")
|
||||
s.set_defaults(fn=cmd_note)
|
||||
|
||||
s = sub.add_parser("recur")
|
||||
|
|
@ -308,7 +753,7 @@ def main(argv=None):
|
|||
|
||||
s = sub.add_parser("show")
|
||||
s.add_argument("id")
|
||||
s.set_defaults(fn=cmd_show)
|
||||
s.set_defaults(fn=lambda a: print(render(load(a.id)), end=""))
|
||||
|
||||
s = sub.add_parser("list")
|
||||
s.add_argument("--status")
|
||||
|
|
@ -319,14 +764,22 @@ def main(argv=None):
|
|||
s.set_defaults(fn=cmd_list)
|
||||
|
||||
s = sub.add_parser("triage")
|
||||
s.set_defaults(fn=cmd_triage)
|
||||
s.set_defaults(fn=lambda a: [print(_line(r)) for r in load_all()
|
||||
if r.meta.get("status") not in STATUSES])
|
||||
|
||||
s = sub.add_parser("env")
|
||||
s.add_argument("binary", nargs="?")
|
||||
s.set_defaults(fn=lambda a: print(collect_env(a.binary)))
|
||||
|
||||
s = sub.add_parser("index")
|
||||
s.set_defaults(fn=lambda a: build_index())
|
||||
|
||||
s = sub.add_parser("next-id")
|
||||
s.set_defaults(fn=lambda a, path: print(next_id(_load(path))))
|
||||
s.set_defaults(fn=lambda a: print(next_id()))
|
||||
|
||||
a = p.parse_args(argv)
|
||||
try:
|
||||
a.fn(a, a.path)
|
||||
a.fn(a)
|
||||
except BuglogEditError as e:
|
||||
sys.exit(str(e))
|
||||
|
||||
|
|
|
|||
7
scripts/git-hooks/pre-commit
Executable file
7
scripts/git-hooks/pre-commit
Executable file
|
|
@ -0,0 +1,7 @@
|
|||
#!/bin/sh
|
||||
# Refuse commits whose bug records (.wolf/bugs/*.md) do not meet
|
||||
# .claude/skills/buglog/bug-record.md, or whose .wolf/buglog.json index is stale.
|
||||
# Install: ln -sf ../../scripts/git-hooks/pre-commit .git/hooks/pre-commit
|
||||
if git diff --cached --name-only -- .wolf/bugs/ .wolf/buglog.json | grep -q .; then
|
||||
exec python3 "$(git rev-parse --show-toplevel)/scripts/buglog.py" check --staged
|
||||
fi
|
||||
|
|
@ -3,7 +3,7 @@
|
|||
xce-gitea-sync: Sync cc-sdd specs and OpenWolf buglog to Gitea issues.
|
||||
|
||||
Reads .kiro/specs/<feature>/ (spec.json, brief.md, requirements.md, tasks.md)
|
||||
and .wolf/buglog.json as ground truth. Creates/updates Gitea milestones, issues,
|
||||
and the bug records in .wolf/bugs/*.md as ground truth. Creates/updates Gitea milestones, issues,
|
||||
and labels to mirror the local state.
|
||||
|
||||
Sync is one-way: files -> Gitea. Never Gitea -> files.
|
||||
|
|
@ -20,6 +20,8 @@ Environment:
|
|||
GITEA_TOKEN - API token (required: write:issue, write:milestone scopes)
|
||||
GITEA_OWNER - repo owner (default: nexa)
|
||||
GITEA_REPO - repo name (default: xce)
|
||||
GITEA_COMMIT_REPO - owner/repo that fix-commit links point at
|
||||
(default: the `origin` remote's owner/repo)
|
||||
"""
|
||||
|
||||
import argparse
|
||||
|
|
@ -28,6 +30,7 @@ import hashlib
|
|||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
|
|
@ -35,7 +38,8 @@ import urllib.request
|
|||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
from buglog import BuglogEditError, set_fields # noqa: E402
|
||||
import buglog # noqa: E402
|
||||
from buglog import BuglogEditError # noqa: E402
|
||||
|
||||
|
||||
# ── Helpers ──────────────────────────────────────────────────────────────────
|
||||
|
|
@ -99,7 +103,6 @@ REPO_API = f"{API_BASE}/repos/{GITEA_OWNER}/{GITEA_REPO}"
|
|||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
PROJECT_ROOT = SCRIPT_DIR.parent
|
||||
SPECS_DIR = PROJECT_ROOT / ".kiro" / "specs"
|
||||
BUGLOG_PATH = PROJECT_ROOT / ".wolf" / "buglog.json"
|
||||
|
||||
|
||||
# ── Label Definitions ────────────────────────────────────────────────────────
|
||||
|
|
@ -190,14 +193,14 @@ STATUS_PRIORITY = ["open", "investigating", "partial", None,
|
|||
"cannot-reproduce", "duplicate", "wontfix", "fixed"]
|
||||
|
||||
|
||||
def bug_status(bug_entry):
|
||||
"""The entry's status, or None when it has none / an unknown one."""
|
||||
status = bug_entry.get("status")
|
||||
def bug_status(rec):
|
||||
"""The record's status, or None when it has none / an unknown one."""
|
||||
status = rec.meta.get("status")
|
||||
return status if status in STATUS_MAP else None
|
||||
|
||||
|
||||
def bug_state(bug_entry):
|
||||
return STATUS_MAP.get(bug_status(bug_entry), ("open", None))[0]
|
||||
def bug_state(rec):
|
||||
return STATUS_MAP.get(bug_status(rec), ("open", None))[0]
|
||||
|
||||
|
||||
def _text(value):
|
||||
|
|
@ -342,14 +345,20 @@ def status_label(status):
|
|||
STATUS_LABELS = {lbl for _, lbl in STATUS_MAP.values() if lbl}
|
||||
|
||||
|
||||
def bug_labels(bug_entry):
|
||||
"""Derive Gitea labels from the entry's status and tags."""
|
||||
SEVERITY_TO_LABEL = {"crash": "bug:crash", "data-loss": "bug:data-loss"}
|
||||
|
||||
|
||||
def bug_labels(rec):
|
||||
"""Derive Gitea labels from the record's status, severity and tags."""
|
||||
labels = ["bug"]
|
||||
if status_label(bug_status(bug_entry)):
|
||||
labels.append(status_label(bug_status(bug_entry)))
|
||||
if status_label(bug_status(rec)):
|
||||
labels.append(status_label(bug_status(rec)))
|
||||
sev = SEVERITY_TO_LABEL.get(rec.meta.get("severity"))
|
||||
if sev:
|
||||
labels.append(sev)
|
||||
|
||||
# Map known tag patterns to labels
|
||||
for tag in (t.lower() for t in bug_entry.get("tags", [])):
|
||||
for tag in (t.lower() for t in rec.meta.get("tags") or []):
|
||||
for pattern, label in TAG_TO_LABEL.items():
|
||||
if pattern.lower() in tag and label not in labels:
|
||||
labels.append(label)
|
||||
|
|
@ -360,88 +369,73 @@ def bug_labels(bug_entry):
|
|||
|
||||
# ── Body Formatting ──────────────────────────────────────────────────────────
|
||||
|
||||
def bug_body(bug_entry):
|
||||
"""Format a buglog entry as a Gitea issue body."""
|
||||
parts = []
|
||||
def _commit_repo():
|
||||
"""owner/repo that fix-commit links point at: GITEA_COMMIT_REPO, else the
|
||||
`origin` remote (fixes land on a fork branch before they reach
|
||||
GITEA_OWNER/GITEA_REPO, so a bare hash there may not resolve yet)."""
|
||||
repo = os.environ.get("GITEA_COMMIT_REPO")
|
||||
if repo:
|
||||
return repo
|
||||
try:
|
||||
url = subprocess.run(["git", "-C", str(PROJECT_ROOT), "remote", "get-url", "origin"],
|
||||
capture_output=True, text=True, check=True).stdout.strip()
|
||||
except (OSError, subprocess.CalledProcessError):
|
||||
return f"{GITEA_OWNER}/{GITEA_REPO}"
|
||||
m = re.search(r"[:/]([^/:]+/[^/]+?)(?:\.git)?$", url)
|
||||
return m.group(1) if m else f"{GITEA_OWNER}/{GITEA_REPO}"
|
||||
|
||||
status = bug_entry.get("status") or "unknown (needs triage)"
|
||||
note = _text(bug_entry.get("status_note"))
|
||||
|
||||
def bug_body(rec):
|
||||
"""Format a bug record as a Gitea issue body: a status header, then the
|
||||
record's own sections (.claude/skills/buglog/bug-record.md)."""
|
||||
m = rec.meta
|
||||
parts = []
|
||||
status = m.get("status") or "unknown (needs triage)"
|
||||
note = _text(m.get("status_note"))
|
||||
parts.append(f"**Status:** {status}" + (f" — {note}" if note else ""))
|
||||
if bug_entry.get("duplicate_of"):
|
||||
parts.append(f"Duplicate of {bug_entry['duplicate_of']}")
|
||||
if bug_entry.get("repro_attempts"):
|
||||
parts.append(f"Repro attempts: {_text(bug_entry['repro_attempts'])}")
|
||||
parts.append(f"**Component:** {m.get('component', 'unknown')} · "
|
||||
f"**Severity:** {m.get('severity', 'unknown')}")
|
||||
commits = m.get("fix_commits") or []
|
||||
if commits:
|
||||
base = f"{GITEA_URL}/{_commit_repo()}/commit"
|
||||
parts.append("**Fix commits:** " + ", ".join(f"[`{c}`]({base}/{c})" for c in commits))
|
||||
if m.get("duplicate_of"):
|
||||
parts.append(f"**Duplicate of:** {m['duplicate_of']}")
|
||||
if m.get("repro_attempts"):
|
||||
parts.append(f"**Repro attempts:** {_text(m['repro_attempts'])}")
|
||||
parts.append("")
|
||||
|
||||
error = _text(bug_entry.get("error_message"))
|
||||
if error:
|
||||
parts.append("## Error\n")
|
||||
parts.append(error)
|
||||
for name in buglog.SECTIONS:
|
||||
text = buglog.unescape_text(rec.sections.get(name, "")).strip()
|
||||
if not text or text == buglog.NOT_RECORDED:
|
||||
continue
|
||||
parts.append(f"## {name}\n")
|
||||
parts.append(text)
|
||||
parts.append("")
|
||||
|
||||
root_cause = _text(bug_entry.get("root_cause"))
|
||||
if root_cause:
|
||||
parts.append("## Root Cause\n")
|
||||
parts.append(root_cause)
|
||||
parts.append("")
|
||||
|
||||
fix = _text(bug_entry.get("fix"))
|
||||
if fix:
|
||||
parts.append("## Fix\n")
|
||||
parts.append(fix)
|
||||
parts.append("")
|
||||
else:
|
||||
parts.append("## Fix\n")
|
||||
parts.append("*Not yet fixed*")
|
||||
parts.append("")
|
||||
|
||||
files = [f.strip() for f in (bug_entry.get("file") or "").split(",")]
|
||||
files += [str(f).strip() for f in bug_entry.get("files") or []]
|
||||
files = [f for f in dict.fromkeys(files) if f]
|
||||
files = m.get("files") or []
|
||||
if files:
|
||||
parts.append("## Affected Files\n")
|
||||
for f in files:
|
||||
parts.append(f"- `{f}`")
|
||||
parts.extend(f"- `{f}`" for f in files)
|
||||
parts.append("")
|
||||
|
||||
tags = bug_entry.get("tags", [])
|
||||
tags = m.get("tags") or []
|
||||
if tags:
|
||||
parts.append("## Tags\n")
|
||||
parts.append(" ".join(f"`{t}`" for t in tags))
|
||||
parts.append("")
|
||||
|
||||
related = bug_entry.get("related_bugs", [])
|
||||
if related:
|
||||
parts.append("## Related Bugs\n")
|
||||
for rb in related:
|
||||
parts.append(f"- {rb}")
|
||||
parts.append("")
|
||||
|
||||
parts.append("---")
|
||||
parts.append(f"*Synced from `.wolf/buglog.json` "
|
||||
f"({bug_entry.get('id', '?')})*")
|
||||
|
||||
parts.append(f"*Synced from `.wolf/bugs/{rec.id}.md`*")
|
||||
return "\n".join(parts)
|
||||
|
||||
|
||||
def bug_title(bug_entry):
|
||||
"""Build a short title from a buglog entry."""
|
||||
bid = bug_entry.get("id", "bug-?")
|
||||
error = _text(bug_entry.get("error_message"))
|
||||
# Drop a leading status banner ("OPEN, NOT FIXED -- ...", "PRIORITY: LOW --
|
||||
# ...") -- status lives in labels/state, not the title.
|
||||
error = re.sub(r"^(?:\s*(?:OPEN|NOT (?:YET )?FIXED|NOT A CRASH[^,.]*|"
|
||||
r"NOT YET ROOT-CAUSED|DEAD-CODE-ONLY|PRIORITY: \w+|"
|
||||
r"low priority|pre-existing)\b\s*(?:,|--|\.|;)?)+\s*",
|
||||
"", error, flags=re.I) or error
|
||||
# Take first sentence or first 100 chars
|
||||
first_line = error.split("\n")[0].strip()
|
||||
first_sentence = re.split(r'[.。]\s+', first_line)[0].strip()
|
||||
if not first_sentence:
|
||||
first_sentence = first_line
|
||||
if len(first_sentence) > 120:
|
||||
first_sentence = first_sentence[:117] + "..."
|
||||
return f"[{bid}] {first_sentence}"
|
||||
def bug_title(rec):
|
||||
title = f"[{rec.id}] {rec.summary}"
|
||||
return title if len(title) <= 120 else title[:119] + "…"
|
||||
|
||||
|
||||
def body_hash(title, body):
|
||||
return hashlib.sha1(f"{title}\n{body}".encode("utf-8")).hexdigest()[:12]
|
||||
|
||||
|
||||
def spec_requirements_body(spec_name, spec_data, brief_path, req_path, design_path):
|
||||
|
|
@ -789,88 +783,90 @@ def _resolve_label_ids(api, label_names):
|
|||
# ── Bug Sync ─────────────────────────────────────────────────────────────────
|
||||
|
||||
def sync_bugs(api, dry_run=False, quiet=False, since=None, ids=None):
|
||||
"""Sync buglog entries to Gitea issues.
|
||||
"""Sync bug records (.wolf/bugs/*.md) to Gitea issues.
|
||||
|
||||
Open/closed and the status label come from each entry's `status` (see
|
||||
STATUS_MAP). `gitea_synced_status` records the status last mirrored, so an
|
||||
entry is only touched when its status changed -- including going back to
|
||||
open (reopen). Sync state is written back per entry, surgically, so a
|
||||
failure part-way through never loses an already-created issue number.
|
||||
Open/closed and the status label come from each record's `status` (see
|
||||
STATUS_MAP). `gitea_synced_status` records the status last mirrored, so a
|
||||
record is only re-mirrored when its status changed -- including going back
|
||||
to open (reopen). `gitea_synced_hash` records the title+body last written,
|
||||
so an issue's body is refreshed whenever the record's text changes (a bug
|
||||
created open gets its Fix/Verification/commits once it is fixed). Sync
|
||||
state is written back per record, so a failure part-way through never
|
||||
loses an already-created issue number.
|
||||
|
||||
Args:
|
||||
since: ISO date (YYYY-MM-DD). Create issues only for bugs with
|
||||
last_seen >= since; update already-synced ones whose
|
||||
last_seen or status_updated >= since.
|
||||
ids: set of bug ids to sync regardless of dates. Combined with
|
||||
since, a bug is synced if it matches either -- bulk triage
|
||||
has bumped last_seen/status_updated across most of the log,
|
||||
so dates alone cannot pick out recently fixed older bugs.
|
||||
since, a bug is synced if it matches either.
|
||||
"""
|
||||
if not BUGLOG_PATH.exists():
|
||||
warn(f"buglog not found: {BUGLOG_PATH}")
|
||||
records = buglog.load_all()
|
||||
if not records:
|
||||
warn(f"no bug records in {buglog.BUGS_DIR}")
|
||||
return
|
||||
|
||||
buglog = json.loads(BUGLOG_PATH.read_text())["bugs"]
|
||||
created = closed = reopened = updated = skipped = 0
|
||||
created = closed = reopened = updated = refreshed = skipped = 0
|
||||
needs_status = []
|
||||
|
||||
# Aggregate status per already-synced issue (see STATUS_PRIORITY).
|
||||
issue_status = {}
|
||||
for bug in buglog:
|
||||
num = bug.get("gitea_issue")
|
||||
for rec in records:
|
||||
num = rec.meta.get("gitea_issue")
|
||||
if num:
|
||||
prev = issue_status.get(num, "fixed")
|
||||
cur = bug_status(bug)
|
||||
issue_status[num] = min(prev, cur, key=STATUS_PRIORITY.index)
|
||||
issue_status[num] = min(prev, bug_status(rec), key=STATUS_PRIORITY.index)
|
||||
done_issues = set()
|
||||
|
||||
for bug in buglog:
|
||||
bid = bug.get("id")
|
||||
if not bid:
|
||||
continue
|
||||
for rec in records:
|
||||
m, bid = rec.meta, rec.id
|
||||
|
||||
# --since: new issues only for recently seen bugs; status changes
|
||||
# (status_updated) are still mirrored for already-synced ones.
|
||||
if since or ids:
|
||||
recent = False
|
||||
if since:
|
||||
recent = (bug.get("last_seen") or "") >= since
|
||||
if bug.get("gitea_issue"):
|
||||
recent = recent or (bug.get("status_updated") or "") >= since
|
||||
recent = (m.get("last_seen") or m.get("timestamp") or "") >= since
|
||||
if m.get("gitea_issue"):
|
||||
recent = recent or (m.get("status_updated") or "") >= since
|
||||
if ids and bid in ids:
|
||||
recent = True
|
||||
if not recent:
|
||||
skipped += 1
|
||||
continue
|
||||
|
||||
status = bug_status(bug)
|
||||
status = bug_status(rec)
|
||||
if status is None:
|
||||
needs_status.append(bid)
|
||||
synced_status = status or "unknown"
|
||||
state = bug_state(bug)
|
||||
issue_num = bug.get("gitea_issue")
|
||||
state = bug_state(rec)
|
||||
issue_num = m.get("gitea_issue")
|
||||
title, body = bug_title(rec), bug_body(rec)
|
||||
digest = body_hash(title, body)
|
||||
|
||||
if issue_num and bug.get("gitea_synced_status") == synced_status:
|
||||
if issue_num and m.get("gitea_synced_status") == synced_status \
|
||||
and m.get("gitea_synced_hash") == digest:
|
||||
skipped += 1
|
||||
continue
|
||||
|
||||
if issue_num:
|
||||
# Status changed since the last sync (or was never recorded:
|
||||
# issues synced before the status field existed were closed
|
||||
# whenever `fix` was non-empty, so re-assert state for them).
|
||||
# The issue itself follows the aggregate status of every entry
|
||||
# linked to it; only status labels are touched, so labels a
|
||||
# person put on the issue survive.
|
||||
# The issue follows the aggregate status of every record linked
|
||||
# to it; only status labels are touched, so labels a person put
|
||||
# on the issue survive.
|
||||
agg = issue_status[issue_num]
|
||||
agg_state = STATUS_MAP.get(agg, ("open", None))[0]
|
||||
text_changed = m.get("gitea_synced_hash") != digest
|
||||
if dry_run:
|
||||
if issue_num not in done_issues:
|
||||
print(f" [DRY-RUN] #{issue_num} ({bid}): -> {agg_state}"
|
||||
f" ({agg or 'unknown'})")
|
||||
done_issues.add(issue_num)
|
||||
if m.get("gitea_synced_status") != synced_status and issue_num not in done_issues:
|
||||
print(f" [DRY-RUN] #{issue_num} ({bid}): -> {agg_state} ({agg or 'unknown'})")
|
||||
done_issues.add(issue_num)
|
||||
if text_changed:
|
||||
print(f" [DRY-RUN] #{issue_num} ({bid}): refresh title/body")
|
||||
continue
|
||||
try:
|
||||
if issue_num not in done_issues:
|
||||
if text_changed:
|
||||
api.update_issue(issue_num, title=title, body=body)
|
||||
refreshed += 1
|
||||
if not quiet:
|
||||
print(f" #{issue_num} {bid}: refreshed title/body")
|
||||
if m.get("gitea_synced_status") != synced_status and issue_num not in done_issues:
|
||||
_, issue = request("GET", f"/issues/{issue_num}")
|
||||
if issue is None:
|
||||
warn(f" {bid}: issue #{issue_num} not found, skipping")
|
||||
|
|
@ -886,9 +882,11 @@ def sync_bugs(api, dry_run=False, quiet=False, since=None, ids=None):
|
|||
api.add_labels(issue_num, [labels[want]["id"]])
|
||||
if issue.get("state") != agg_state:
|
||||
api.update_issue(issue_num, state=agg_state)
|
||||
note = _text(bug.get("status_note"))
|
||||
note = _text(m.get("status_note"))
|
||||
commits = ", ".join(m.get("fix_commits") or [])
|
||||
api.add_comment(issue_num, f"{bid} status: **{synced_status}**"
|
||||
+ (f" — {note}" if note else ""))
|
||||
+ (f" — {note}" if note else "")
|
||||
+ (f" (fix commits: {commits})" if commits else ""))
|
||||
if agg_state == "closed":
|
||||
closed += 1
|
||||
else:
|
||||
|
|
@ -899,26 +897,27 @@ def sync_bugs(api, dry_run=False, quiet=False, since=None, ids=None):
|
|||
else:
|
||||
updated += 1
|
||||
done_issues.add(issue_num)
|
||||
set_fields(BUGLOG_PATH, bid, {"gitea_synced_status": synced_status})
|
||||
buglog.set_meta(bid, {"gitea_synced_status": synced_status,
|
||||
"gitea_synced_hash": digest})
|
||||
except (RuntimeError, BuglogEditError) as e:
|
||||
warn(f" {bid}: could not update issue #{issue_num}: {e}")
|
||||
skipped += 1
|
||||
continue
|
||||
|
||||
# New bug -- create issue
|
||||
title = bug_title(bug)
|
||||
if dry_run:
|
||||
print(f" [DRY-RUN] create {state.upper()} issue: {title}")
|
||||
created += 1
|
||||
continue
|
||||
|
||||
label_ids = _resolve_label_ids(api, bug_labels(bug))
|
||||
issue = api.create_issue(title, bug_body(bug), labels=label_ids)
|
||||
label_ids = _resolve_label_ids(api, bug_labels(rec))
|
||||
issue = api.create_issue(title, body, labels=label_ids)
|
||||
if state == "closed":
|
||||
# Gitea may ignore `state` on create; close explicitly.
|
||||
api.update_issue(issue["number"], state="closed")
|
||||
set_fields(BUGLOG_PATH, bid, {"gitea_issue": issue["number"],
|
||||
"gitea_synced_status": synced_status})
|
||||
buglog.set_meta(bid, {"gitea_issue": issue["number"],
|
||||
"gitea_synced_status": synced_status,
|
||||
"gitea_synced_hash": digest})
|
||||
if not quiet:
|
||||
mark = " (closed)" if state == "closed" else ""
|
||||
print(f" created #{issue['number']}: {title}{mark}")
|
||||
|
|
@ -926,7 +925,7 @@ def sync_bugs(api, dry_run=False, quiet=False, since=None, ids=None):
|
|||
|
||||
if not quiet:
|
||||
print(f"Bugs: {created} created, {closed} closed, {reopened} reopened, "
|
||||
f"{updated} relabelled, {skipped} skipped")
|
||||
f"{updated} relabelled, {refreshed} refreshed, {skipped} skipped")
|
||||
if needs_status:
|
||||
warn(f"{len(needs_status)} bug(s) have no status -- triage per "
|
||||
f".claude/rules/buglog-status.md: {' '.join(needs_status)}")
|
||||
|
|
|
|||
340
scripts/tests/test_buglog.py
Normal file
340
scripts/tests/test_buglog.py
Normal file
|
|
@ -0,0 +1,340 @@
|
|||
"""Tests for scripts/buglog.py (one Markdown record per bug).
|
||||
|
||||
Run: python3 -m unittest discover -s scripts/tests -v
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
import buglog # noqa: E402
|
||||
|
||||
|
||||
FULL_ENV = """\
|
||||
- OS: Ubuntu 24.04 LTS; kernel Linux 6.18.33 x86_64
|
||||
- libc: glibc 2.42 (from the binary's interpreter)
|
||||
- Toolchain: gcc 15.2.0, meson 1.9.1
|
||||
- Build: asan (xce.build)
|
||||
- Code: scheduler-framework @ {sha}
|
||||
- Deployment: single host, xcecd_virtual"""
|
||||
|
||||
FULL_BODY = """\
|
||||
# {id}: xcecd_virtual leaks worker strings on every finished job
|
||||
|
||||
## Description
|
||||
**Observed:**
|
||||
```
|
||||
Direct leak of 12 byte(s) in 12 object(s)
|
||||
```
|
||||
**Expected:** no leak at exit
|
||||
**Impact:** grows with job count
|
||||
**Found by:** LeakSanitizer after a job burst
|
||||
|
||||
## Environment
|
||||
{env}
|
||||
|
||||
## Reproduction
|
||||
1. `/xce-build worker`
|
||||
2. run `finished_job_retention_test`
|
||||
- Frequency: deterministic
|
||||
- Evidence: the unit test
|
||||
|
||||
## Root cause
|
||||
{root}
|
||||
|
||||
## Fix
|
||||
{fix}
|
||||
|
||||
## Verification
|
||||
{verif}
|
||||
|
||||
## References
|
||||
- Related: none
|
||||
"""
|
||||
|
||||
|
||||
def git(root, *args):
|
||||
return subprocess.run(["git", "-C", str(root), *args], check=True,
|
||||
capture_output=True, text=True).stdout.strip()
|
||||
|
||||
|
||||
class BuglogTestCase(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.root = Path(self.tmp.name)
|
||||
(self.root / ".wolf" / "bugs").mkdir(parents=True)
|
||||
git(self.root, "init", "-q")
|
||||
git(self.root, "-c", "user.email=t@t", "-c", "user.name=t",
|
||||
"commit", "-q", "--allow-empty", "-m", "init")
|
||||
self.sha = git(self.root, "rev-parse", "--short=8", "HEAD")
|
||||
buglog.configure(self.root)
|
||||
|
||||
def tearDown(self):
|
||||
self.tmp.cleanup()
|
||||
|
||||
def write_record(self, bug_id="bug-100", status="open", legacy=False,
|
||||
root="TODO", fix="TODO", verif="TODO", commits=None,
|
||||
env=None, extra_meta=None):
|
||||
meta = {"id": bug_id, "status": status, "status_updated": "2026-09-27",
|
||||
"status_note": "" if status == "fixed" else "why",
|
||||
"component": "worker", "severity": "leak",
|
||||
"timestamp": "2026-09-25T15:42:02Z"}
|
||||
if legacy:
|
||||
meta["legacy"] = True
|
||||
if commits is not None:
|
||||
meta["fix_commits"] = commits
|
||||
meta.update(extra_meta or {})
|
||||
body = FULL_BODY.format(id=bug_id, env=env if env is not None
|
||||
else FULL_ENV.format(sha=self.sha),
|
||||
root=root, fix=fix, verif=verif)
|
||||
text = buglog.render_meta(meta) + "\n" + body
|
||||
path = self.root / ".wolf" / "bugs" / f"{bug_id}.md"
|
||||
path.write_text(text, encoding="utf-8")
|
||||
return path
|
||||
|
||||
|
||||
class ParseRender(BuglogTestCase):
|
||||
def test_round_trip_keeps_every_field(self):
|
||||
path = self.write_record(extra_meta={"tags": ["worker", "asan"],
|
||||
"gitea_issue": 285})
|
||||
rec = buglog.load("bug-100")
|
||||
self.assertEqual(rec.meta["tags"], ["worker", "asan"])
|
||||
self.assertEqual(rec.meta["gitea_issue"], 285)
|
||||
self.assertEqual(rec.summary,
|
||||
"xcecd_virtual leaks worker strings on every finished job")
|
||||
self.assertIn("Direct leak", rec.sections["Description"])
|
||||
self.assertEqual(buglog.render(rec), path.read_text(encoding="utf-8"))
|
||||
|
||||
def test_section_header_inside_code_fence_is_not_a_section(self):
|
||||
self.write_record(root="See:\n```\n## Fix\nnot a header\n```")
|
||||
rec = buglog.load("bug-100")
|
||||
self.assertIn("not a header", rec.sections["Root cause"])
|
||||
self.assertEqual(rec.sections["Fix"], "TODO")
|
||||
|
||||
def test_escaped_section_header_round_trips(self):
|
||||
text = "a\n## Fix\nb"
|
||||
rec = buglog.Record({"id": "bug-7", "status": "open"}, "s",
|
||||
{"Description": buglog.escape_text(text)})
|
||||
again = buglog.parse(buglog.render(rec))
|
||||
self.assertEqual(buglog.unescape_text(again.sections["Description"]), text)
|
||||
self.assertNotIn("Fix", again.sections)
|
||||
|
||||
def test_set_meta_leaves_body_bytes_unchanged(self):
|
||||
path = self.write_record()
|
||||
body_before = path.read_text(encoding="utf-8").split("\n---\n", 1)[1]
|
||||
buglog.set_meta("bug-100", {"gitea_issue": 9, "status_note": "x: y"})
|
||||
text = path.read_text(encoding="utf-8")
|
||||
self.assertEqual(text.split("\n---\n", 1)[1], body_before)
|
||||
self.assertEqual(buglog.load("bug-100").meta["status_note"], "x: y")
|
||||
|
||||
|
||||
class Check(BuglogTestCase):
|
||||
def problems(self, bug_id="bug-100", target=None):
|
||||
return buglog.check(buglog.load(bug_id), target_status=target)
|
||||
|
||||
def test_complete_open_record_passes(self):
|
||||
self.write_record()
|
||||
self.assertEqual(self.problems(), [])
|
||||
|
||||
def test_environment_needs_every_item(self):
|
||||
env = FULL_ENV.format(sha=self.sha).replace(
|
||||
"- libc: glibc 2.42 (from the binary's interpreter)\n", "")
|
||||
self.write_record(env=env)
|
||||
self.assertTrue(any("libc" in p for p in self.problems()))
|
||||
|
||||
def test_libc_needs_a_version(self):
|
||||
env = FULL_ENV.format(sha=self.sha).replace("glibc 2.42", "glibc")
|
||||
self.write_record(env=env)
|
||||
self.assertTrue(any("libc" in p for p in self.problems()))
|
||||
|
||||
def test_code_needs_a_commit(self):
|
||||
env = FULL_ENV.format(sha=self.sha).replace(self.sha, "latest")
|
||||
self.write_record(env=env)
|
||||
self.assertTrue(any("Code" in p for p in self.problems()))
|
||||
|
||||
def test_todo_counts_as_missing(self):
|
||||
env = FULL_ENV.format(sha=self.sha).replace(
|
||||
"single host, xcecd_virtual", "TODO: topology")
|
||||
self.write_record(env=env)
|
||||
self.assertTrue(any("Deployment" in p for p in self.problems()))
|
||||
|
||||
def test_fixed_needs_root_cause_fix_verification_and_commit(self):
|
||||
self.write_record(status="fixed")
|
||||
probs = " | ".join(self.problems())
|
||||
for word in ("Root cause", "Fix", "Verification", "fix_commits"):
|
||||
self.assertIn(word, probs)
|
||||
|
||||
def test_fixed_with_everything_passes(self):
|
||||
self.write_record(status="fixed", root="remove_worker skips five strings",
|
||||
fix="free them", commits=[self.sha],
|
||||
verif="- Test: test_x: RED before, GREEN after\n"
|
||||
"- Suites: worker 12/12, asan")
|
||||
self.assertEqual(self.problems(), [])
|
||||
|
||||
def test_unknown_commit_is_rejected(self):
|
||||
self.write_record(status="fixed", root="r", fix="f", commits=["deadbeef"],
|
||||
verif="- Test: t: RED before, GREEN after\n- Suites: s")
|
||||
self.assertTrue(any("deadbeef" in p for p in self.problems()))
|
||||
|
||||
def test_open_status_needs_note(self):
|
||||
self.write_record(extra_meta={"status_note": ""})
|
||||
self.assertTrue(any("status_note" in p for p in self.problems()))
|
||||
|
||||
def test_legacy_record_is_exempt_until_it_changes_status(self):
|
||||
self.write_record(legacy=True, env=buglog.NOT_RECORDED)
|
||||
self.assertEqual(self.problems(), [])
|
||||
# marking it fixed now demands the fix-time sections
|
||||
probs = " | ".join(self.problems(target="fixed"))
|
||||
self.assertIn("Verification", probs)
|
||||
self.assertIn("fix_commits", probs)
|
||||
|
||||
def test_migrated_placeholder_is_not_verification(self):
|
||||
self.write_record(legacy=True, root="r", fix="f", commits=[self.sha],
|
||||
verif=buglog.NOT_RECORDED)
|
||||
self.assertTrue(any("Verification" in p
|
||||
for p in self.problems(target="fixed")))
|
||||
|
||||
|
||||
class Commands(BuglogTestCase):
|
||||
def run_cli(self, *argv):
|
||||
try:
|
||||
buglog.main(list(argv))
|
||||
return 0
|
||||
except SystemExit as e:
|
||||
return e.code
|
||||
|
||||
def test_status_refuses_incomplete_fixed_and_leaves_file_alone(self):
|
||||
path = self.write_record()
|
||||
before = path.read_text(encoding="utf-8")
|
||||
rc = self.run_cli("status", "bug-100", "fixed", "--commit", self.sha)
|
||||
self.assertNotEqual(rc, 0)
|
||||
self.assertEqual(path.read_text(encoding="utf-8"), before)
|
||||
|
||||
def test_status_fixed_records_commits(self):
|
||||
self.write_record(root="r", fix="f",
|
||||
verif="- Test: t: RED before, GREEN after\n- Suites: s")
|
||||
rc = self.run_cli("status", "bug-100", "fixed", "--commit", self.sha)
|
||||
self.assertIn(rc, (0, None))
|
||||
meta = buglog.load("bug-100").meta
|
||||
self.assertEqual(meta["status"], "fixed")
|
||||
self.assertEqual(meta["fix_commits"], [self.sha])
|
||||
|
||||
def test_new_creates_record_with_environment_filled(self):
|
||||
self.run_cli("new", "--summary", "scheduler hangs on empty queue",
|
||||
"--component", "scheduler", "--severity", "hang")
|
||||
rec = buglog.load("bug-001")
|
||||
self.assertEqual(rec.meta["status"], "open")
|
||||
env = rec.sections["Environment"]
|
||||
self.assertRegex(env, r"- OS: .*Linux")
|
||||
self.assertRegex(env, r"- Code: .*" + self.sha)
|
||||
# the author still has to write the description
|
||||
self.assertTrue(buglog.check(rec))
|
||||
|
||||
def test_next_id_counts_archive(self):
|
||||
self.write_record(bug_id="bug-100")
|
||||
(self.root / ".wolf" / "buglog-archive.json").write_text(
|
||||
json.dumps({"bugs": [{"id": "bug-900"}]}), encoding="utf-8")
|
||||
self.assertEqual(buglog.next_id(), "bug-901")
|
||||
|
||||
def test_index_keeps_openwolf_fields(self):
|
||||
self.write_record(extra_meta={"files": ["src/worker/a.c", "src/worker/b.c"],
|
||||
"tags": ["worker"]}, root="cause text")
|
||||
buglog.build_index()
|
||||
idx = json.loads((self.root / ".wolf" / "buglog.json").read_text())
|
||||
b = idx["bugs"][0]
|
||||
self.assertEqual(b["id"], "bug-100")
|
||||
self.assertTrue(b["error_message"].startswith("xcecd_virtual leaks"))
|
||||
self.assertIn("Direct leak", b["error_message"])
|
||||
self.assertEqual(b["root_cause"], "cause text")
|
||||
self.assertEqual(b["file"], "src/worker/a.c, src/worker/b.c")
|
||||
self.assertEqual(b["tags"], ["worker"])
|
||||
self.assertEqual(b["status"], "open")
|
||||
|
||||
def test_write_commands_refresh_the_index(self):
|
||||
self.write_record()
|
||||
self.run_cli("note", "bug-100", "seen again under load")
|
||||
idx = json.loads((self.root / ".wolf" / "buglog.json").read_text())
|
||||
self.assertIn("seen again under load", idx["bugs"][0]["error_message"])
|
||||
|
||||
|
||||
class Staged(BuglogTestCase):
|
||||
"""`check --staged`, which the pre-commit hook runs."""
|
||||
|
||||
def commit(self, *paths):
|
||||
git(self.root, "add", *paths)
|
||||
git(self.root, "-c", "user.email=t@t", "-c", "user.name=t",
|
||||
"commit", "-q", "-m", "x")
|
||||
|
||||
def check_staged(self):
|
||||
try:
|
||||
buglog.main(["check", "--staged"])
|
||||
return 0
|
||||
except SystemExit as e:
|
||||
return e.code or 0
|
||||
|
||||
def test_complete_record_with_fresh_index_passes(self):
|
||||
self.write_record()
|
||||
buglog.build_index()
|
||||
git(self.root, "add", ".wolf")
|
||||
self.assertEqual(self.check_staged(), 0)
|
||||
|
||||
def test_incomplete_record_is_refused(self):
|
||||
self.write_record(env="- OS: TODO")
|
||||
buglog.build_index()
|
||||
git(self.root, "add", ".wolf")
|
||||
self.assertNotEqual(self.check_staged(), 0)
|
||||
|
||||
def test_stale_index_is_refused(self):
|
||||
self.write_record()
|
||||
git(self.root, "add", ".wolf")
|
||||
self.assertNotEqual(self.check_staged(), 0)
|
||||
|
||||
def test_hand_edited_index_alone_is_refused(self):
|
||||
self.write_record()
|
||||
buglog.build_index()
|
||||
self.commit(".wolf")
|
||||
idx = self.root / ".wolf" / "buglog.json"
|
||||
idx.write_text(idx.read_text().replace("leaks", "LEAKS"))
|
||||
git(self.root, "add", ".wolf/buglog.json")
|
||||
self.assertNotEqual(self.check_staged(), 0)
|
||||
|
||||
def test_legacy_record_marked_fixed_by_hand_needs_fix_sections(self):
|
||||
self.write_record(legacy=True, env=buglog.NOT_RECORDED)
|
||||
buglog.build_index()
|
||||
self.commit(".wolf")
|
||||
buglog.set_meta("bug-100", {"status": "fixed", "status_note": ""})
|
||||
git(self.root, "add", ".wolf")
|
||||
self.assertNotEqual(self.check_staged(), 0)
|
||||
|
||||
def test_migrated_record_matching_the_old_index_passes(self):
|
||||
# the migration commit: the record is new, but HEAD's buglog.json
|
||||
# already had this bug with the same status
|
||||
(self.root / ".wolf" / "buglog.json").write_text(
|
||||
json.dumps({"bugs": [{"id": "bug-100", "status": "fixed"}]}))
|
||||
self.commit(".wolf")
|
||||
self.write_record(legacy=True, status="fixed", env=buglog.NOT_RECORDED)
|
||||
buglog.build_index()
|
||||
git(self.root, "add", ".wolf")
|
||||
self.assertEqual(self.check_staged(), 0)
|
||||
|
||||
def test_new_record_cannot_claim_legacy(self):
|
||||
self.write_record(legacy=True, status="fixed", env=buglog.NOT_RECORDED)
|
||||
buglog.build_index()
|
||||
git(self.root, "add", ".wolf")
|
||||
self.assertNotEqual(self.check_staged(), 0)
|
||||
|
||||
def test_untouched_legacy_record_passes(self):
|
||||
self.write_record(legacy=True, env=buglog.NOT_RECORDED)
|
||||
buglog.build_index()
|
||||
self.commit(".wolf")
|
||||
buglog.set_meta("bug-100", {"gitea_issue": 5})
|
||||
git(self.root, "add", ".wolf")
|
||||
self.assertEqual(self.check_staged(), 0)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Loading…
Add table
Add a link
Reference in a new issue