myxce/.wolf/bugs/bug-1726.md
Rongsong Shen d26ff8870d
feat(buglog): one Markdown record per bug, checked against a record standard
The buglog was one 27K-line JSON file whose fields covered only symptom,
root cause and fix, as free text: 2% of entries named the code version a
bug was found on, 12% the build profile, 36% a way to reproduce, and 58% of
the file was OpenWolf hook auto-detections ("Missing guard clause",
"Function not marked async" on .md files).

- .claude/skills/buglog/bug-record.md: the record standard. Eight
  sections (Summary as the title, Description, Environment, Reproduction;
  Root cause, Fix, Verification, References once fixed), each with the
  content it must contain -- e.g. Environment needs the OS, the libc the
  failing binary loads (from its ELF interpreter: glibc 2.42 in the nix
  shell vs 2.40 on the host), toolchain, build profile and commit.
- .wolf/bugs/<id>.md: 500 records migrated from buglog.json; every field
  verified to round-trip. They carry legacy: true and an explicit "not
  recorded" where the old log had no such field; component derived from
  file paths, severity from keywords (else unknown), fix_commits from
  hashes in status_note/fix that git resolves (91 of 463 fixed).
- .wolf/buglog-archive.json: the 710 hook auto-detections, verbatim.
- .wolf/buglog.json is now an index generated from the records, keeping
  the fields OpenWolf's hooks and `openwolf bug search` read.
- scripts/buglog.py rewritten: new (template + Environment collected from
  the machine and --binary), check, status (refuses a status the record
  doesn't support; fixed needs Root cause, Fix, Verification and
  --commit hashes git resolves), note, recur, set, show, list, triage,
  env, index. 29 unittest cases in scripts/tests/test_buglog.py.
- scripts/git-hooks/pre-commit runs `check --staged`: incomplete staged
  records, a legacy record hand-changed to fixed without the fix-time
  sections, a new record claiming legacy: true, or a stale index refuse
  the commit.
- scripts/gitea-sync.py reads the records: issue body = the record's
  sections plus linked fix commits (origin remote, GITEA_COMMIT_REPO to
  override), and a body hash so an issue's text is refreshed when the
  record changes.
- bug-1866 rewritten to the full standard as the worked example.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 03:40:11 +00:00

3.5 KiB

id status status_updated status_note component severity timestamp last_seen occurrences files tags related repro_attempts legacy
bug-1726 cannot-reproduce 2026-09-24 single occurrence, no surviving logs xceadmin unknown 2026-08-28T08:15:00Z 2026-09-19T09:00:00Z 1
src/xceadmin/tests/test_e2e_live_ha_push.py (live_ha_cluster fixture
_wait_for_primary)
flaky-test
ha
raft-election
live-ha-push
unresolved
unreproduced
low-priority
distinct-from-bug-1485
found-live
bug-1485
bug-1486
5 immediate retries, 0 reproduced true

bug-1726: test_e2e_live_ha_push.py::test_live_config_push_against_standby_redirects_and_succeeds ERRORed at…

Description

test_e2e_live_ha_push.py::test_live_config_push_against_standby_redirects_and_succeeds ERRORed at live_ha_cluster fixture setup (single occurrence, user-run via nix develop ./src + direct pytest against the built build.dbg mirror, not via any Claude-driven build): _wait_for_primary(nodes, 30.0) raised TimeoutError -- none of the 3 freshly-spawned xce_controller processes ever printed the CTRL_PRIMARY election-complete marker within the 30s timeout (normal election completes ~10.2s per this file's own docstring, keyed to HA_DEFAULT_HEARTBEAT_TIMEOUT_MS=10000).

Environment

Not recorded (migrated from buglog.json).

Reproduction

  • Attempts: 5 immediate retries, 0 reproduced

Root cause

NOT ISOLATED -- deliberately distinct from bug-1485 (do not merge into it): bug-1485's signature is election COMPLETING (all 3 markers print) then the freshly-promoted primary going /unreachable shortly after; this occurrence is election never visibly settling AT ALL within 3x the normal budget, a different symptom with a different plausible cause. XCE_KEEP_DEPLOY_DIR was not set for this run, so the 3 nodes' stdout/stderr logs from the actual failure were deleted by the fixture's own cleanup before this could be forensically investigated -- no diagnostic evidence survives from this specific occurrence. Immediate reruns (5/5, by Claude directly after the report, same build.dbg binaries, same test, quiet system: load avg ~0.6-1.5/22 cores, no concurrent xce_controller/meson/nix processes found) all passed cleanly in ~14-15s each, well inside the 30s budget -- consistent with (but not proof of) transient CPU/IO contention from another concurrent session on this shared multi-agent sandbox (the established pattern documented at length in bug-1486 and this same file's bug-1485 investigation) delaying all 3 nodes' heartbeat timers past the window, rather than a deterministic protocol liveness bug. Not ruled out: a genuine, rare election-liveness edge case (e.g. a PreVote/RequestVote split-vote condition) that happens not to leave a marker before some other transient condition times it out.

Fix

NOT FIXED -- single occurrence, unreproduced on 5 immediate retries, no surviving log evidence to root-cause further. Logged per this project's own established convention (bug-1485) of tracking an inconclusive live-cluster-election flake rather than silently dismissing it. Next step if this recurs: re-run with XCE_KEEP_DEPLOY_DIR=1 (already supported by the fixture, added for exactly this class of investigation per bug-1485's own remediation) to preserve deploy_dir/logs/*.log and *.stderr.log for real forensic inspection instead of losing them to the fixture's own cleanup.

Verification

Not recorded (migrated from buglog.json).

References

  • Related: bug-1485, bug-1486