myxce/.wolf/bugs/bug-1747.md
Rongsong Shen d26ff8870d
feat(buglog): one Markdown record per bug, checked against a record standard
The buglog was one 27K-line JSON file whose fields covered only symptom,
root cause and fix, as free text: 2% of entries named the code version a
bug was found on, 12% the build profile, 36% a way to reproduce, and 58% of
the file was OpenWolf hook auto-detections ("Missing guard clause",
"Function not marked async" on .md files).

- .claude/skills/buglog/bug-record.md: the record standard. Eight
  sections (Summary as the title, Description, Environment, Reproduction;
  Root cause, Fix, Verification, References once fixed), each with the
  content it must contain -- e.g. Environment needs the OS, the libc the
  failing binary loads (from its ELF interpreter: glibc 2.42 in the nix
  shell vs 2.40 on the host), toolchain, build profile and commit.
- .wolf/bugs/<id>.md: 500 records migrated from buglog.json; every field
  verified to round-trip. They carry legacy: true and an explicit "not
  recorded" where the old log had no such field; component derived from
  file paths, severity from keywords (else unknown), fix_commits from
  hashes in status_note/fix that git resolves (91 of 463 fixed).
- .wolf/buglog-archive.json: the 710 hook auto-detections, verbatim.
- .wolf/buglog.json is now an index generated from the records, keeping
  the fields OpenWolf's hooks and `openwolf bug search` read.
- scripts/buglog.py rewritten: new (template + Environment collected from
  the machine and --binary), check, status (refuses a status the record
  doesn't support; fixed needs Root cause, Fix, Verification and
  --commit hashes git resolves), note, recur, set, show, list, triage,
  env, index. 29 unittest cases in scripts/tests/test_buglog.py.
- scripts/git-hooks/pre-commit runs `check --staged`: incomplete staged
  records, a legacy record hand-changed to fixed without the fix-time
  sections, a new record claiming legacy: true, or a stale index refuse
  the commit.
- scripts/gitea-sync.py reads the records: issue body = the record's
  sections plus linked fix commits (origin remote, GITEA_COMMIT_REPO to
  override), and a body hash so an issue's text is refreshed when the
  record changes.
- bug-1866 rewritten to the full standard as the worked example.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 03:40:11 +00:00

5.5 KiB

id status status_updated status_note component severity timestamp last_seen occurrences files tags related fix_commits legacy
bug-1747 fixed 2026-09-26 FINISH half eec2596c; FAIL half 1d6c04c2: WAL_JOB_FAIL trailer carries finish_time, record_job_fail_fact_at() for resync. controller unknown 2026-09-01T00:00:00Z 2026-09-19T09:00:00Z 1
src/controller/job.c
worker-rejoin-resync
task-3.2
resync-c
job-c
requirement-4.2
finish-time
wal-job-fail
wal-job-finish
apply_job_fail_transition
pre-existing
predates-feature
systemic
resync_apply_report
record_job_finish_fact_at
bug-1689
bug-1678
bug-1746
bug-1631
eec2596c
1d6c04c2
true

bug-1747: systemic gap predating worker-rejoin-resync entirely, discovered while implementing task 3.2…

Description

Pre-existing, systemic gap predating worker-rejoin-resync entirely, discovered while implementing task 3.2 (resync_apply_report, resync.c): WAL_JOB_FAIL carries no fail-time/finish-time slot in its wire payload at ANY layer, and apply_job_fail_transition() (job.c) unconditionally stamps job.finish_time = time(NULL) at APPLY time (not even receipt time) for EVERY fail-reporting caller in this codebase -- not just reconciliation. This means Requirement 4.2 ('the Controller shall record the finish time the Worker reported ... rather than the time the Controller processed the reconciliation') is currently unsatisfiable for a reconciled finished-FAILURE outcome via any existing path. UPDATE (2026-09-01, coordinator-directed re-scope): the sibling FINISH-success half of this gap is FIXED -- see 'fix' field below; only the FAIL half remains open.

Environment

Not recorded (migrated from buglog.json).

Reproduction

Not recorded (migrated from buglog.json).

Root cause

job_failinfo_t (xce/types.h) is {jobid, stage, code, detail} -- no finish_time field, and WAL_JOB_FAIL's durable payload (record_job_fail_fact(), job.c: persist_submit_job_fact_ex(&c->persist, WAL_JOB_FAIL, jobid, 0, WAL_JOB_FAIL_STAGE_FLAG|stage)) carries no finish-time slot at all -- confirmed by direct inspection of the payload construction. apply_job_fail_transition() (job.c, the guarded core BOTH the live job_ctrl path (mark_job_fail) and the deferred apply seam (apply_committed_entry's WAL_JOB_FAIL case) invoke) unconditionally executes job->job.finish_time = time(NULL); with no external value ever consulted, at APPLY time -- meaning even a caller willing to plumb a real value through record_job_fail_fact's WAL payload would still need a apply_job_fail_transition() signature change to have it actually land in job.job.finish_time. This is a pre-existing, already-documented residual: job.c's own comment near apply_job_postdone_transition (task 4.1) already states 'finish_time is not part of the wal_jobstate_t wire format ... has no apply-time equivalent' for a sibling fact type, confirming this is systemic to the wal_jobstate_t-based fact family, not something worker-rejoin-resync introduced. Fixing it for real would require extending WAL_JOB_FAIL's wire payload (wal_types.h, mirroring bug-1689's own WAL_JOB_FINISH trailing-fields precedent) PLUS an apply_job_fail_transition() signature/behavior change reaching every fail-reporting call site in the system (job_ctrl's mark_job_fail, on_job_fail's ordinary worker-fail receipt path, and this feature's own reconciliation caller) -- real, broad-blast-radius work, not a narrow patch.

Fix

SPLIT (2026-09-01, coordinator-directed): the FINISH-success half of this gap is FIXED as part of worker-rejoin-resync task 3.2 -- added record_job_finish_fact_at(cluster_t*, job_t*, xce_jobfinish_t*, uint64_t finish_time_ms) (job.c), record_job_finish_fact()'s own worker-reported-finish-time sibling (identical guard/hook-fire/durable-append body, differing only in passing the caller-supplied finish_time -- converted ms->s -- to persist_submit_job_finish_fact() instead of time(NULL)); record_job_finish_fact() itself is UNMODIFIED, its own caller (on_job_finish, an ordinary near-instantaneous worker report) correctly keeps using receipt-time. resync_apply_report()'s finishedOk branch now calls record_job_finish_fact_at(c, job, &finish, entry->finish_time_ms). Verified via a real restart-replay test (test_apply_report_finished_success_uses_worker_reported_finish_time, test_worker_resync.c): a finished-success entry with finish_time_ms set to 'now minus 1 day' recovers job.finish_time equal to that worker-reported value after persist_recover(), not reconciliation-processing wall-clock time. This was possible narrowly for FINISH because persist_submit_job_finish_fact() already accepted an explicit finish_time parameter at the primitive level -- record_job_finish_fact()'s specific call site just never exposed it. The FAIL half (see root_cause) is NOT YET FIXED and is NOT task 3.2's (or any single narrow task's) job to fix -- it needs the broader WAL_JOB_FAIL payload extension + apply_job_fail_transition() signature change described above, deliberately deferred pending a dedicated task, matching the same 'deliberately deferred, documented, not blocking' treatment bug-1631's own sibling gap received earlier in this same feature. The reconciled job's STATE (EXIT) is always correct for a finished-failure outcome either way -- only its recorded finish_time reflects apply-processing time rather than the worker's true report.

Verification

Not recorded (migrated from buglog.json).

References

  • Related: bug-1689, bug-1678, bug-1746, bug-1631