Rongsong Shen
d26ff8870d
The buglog was one 27K-line JSON file whose fields covered only symptom,
root cause and fix, as free text: 2% of entries named the code version a
bug was found on, 12% the build profile, 36% a way to reproduce, and 58% of
the file was OpenWolf hook auto-detections ("Missing guard clause",
"Function not marked async" on .md files).
- .claude/skills/buglog/bug-record.md: the record standard. Eight
sections (Summary as the title, Description, Environment, Reproduction;
Root cause, Fix, Verification, References once fixed), each with the
content it must contain -- e.g. Environment needs the OS, the libc the
failing binary loads (from its ELF interpreter: glibc 2.42 in the nix
shell vs 2.40 on the host), toolchain, build profile and commit.
- .wolf/bugs/<id>.md: 500 records migrated from buglog.json; every field
verified to round-trip. They carry legacy: true and an explicit "not
recorded" where the old log had no such field; component derived from
file paths, severity from keywords (else unknown), fix_commits from
hashes in status_note/fix that git resolves (91 of 463 fixed).
- .wolf/buglog-archive.json: the 710 hook auto-detections, verbatim.
- .wolf/buglog.json is now an index generated from the records, keeping
the fields OpenWolf's hooks and `openwolf bug search` read.
- scripts/buglog.py rewritten: new (template + Environment collected from
the machine and --binary), check, status (refuses a status the record
doesn't support; fixed needs Root cause, Fix, Verification and
--commit hashes git resolves), note, recur, set, show, list, triage,
env, index. 29 unittest cases in scripts/tests/test_buglog.py.
- scripts/git-hooks/pre-commit runs `check --staged`: incomplete staged
records, a legacy record hand-changed to fixed without the fix-time
sections, a new record claiming legacy: true, or a stale index refuse
the commit.
- scripts/gitea-sync.py reads the records: issue body = the record's
sections plus linked fix commits (origin remote, GITEA_COMMIT_REPO to
override), and a body hash so an issue's text is refreshed when the
record changes.
- bug-1866 rewritten to the full standard as the worked example.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
6.6 KiB
| id | status | status_updated | status_note | component | severity | timestamp | last_seen | occurrences | files | tags | related | legacy | |||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| bug-1759 | fixed | 2026-09-24 | scheduler | unknown | 2026-09-02T03:47:00Z | 2026-09-19T09:00:00Z | 1 |
|
|
|
true |
bug-1759: Live simulation run (xce_scheduler --sim, user-observed, not reproduced by me in this session)…
Description
Live simulation run (xce_scheduler --sim, user-observed, not reproduced by me in this session) shows a JOBS_PUSH batch of 50 jobs where the first 22 (indices [0]-[21]) receive [sim] DISPATCH decisions, immediately followed by [sim] PEND decisions for ALL 50 jobs (indices [0]-[49]) -- including the 22 that had JUST been dispatched. Design expectation (confirmed correct, see root_cause): a job that has received a DISPATCH decision must be treated as RUN immediately, without waiting for any downstream job-start acknowledgment, and must never be reconsidered as a fresh scheduling candidate afterward -- in BOTH the live scheduler and simulation/replay mode.
Environment
Not recorded (migrated from buglog.json).
Reproduction
Not recorded (migrated from buglog.json).
Root cause
ROOT-CAUSED FOR REAL (2026-09-07), and turns out to be a SPEC CONFLICT, not a logic bug -- a patch was attempted and REVERTED after it broke a pre-existing, passing regression test. Confirmed mechanism: sim mode's REAL DISPATCH decisions come entirely from sim_reconcile_unaccounted_jobs() (sim_loop.c), which reconstructs a job's dispatch PURELY from replay data (populating job->allocs/job->accounts, reporting a synthesized DISPATCH decision) but never calls picker->picker_iface->on_dispatch() -- unlike the real/live allocator path (sched_allocator(), scheduler.c line 507-510), which ALWAYS calls on_dispatch() to retire the job from the picker's own candidate list (fcfs_queue_t.pendJobs) before building the decision. Because reconciliation skips this, the job stays a live candidate in the picker's bookkeeping. sim_picker_thread_fn() (sim_loop.c) -- a SEPARATE, always-running thread that walks next_job() every round and unconditionally PENDs whatever it is still offered, since it has no real resource-matching allocation of its own -- keeps re-offering and re-PENDing that job every subsequent round until an explicit JOB_STATE_CHANGE-to-RUN event (also replayed from the original recording, can legitimately arrive several rounds later) removes it from pendJobs via builtin_fcfs_on_job_state_change(). CRITICAL DISCOVERY: this exact behavior is NOT accidental -- it is DOCUMENTED, INTENTIONAL, and covered by an existing passing test, test_sim_verify_no_false_mismatch_across_round_separated_dispatch (test_sim_verify.c, scheduler-sim-eager-reconciliation spec task 3.2), whose own header comment states verbatim: 'sim_reconcile_unaccounted_jobs() captures job->accounts but never calls picker_iface->on_dispatch() to retire the job from the picker's own internal pendJobs (design.md's Non-Goals/Revalidation Triggers explicitly leave live-picker DISPATCH emission to a future scheduler-allocation spec)'. That spec's own design.md (.kiro/specs/scheduler-sim-eager-reconciliation/design.md, Non-Goals section) explicitly lists 'Any change to simulation mode's live picker (still PEND-only for freshly-arriving jobs, per scheduler-allocation)' as OUT OF SCOPE. A fix attempt (calling on_dispatch() from the reconciliation FOUND branch, mirroring sched_allocator()'s exact ordering) was implemented with a new TDD test, confirmed the bug reproduces deterministically (RED), and confirmed the fix resolves it (GREEN) -- but broke test_sim_verify_no_false_mismatch_across_round_separated_dispatch, whose fixture explicitly bakes in 'PEND(jobid) recorded in the SAME round as its own DISPATCH, and again for every round until the RUN confirmation arrives' as the CORRECT, expected --sim-verify-clean shape. The fix was REVERTED (both the sim_loop.c change and its new test) rather than kept, since it silently violates an already-agreed, tested design contract from a sibling spec.
Fix
FIXED FOR REAL (2026-09-07), after a first fix attempt was correctly reverted (see prior root_cause note) upon discovering it broke test_sim_verify_no_false_mismatch_across_round_separated_dispatch. That test's OWN fixture turned out to be baking in the bug's exact symptom as 'expected' -- but its parent spec's own design.md ('Integration with --sim-verify') already documented the CORRECT assumption ('the picker never re-decides an already-resolved job'), which the implementation had silently drifted from. This was a genuine implementation/test drift from an already-agreed design intent, not a live spec conflict requiring a product decision -- confirmed by re-reading both scheduler-sim-decision-parity's and scheduler-sim-eager-reconciliation's own design.md files (both updated in place with 2026-09-07 correction notes explaining this). Fix: sim_reconcile_unaccounted_jobs() (sim_loop.c) now resolves the job picker and calls picker_iface->on_dispatch() immediately after reconstructing job->allocs/job->accounts in its FOUND branch, mirroring sched_allocator()'s own live-dispatch ordering (scheduler.c line 507-515) exactly. This retires the reconciled job from the picker's own pendJobs bookkeeping the moment its dispatch is reconstructed, so sim_picker_thread_fn()'s unconditional-PEND thread never re-offers/re-PENDs it on a later round. TDD: new test test_reconcile_removes_reconciled_job_from_picker_candidates (test_sim_reconcile_unaccounted.c) directly proves a job is no longer offered by next_job() after reconciliation, using sim_bootstrap_picker() to register a real, resolvable builtin_fcfs instance -- confirmed RED before the fix, GREEN after. test_sim_verify_no_false_mismatch_across_round_separated_dispatch (test_sim_verify.c) was updated to match the corrected behavior: round 1 now records DISPATCH-only (new helper append_replay_session_with_dispatch(), no accompanying PEND), and rounds 2-3 are now empty ReplaySessions (matching round 4's own existing pattern) instead of baking in a stale re-PEND echo as expected. Full scheduler suite: 76/77 pass (same pre-existing unrelated test_core_events.c failure, bug-1762, untouched by this fix).
Verification
Not recorded (migrated from buglog.json).
References
- Related: bug-1567, bug-1623