--- id: "bug-1844" status: "fixed" status_updated: "2026-09-24" status_note: "" component: "scheduler" severity: "unknown" timestamp: "2026-09-23T09:15:00Z" last_seen: "2026-09-23T09:15:00Z" occurrences: 1 files: ["src/scheduler/src/sim_verify.c (sim_verify_alloc_eq())"] tags: ["scheduler", "sim", "sim-verify", "replay", "instid", "nextid", "monotonic-counter", "non-deterministic-field", "tdd"] related: ["bug-1842", "bug-1838-followup", "bug-1843", "bug-1845"] legacy: true gitea_issue: 81 gitea_synced_status: "fixed" --- # bug-1844: User-reported follow-up to bug-1843: replaying a fresh capture… ## Description User-reported follow-up to bug-1843: replaying a fresh capture (scheduler-replay-20260923-085356.replay, built after bug-1843's fix) with --sim --sim-verify still shows a huge volume of MISMATCH lines (4881 in one run) on DISPATCH decisions whose printed description is IDENTICAL on both sides -- e.g. 'expected: DISPATCH jobid=...[40] nhosts=1 -- actual: DISPATCH jobid=...[40] nhosts=1' still flagged as a mismatch. ## Environment _Not recorded (migrated from buglog.json)._ ## Reproduction _Not recorded (migrated from buglog.json)._ ## Root cause CONFIRMED via temporary field-by-field diagnostic instrumentation in sim_verify_alloc_eq() (added, verified, then reverted -- not part of the fix) replayed against the real captured .replay file: `instid` was the ONLY field ever differing (value/duration/resname/reserve matched exactly every single time, e.g. recorded instid=826 vs replay instid=1, both otherwise byte-identical). `instid` is stamped from `sched->core.nextid++` (scheduler.c's sched_get_id()) at allocation-build time -- a bare per-process monotonic counter tracking how many allocation instances THIS core has built over its own entire execution history, not a semantic property of what got allocated. The ORIGINAL recording's counter (running, and building allocations, for potentially far longer before this capture window started) and a fresh replay's own counter (starting from whatever its loaded snapshot's nextid happens to be) have no reason to reach the same value for 'the same' dispatch. Exact same class of bug as bug-1838-followup/bug-1842's `from` field (also non-deterministic bookkeeping, already excluded) -- this comparator simply had a second field of the same kind left in. ## Fix FIXED (2026-09-23, TDD). Dropped the `a->instid == b->instid` clause from sim_verify_alloc_eq() (sim_verify.c), same treatment as bug-1842's `from` exclusion. Updated that function's doc comment to document the exclusion and cite the real captured-file evidence (826 vs 1). Added test_dispatch_decisions_differing_instid_still_match() (test_sim_verify.c), mirroring the existing test_dispatch_decisions_differing_from_timestamp_still_match() pattern -- two allocations identical in every field except instid (826 vs 1, matching the real observed divergence) must still compare equal. Verified genuine red->green: temporarily re-added the instid comparison and reran -- scheduler sim verify test FAILED (exit 1); restored the fix and reran -- full 85/85 scheduler suite green. NOTE: this fix does NOT address the second, separate anomaly found in the same replay file during this investigation (see bug-1845): a genuine one-round-early DISPATCH timing difference for one job (a resource-accounting/capacity divergence between live and replay, not a field-comparison issue) -- open, tracked separately. ## Verification _Not recorded (migrated from buglog.json)._ ## References - Related: bug-1842, bug-1838-followup, bug-1843, bug-1845 - Gitea: #81