---
id: "bug-1845"
status: "fixed"
status_updated: "2026-09-24"
status_note: ""
component: "scheduler"
severity: "unknown"
timestamp: "2026-09-23T10:00:00Z"
last_seen: "2026-09-23T10:00:00Z"
occurrences: 1
files: ["src/scheduler/src/sim_verify.c (sim_verify_job_preason_eq())"]
tags: ["scheduler", "sim", "sim-verify", "replay", "pend-reason", "unsatisfiable-cache", "alloc_cache", "simulation-isolation", "tdd"]
related: ["bug-1843", "bug-1844", "bug-1842", "bug-1846"]
legacy: true
gitea_issue: 83
gitea_synced_status: "fixed"
---

# bug-1845: Follow-up to bug-1843/bug-1844: replaying a fresh capture…

## Description
Follow-up to bug-1843/bug-1844: replaying a fresh capture (scheduler-replay-20260923-085356.replay) with --sim --sim-verify still showed a large volume of MISMATCH lines on PEND decisions where recorded and replay agreed on jobid/main_reason but the deep host/resource reason-detail comparison disagreed (e.g. recorded {main_reason=0, nhost_reasons=0, nres_reasons=0} vs replay {main_reason=0, nhost_reasons=0, nres_reasons=2}), live-inspected via gdb by the user.

## Environment
_Not recorded (migrated from buglog.json)._

## Reproduction
_Not recorded (migrated from buglog.json)._

## Root cause
CONFIRMED (2026-09-23) as a documented, intentional architectural asymmetry, not a defect: core.h states outright that 'a simulation-mode session never touches the real cache implementation at all' (simulation isolation). core->unsatisfiable_lookup_cb/unsatisfiable_populate_cb (alloc_cache.c's xce_alloc_cache_lookup_cb/populate_cb) are registered only by main.c's normal-mode branch -- sim_loop.c never registers them. sched_make_primary_alloc_() (scheduler.c) consults that cache FIRST; on a hit it records reason detail via sched_pend_from_cached_reason_(), which sets ONLY main_reason (no host/resource detail at all -- skipping that expensive per-host computation is the entire point of the cache). Replay, having no cache, always falls through to the full per-host computation (sched_try_alloc_on_host_(), resource.c) and therefore always produces full detail. Two independent, architecturally-legitimate detail levels for the exact same semantic outcome (this job is PEND, for this main reason), not a decision correctness signal. Separately verified the cache mechanism itself is SOUND (not a distinct bug): its key (alloc_cache.c's build_key_(): resreq_fp + user + queue + project, no host/capacity dimension) is safe within a round because capacity only decreases monotonically as earlier jobs dispatch and the cache is cleared fresh every round (xce_alloc_cache_clear_cb() at begin_session) -- a cached 'unsatisfiable' verdict from an earlier job this round remains valid for a later job sharing the same key.

## Fix
FIXED (2026-09-23, TDD). sim_verify_job_preason_eq() (sim_verify.c) now compares ONLY main_reason, dropping the nhost_reasons/nres_reasons/host_based_reasons/res_based_reasons deep comparison entirely (that deep check was added as an optional 'cheap to add' enhancement, never proven necessary by a dedicated test, and is provably wrong given the cache-isolation asymmetry above). Doc comment updated to explain why. Added test_pend_decisions_differing_reason_detail_still_match() (test_sim_verify.c) -- two PEND decisions with the same jobid/main_reason but one with empty (cache-hit-shaped) and one with populated (full-computation-shaped) host/resource reason detail must still compare equal. Verified genuine red->green: temporarily re-added the deep comparison and reran -- scheduler sim verify test FAILED; restored the fix and reran -- full 85/85 scheduler suite green.

## Verification
_Not recorded (migrated from buglog.json)._

## References
- Related: bug-1843, bug-1844, bug-1842, bug-1846
- Gitea: #83