--- id: "bug-1772" status: "fixed" status_updated: "2026-09-24" status_note: "" component: "scheduler" severity: "crash" timestamp: "2026-09-05T00:00:00Z" last_seen: "2026-09-23T10:05:00Z" occurrences: 1 files: [] tags: ["scheduler", "snapshot-restore", "resreq-fingerprint", "cache", "pre-existing", "schedobj-duplication", "severity-medium", "tdd", "audit-reopened"] related: ["bug-1767", "bug-1770"] legacy: true gitea_issue: 52 gitea_synced_status: "fixed" --- # bug-1772: out-of-scope gap discovered during requirements scoping for the resreq-maxtasks-wiring spec… ## Description OPEN, NOT A CRASH -- pre-existing, out-of-scope gap discovered during requirements scoping for the resreq-maxtasks-wiring spec (bug-1770's fix). snapshot_build_primary_schedobj() (src/scheduler/src/snapshot_load.c:81-96) never computes primary->resreq_fp at all, unlike core_build_primary_schedobj() (src/scheduler/src/core_events.c:130-135), which sets resreq_fp.min_tasks/max_tasks/select_node_id/rusage_hash from the parsed xce_primary_resreq_t immediately after parsing. Every job reconstructed from a snapshot (post-restart or post-failover restore) therefore gets a zeroed resreq_fp -- the same class of gap as bug-1767's broader 'recurring schedobj-reconstruction-duplication' pattern (snapshot_load.c independently re-implements core_events.c's schedobj construction rather than sharing it, and each independent copy has repeatedly drifted field-by-field). TRIGGER CORRECTION (2026-09-05, resreq-maxtasks-wiring design review): this entry's original phrasing ('post-restart or post-failover restore') overstates the live trigger. xce_sched_core_load_snapshot()'s ONLY real (non-test) caller anywhere in src/ is sim.c's sim_locate_and_load_snapshot(), reachable only via main.c's --sim FILE offline replay flag. Normal-mode main.c has no snapshot-load call or flag at all; a live scheduler restart (including post-failover, e.g. run-ha-soak.sh's restart_scheduler_for_new_primary()) starts with empty in-memory state and rejoins via on_scheduler_join() -> the same core_build_job() path fresh ingest uses -- never touching snapshot_load.c. The accurate trigger is: a job in a .replay-recorded snapshot loaded by the --sim simulation/replay driver, not any live-cluster crash-recovery or failover scenario. This does not change the bug's validity or fix (the gap is real for --sim/replay fidelity, which matters per this project's driver-equivalence invariant since --sim drives the identical allocator dispatch) -- it only corrects when it's actually reachable, which matters for scoping test coverage (an HA soak does NOT exercise this path at all). ## Environment _Not recorded (migrated from buglog.json)._ ## Reproduction _Not recorded (migrated from buglog.json)._ ## Root cause snapshot_load.c's snapshot_build_primary_schedobj() is a hand-duplicated, field-for-field port of core_events.c's core_build_primary_schedobj() (kept independent because core_build_primary_schedobj() is static and the two files are compiled as separate objects -- see bug-1767's own root-cause text for the deliberate-duplication rationale). The port was written before task 1.2 (scheduler-allocation feature) added resreq_fp/hash_rusages() wiring to the core_events.c original, and the snapshot_load.c copy was never updated to match. Effect: a snapshot-restored job's unsatisfiable-resreq cache fingerprint (used by scheduler-allocation's alloc_cache.c) is always the zero value, which is a valid-looking but WRONG fingerprint -- it could produce a false cache hit against an unrelated job that also happens to have a zeroed fingerprint (extremely unlikely today since real jobs always populate resreq_fp correctly at first ingest, but a snapshot-restored job's own fingerprint stays wrong for its entire post-restore lifetime, and any code that later relies on comparing/keying by resreq_fp for a restored job silently gets 0/0/0/0). Discovered while scoping resreq-maxtasks-wiring (bug-1770's fix spec) because that fix will make resreq_fp carry real, meaningful, non-degenerate maxTasks values for the first time -- increasing the practical consequence of this gap versus its current all-zero-everywhere baseline. ## Fix FIXED (2026-09-23, TDD, /xce-audit-invariants scheduler dimension-4 audit -- independently re-found by both a dedicated subagent and direct investigation into bug-1845; this entry's own prior 'fixed'/'closed-by-scheduler-sim-live-allocation' tags were STALE/WRONG -- direct code inspection confirmed snapshot_build_primary_schedobj() still had no resreq_fp computation immediately before this fix). Added the same four-field resreq_fp computation core_build_primary_schedobj() (core_events.c) performs -- min_tasks/max_tasks/select_node_id/rusage_hash -- to snapshot_build_primary_schedobj() (snapshot_load.c), matching that function's own documented 'field-for-field port' contract, which had simply never included this field. Added test_snapshot_load_computes_resreq_fingerprint_for_restored_primary() (test_snapshot_save_load.c) -- a job restored from a snapshot with mintasks=4/maxtasks=6 must have schedobj->u.primary->resreq_fp.min_tasks/max_tasks reflect those real values, not {0,0,0,0}. Verified genuine red->green: temporarily reverted just the new resreq_fp assignment and reran -- scheduler snapshot save load test FAILED (exit 1); restored the fix and reran -- full 85/85 scheduler suite green. ## Verification _Not recorded (migrated from buglog.json)._ ## References - Related: bug-1767, bug-1770 - Gitea: #52