---
id: "bug-1881"
status: "fixed"
status_updated: "2026-09-29"
status_note: ""
component: "scheduler"
severity: "wrong-result"
timestamp: "2026-09-28T07:26:09Z"
last_seen: "2026-09-28T07:26:09Z"
occurrences: 1
files: ["src/scheduler/src/sim_verify.c"]
tags: ["scheduler", "sim", "sim-verify", "logging"]
related: []
fix_commits: ["24b05718", "c1960457", "2d7ba660", "4e57bde6"]
gitea_issue: 332
gitea_synced_status: "fixed"
gitea_synced_hash: "e7258ca9149c"
---

# bug-1881: sim_log_mismatch prints identical expected/actual, hiding the field that differs

## Description
**Observed:**
```
... MISMATCH ... -- expected: DISPATCH jobid=0-12-1790578108-0[49] nhosts=1 -- actual: DISPATCH jobid=0-12-1790578108-0[49] nhosts=1
```
**Expected:** the message names the field(s) that differ, with both values.
**Impact:** a mismatch cannot be diagnosed from the log (bug-1880: 43 such lines, none showing the difference).
**Found by:** user reading the bug-1880 replay log, 2026-09-28

UPDATE (2026-09-28): Fix planned in spec libxce-alloc-preason-equality (Requirement 3, Requirement 5.3, task 5): the libxce equality functions gain an optional why/whylen buffer describing the first difference, and sim_log_mismatch appends it to the MISMATCH line.

## Environment
- OS: NixOS 26.05 (Yarara); NixOS kernel; kernel Linux 6.18.33 x86_64
- libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2)
- Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
- Build: debug (build.dbg)
- Code: scheduler-framework @ 4c134127 (`xce_scheduler --version`: v0.0.0.1-4c134127-85e8fc00, built 2026-09-28T06:40:21Z)
- Deployment: pc13v: xce_scheduler in simulation mode (pid 93231) replaying a capture of the live pc13v cluster's scheduler (scheduler-replay-20260928-064842.replay, captured from 06:48:42); exact command line and --sim-start-ts not recorded

## Reproduction
1. Any --sim-verify mismatch whose decisions differ only below the jobid/nhosts level (bug-1880's replay)
- Frequency: deterministic (43/43 lines in the pc13v run)
- Evidence: data/xce_scheduler.log

## Root cause
sim_log_mismatch() (src/scheduler/src/sim_verify.c) summarizes each decision as kind + jobid + nhosts, while sim_decision_eq() also compares every host's allocations (resname, value, duration, reserve), so a difference there prints two identical summaries.

## Fix
Spec libxce-alloc-preason-equality (commits 8f25c1c3..dcf3acf3). sim_decision_eq() now compares DISPATCH decisions with libxce's public job_alloc_eq() and returns a caller-owned first-difference description (`char** why`) naming the field path and both values; sim_log_mismatch() appends it as ` -- diff: <why>`, and the replay loop in sim_loop.c passes and frees it (fixed text "no live decision" for leftover recorded decisions) — 24b05718. job_alloc_eq() also compares `from`, so the scheduler now stamps every allocation's `from` with the deciding session's start (scheduler.c sched_make_primary_alloc_, c1960457) and the normal-mode recorder takes beginRound from the same reading (driver.c, 2d7ba660), making `from` reproducible on replay.

The remaining gap (the simulator's own session-begin clock read drifting at --sim-speed >= 1) was closed by spec scheduler-sim-discrete-clock (4f653d8f..d9bc36cd: the simulator clock is pinned to recorded timestamps, --sim-speed is a no-op), so a replayed session starts at exactly the recorded beginRound and every replayed `from` matches; task 5.3 adds the proof (4e57bde6).

## Verification
- Test: test_log_mismatch_appends_provided_description and test_verify_against_altered_copy_reports_exact_mismatch (asserts ` -- diff: main_reason: 99 != 0`) plus test_verify_leftover_recorded_decision_names_missing_side in src/scheduler/testsuite/test_sim_verify.c: RED before 24b05718 (no diff text on the MISMATCH line), GREEN after. test_sim_replay_reproduces_recorded_alloc_from in src/scheduler/testsuite/test_sim_live_allocation.c (4e57bde6): GREEN at default settings; a +0.5 s clock mutant at sim_loop.c's sim_advance_virtual_clock(sim, begin_round) makes it FAIL
- Suites: ASAN debug build (src/xce.build.discrete), `meson test --no-suite xceadmin`: 117/117 OK, 0 Unity :FAIL lines, 0 sanitizer reports (2026-09-29)
- Live: 2-round sequential HA soak (/home/claude/tmp/xce-alloceq-53, PASS, zero job loss) recorded 3 clean .replay files with 11 DISPATCH decisions; each replayed with `xce_scheduler --sim --sim-verify` and no --sim-speed (ASAN_OPTIONS=detect_leaks=0): rc=0, 0 MISMATCH lines

## References
- Related: bug-1880