fix(scheduler): recorder silently truncated a round's decisions on realloc failure (bug-1843)

sched_record_session_add_decision() (recording_session.c) grows its
per-round decisions array via realloc(), doubling from 8. On a failed
growth it silently dropped the current decision and every one after it
that round, without marking the round incomplete -- finalize() then
wrote the short array as an ordinary, apparently-complete ReplaySession.
--sim-verify's replay comparison had no way to distinguish that from a
genuinely smaller live round, producing spurious "expected: <none>"
mismatches for jobs that were, in reality, correctly decided live and
simply never recorded. Live scheduling itself was unaffected either
way -- only the .replay file's own record of that one round lost its
tail.

This refutes the leading theory from the prior investigation (replay
seeing a job's first appearance "one round early" via coarser event
batching in sim_loop.c) -- confirmed via targeted diagnostic
instrumentation against the original captured .replay file: replay
deterministically reproduces the full candidate set every round;
the recording itself has the one-round gap.

Fix: track truncation on the session ctx; finalize() now refuses to
write an incomplete session and fails loudly through the same
sched_record_abort_on_error() path every other recording failure in
this subsystem already uses.

TDD: added a regression test forcing a real, deterministic realloc()
failure via -Wl,--wrap=realloc (scoped to this one test binary --
no prior malloc/realloc fault-injection harness existed in this
codebase). Verified genuine red -> green: the test fails without the
fix, passes with it. Full 85/85 scheduler suite green.

Note: this prevents future recordings from losing decisions this way;
it cannot retroactively repair an already-captured .replay file.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-09-23 08:37:42 +00:00
parent 88b370ee84
commit 5ee329b1b8
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
5 changed files with 224 additions and 18 deletions

View file

@ -520,6 +520,18 @@ sched_record_lifecycle_test = executable(
['test_record_lifecycle.c'],
include_directories:[incdir],
dependencies: [scheduler_test_deps],
# bug-1843 regression: `-Wl,--wrap=realloc` linker symbol interposition
# (same technique/idiom as scheduler_decision_publish_retry_test's own
# xcecli_* wraps above), so
# test_decision_batch_growth_failure_drops_incomplete_session() can force
# a REAL, deterministic realloc() failure inside
# sched_record_session_add_decision()'s own decisions-array growth --
# this codebase has no other malloc/realloc fault-injection harness, and
# bug-1843's root cause was exactly this growth-failure path silently
# dropping decisions instead of aborting the recording. __wrap_realloc()
# passes through to __real_realloc() for every call except the one this
# test arms, so the other 12 tests in this same binary are unaffected.
link_args: ['-Wl,--wrap=realloc'],
install: false)
# Contract test for the scheduler-side OPCode_schedAdmin request handler