14 KiB
Requirements Document
Project Description (Input)
Who has the problem: operators and engineers who replay scheduler recordings with xce_scheduler --sim / --sim-verify or the simulation REPL, and the libxce-alloc-preason-equality work, which needs a replay to reproduce values derived from a session's start time bit-for-bit.
Current situation: the simulator is already event-driven: sim_loop.c never sleeps, and it jumps the virtual clock with sim_advance_virtual_clock() to each recorded event timestamp and to each recorded session's beginRound. --sim-speed (default 1, help text "1=real-time") does not pace anything. It only scales how far the clock drifts with real elapsed time between those jumps (xce_core_now() = virtual_clock_base + elapsed_real * speed_num / speed_den). So at the default speed, every clock reading in a replay (core->session_start, the beginRound written to simulation output (--sim-output FILE or the REPL set output FILE), decision timestamps, REPL position) depends on how fast the host happens to run. For example, a replayed session's session_start lands slightly after the recorded beginRound. Only --sim-speed 0 is deterministic. The REPL exposes the same knob as its speed parameter.
What should change: make the simulator a pure discrete-event simulation. Retire --sim-speed (see Decided below), remove the REPL speed parameter, and remove the simulator's use of speed_num/speed_den, so the virtual clock moves only when the replay reaches a recorded event or round timestamp and never drifts with real time. Guarantee that a replayed session's core->session_start equals the recorded ReplaySession beginRound exactly, so values derived from it (for example xce_alloc_t.from, spec libxce-alloc-preason-equality Requirement 4.3) reproduce under --sim-verify. Amend the scheduler-core spec wherever it describes speed modes or wall-clock pacing (Requirement 5.2 lists the places). Expected touch points: main.c CLI, sim.c sim_set_virtual_clock(), include/sim.h (sched_sim_args_t.sim_speed, the sim_speed_num/sim_speed_den shadow copy), sim_loop.c, repl.c (help text), repl_params.c/repl_info.c, core_clock.c/core.h, the 27 scheduler test files that set a speed, plus test_repl_help.c, which asserts the help text lists set speed <N>, and src/docs/deployment.md. Normal-mode (live) scheduling is unchanged.
Decided (2026-09-28): --sim-speed is kept for one release as an accepted, deprecated no-op that logs one warning, so existing scripts and command lines keep working; the REPL speed parameter is removed.
Introduction
The scheduler simulator (xce_scheduler --sim, with or without --sim-verify, --sim-output, --sim-alt-events, --interactive) replays a recording by moving its logical clock from one recorded timestamp to the next. The --sim-speed option and the REPL set speed command were specified (scheduler-core spec, Requirement 5's mode-selection flag table, AC 3/13/14 and the REPL parameter table) to pace a replay against the wall clock, but no pacing was ever implemented: the simulator never waits. Their only effect is that the logical clock drifts with real elapsed time between recorded timestamps, which makes every time value observed during a replay depend on host speed and load. Only speed 0 is deterministic, and every simulator test that runs a replay already uses it. This feature makes the simulator's logical clock purely event-driven in every run, keeps --sim-speed accepted as a deprecated option with no effect, removes the REPL speed parameter, and records the change in the documentation and in the scheduler-core spec.
Boundary Context
- In scope: the simulator's logical-clock behavior in simulation mode (batch replay,
--sim-verify, simulation output (--sim-output FILEor the REPLset output FILE),--sim-alt-events, and the interactive REPL); the--sim-speedcommand-line option becoming a deprecated no-op; removal of the REPLspeedparameter fromset,help, andinfo config; the operator documentation of simulation mode; the scheduler-core spec's description of simulation speed modes. - Out of scope: normal-mode (live, controller-connected) scheduling and its wall-clock-anchored clock; the
.replayfile format; pacing a replay against the wall clock (not provided today and not added here; a future feature that wants it must not change logical time); removing--sim-speedfrom the command line entirely (a later release); howfromis computed and compared (spec libxce-alloc-preason-equality); the decomposedscheduler-frameworkspec, which is kept only as discovery context and whose speed-mode text is superseded by scheduler-core. - Adjacent expectations: spec libxce-alloc-preason-equality relies on Requirements 1.2 and 1.4 (a replayed session starts at exactly the recorded
beginRound), and 1.3 for the case where that moves the clock backwards. Recordings made before this change remain replayable unchanged. Logical time no longer passes inside a replayed session, so a session's elapsed time reported by the simulator (SessionSummary.durationNsin simulation output) is zero, because the replay reaches no recorded timestamp while a session is in progress; this is expected, and is what speed 0 already reports today.
Requirements
Requirement 1: Event-driven logical clock in simulation mode
Objective: As an operator replaying a recording, I want the simulator's logical time to depend only on the recording, so that every replay of the same recording shows the same times and decisions on any host.
Acceptance Criteria
- When a simulation run starts, the scheduler simulator shall set its logical clock to the timestamp of the snapshot it starts from (the last snapshot in the file whose timestamp is at or before
--sim-start-ts, or the last snapshot in the file when--sim-start-tsis not given). - When the replay reads a recorded event (other than the end-of-stream
endSessionmarker) or a recorded session, the scheduler simulator shall set its logical clock to exactly that event's timestamp or that session'sbeginRound. - If a recorded event timestamp or
beginRoundis earlier than the current logical clock, the scheduler simulator shall still set its logical clock to exactly that recorded value, without clamping it and without treating it as an error. This happens with well-formed recordings (the normal-mode recorder stamps a drained event batch with a time read beforerecord startstamps the initial snapshot, so the first events after that snapshot can be earlier than it) as well as with a--sim-alt-eventsfile or recording whose timestamps are out of order. - When the scheduler simulator starts a replayed session, the scheduler simulator shall set the session's start time (
core->session_start) to exactly the recorded session'sbeginRound. - The scheduler simulator shall change its logical clock only as described in 1.1, 1.2 and 1.3, and shall keep it unchanged at all other times, however much real time passes: before the first recorded event, between recorded events and sessions, when it skips a mid-replay snapshot, after the replay reaches the end of the recording, and while the interactive REPL waits for operator input.
- When the same recording is replayed twice with the same options and, for interactive runs, the same sequence of REPL commands, the scheduler simulator shall produce identical logical timestamps in both runs (session start times;
ReplayDecision.ts,SessionSummary.durationNs, snapshot timestamps andReplayHeader.recordingStartTsin simulation output; and the REPL's reported position) and identical decision sequences, regardless of host speed or load.ReplayHeader.recordingStartWall, a wall-clock field, is excluded. - Where simulation output (
--sim-output FILEor the REPLset output FILE) is enabled, the scheduler simulator shall write each session it writes to that output withbeginRoundequal to that session's start time. - Where simulation output (
--sim-output FILEor the REPLset output FILE) is enabled, the scheduler simulator shall write each session it writes to that output withsummary.durationNsas 0 and each of itsReplayDecision.tsvalues equal to that session'sbeginRound.
Requirement 2: --sim-speed accepted as a deprecated no-op
Objective: As an operator with existing scripts that pass --sim-speed, I want those command lines to keep working, so that the change does not break my tooling while I remove the option.
Acceptance Criteria
- When
xce_scheduleris started with--sim-speedand any value, in simulation mode or in normal mode, the scheduler shall accept the option and start normally. - When
xce_scheduleris started with--sim-speed, the scheduler shall behave exactly as it would without the option, including the logical-clock behavior of Requirement 1; the only difference shall be the warning of 2.3. - When
xce_scheduleris started with--sim-speed, in simulation mode or in normal mode, the scheduler shall log exactly one warning at startup stating that--sim-speedis deprecated, has no effect, and will be removed in a future release, and naming the value it received (the last value, if the option is given more than once). - If the value given to
--sim-speedis not a valid number (including a non-numeric, negative, or empty value), the scheduler shall still start normally and log the same single warning, instead of rejecting the command line. (--sim-speedas the last command-line argument, with no value after it, remains a command-line error, as for any option that takes a value.) - When
xce_scheduleris started without--sim-speed, the scheduler shall not log the deprecation warning. - The scheduler's command-line help shall describe
--sim-speedas deprecated with no effect, and shall not describe any speed or real-time replay mode.
Requirement 3: REPL speed parameter removed
Objective: As an operator using the interactive REPL, I want the REPL to stop offering a speed setting that does nothing, so that its commands and output reflect how the simulator actually behaves.
Acceptance Criteria
- When the operator enters
set speed, with or without an argument, the REPL shall change no simulation state and shall print a message stating that the speed parameter was removed because the simulator's clock moves only at recorded timestamps. - When the REPL has handled
set speed, the REPL shall return to its prompt as it does afterhelp, and shall not end the simulation or stop reading further commands, whether they are typed or piped in. - The REPL's
helpoutput and itssetusage message shall not list aspeedparameter. - When the operator enters
info config, the REPL shall show the remaining configuration fields (output, verify, events processed, current timestamp, sessions) and shall not show a speed field (the[sim] config:line loses itsspeed=item). - The REPL's other
setparameters (output,verify,units) shall behave as before this change.
Requirement 4: Normal mode and recordings unchanged
Objective: As an operator running a live cluster, I want this change confined to simulation, so that live scheduling and existing recordings are unaffected.
Acceptance Criteria
- While the scheduler runs in normal mode, the scheduler shall keep its existing wall-clock-anchored time behavior unchanged.
- When the scheduler simulator replays a recording made before this change, the scheduler simulator shall read and replay it without error.
- The scheduler shall keep the
.replayfile format unchanged, so recordings made after this change remain readable by the same tools as before.
Requirement 5: Documentation and specification alignment
Objective: As an engineer reading the docs or the scheduler-core spec, I want them to describe the simulator as it actually behaves, so that nobody relies on a pacing mode that does not exist.
Acceptance Criteria
- The operator documentation (
src/docs/deployment.md) shall describe the simulator's logical clock as moving only at recorded timestamps, shall describe--sim-speedas a deprecated option with no effect, shall not describe any pacing difference between REPL and batch runs, and shall not use--sim-speedin its example command lines. - The scheduler-core specification shall record that simulation speed modes and wall-clock pacing are not provided, that
--sim-speedis a deprecated no-op, and that the REPLspeedparameter was removed. It shall replace the speed-mode text in Requirement 5's mode-selection flag table, AC 3 (clock-virtualization mode table), AC 12 (simulation-mode session-duration bullet), AC 13 (simulation startup), AC 14 (the--sim-speedbullet, and its "strictly monotonically increasing" clause restated to match Requirement 1.3 of this spec), AC 22 (REPL parameter table, the speed step of the replay pseudocode, the "Why interactive parameters matter" rationale, and the out-of-scopesetnote), and thesim_speed_num, sim_speed_denreal-time/fast comment in its Data Structure Changes Summary. It shall likewise replace the speed-mode text in the scheduler-core design (the 5.14 traceability row, thesim_speed_num/sim_speed_densim-driver fields, the clock-virtualization test's mode rows, and the "Simulation fast mode" note). - The scheduler shall let in-repository callers that pass
--sim-speed(for example the HA soak script's--sim-verify --sim-speed=0step,.claude/skills/xce-test-ha/run-ha-soak.sh) run and pass without modification, with the deprecation warning in their logs.