feat(scheduler-core): implement scheduler-side admin request handler (task 9.4)

Implements the scheduler's first request-reply opcode (OPCode_schedAdmin).
sched_handle_admin_request() (handler.c) decodes SchedAdminRequest and
dispatches by action: recordStatus answers synchronously (pure read);
recordStart/recordStop are deferred onto the existing pend_updates queue
(XCE_SCHED_OP_ADMIN_RECORD) since sched_record_start()'s snapshot call
requires the session_active==0 precondition only guaranteed on the driver
thread, not the async request-handling thread. driver.c's
xce_sched_driver_translate_pending_events() performs the deferred action
and sends the reply via a new xcecli_reply_schedadmin() (client.c/client.h),
correlated by seq copied before the original request is freed.

Adversarial review found a genuine cross-thread race: sched_record_status()
read driver-owned recording state (including a pointer the driver thread
frees) from the vtask thread with no lock -- the codebase's first such
race, missed by ASan (doesn't detect races). Fixed with a dedicated
record_lock scoped strictly to in-memory field access, never spanning file
I/O; sched_record_status() now strdup()s under the lock instead of
returning a borrowed live pointer. Verified with a real two-thread
lock-contention test (independently confirmed non-vacuous by removing the
lock and observing the test fail) plus a deterministic use-after-free
reproduction.

26/26 scheduler suite, clang-format clean.
This commit is contained in:
Rongsong Shen 2026-08-05 14:11:43 +00:00
parent 7494542a60
commit e819a50600
Signed by: rshen
SSH key fingerprint: SHA256:8Vf2pZ3h6rJa7EhGK+WaAQ9Mta/apZvteW6g9U/WKKk
19 changed files with 1761 additions and 115 deletions

View file

@ -170,6 +170,12 @@ extern int sched_validate_record_filename(const char* record_dir,
*
* `record_file` may be NULL or empty to auto-generate a timestamped name
* (`scheduler-replay-<UTC timestamp>.replay`) beneath `sched->record_dir`.
*
* Concurrency (task 9.4 fix round): step 4's field commit takes
* `sched->record_lock` (scheduler.h's field comments) -- see
* sched_record_status()'s doc comment below for the cross-thread reader
* this protects against. The lock is held only around the five assignments
* themselves, never across steps 1-3's file I/O.
*/
extern int sched_record_start(xce_scheduler_t* sched, const char* record_file);
@ -192,6 +198,13 @@ extern int sched_record_start(xce_scheduler_t* sched, const char* record_file);
* marker failed to write (state is still fully cleared either way, per the
* error-handling clause above -- the negative return is diagnostic only,
* not a signal that recording is still active).
*
* Concurrency (task 9.4 fix round): the fflush()+fclose()+state-clear
* sequence above takes `sched->record_lock` (scheduler.h's field
* comments) around the state-clear portion only -- see
* sched_record_status()'s doc comment below for the cross-thread reader
* this protects against. In particular, freeing `record_file_path` and
* setting it NULL happens atomically from that reader's point of view.
*/
extern int sched_record_stop(xce_scheduler_t* sched);
@ -225,6 +238,13 @@ extern int sched_record_stop(xce_scheduler_t* sched);
* rationale. A checkpoint write failure disables recording via
* sched_record_abort_on_error(), the exact same path as any other
* recording write failure -- never special-cased.
*
* Concurrency (task 9.4 fix round): the per-event increment of
* `sched->record_event_count` takes `sched->record_lock` (scheduler.h's
* field comments), scoped to just the increment -- see
* sched_record_status()'s doc comment below for the cross-thread reader
* this protects against. `record_snapshot_interval` is driver-thread-only
* and not protected by this lock (scheduler.h's field comment).
*/
extern int sched_record_write_events(xce_scheduler_t* sched,
xce_core_event_t* events, int n);
@ -283,4 +303,48 @@ extern int sched_record_session_end(xce_scheduler_t* sched,
sched_record_session_ctx_t* ctx,
uint64_t end_ts);
/*
* sched_record_status - Req 5 AC 10 (scheduler-core spec, task 9.4,
* "Reply field semantics by action" -- the `recordStatus` column). A pure
* read of `sched`'s CURRENT recording-state fields into a
* `sched_admin_reply_t` the caller (handler.c's scheduler_req_handler())
* sends back over the wire immediately -- no file I/O, no state mutation,
* and (unlike sched_record_start()/sched_record_stop()) no session_active
* dependency, so it is safe to answer directly from the request-handling
* thread rather than deferring through the driver-thread pend_updates
* queue the way recordStart/recordStop must.
*
* Concurrency (task 9.4 fix round): this function is called from a vtask
* worker pthread (handler.c's scheduler_req_handler(), invoked with no
* lock held -- lib/xcectx.c), which runs concurrently with the
* single-threaded driver loop's sched_record_start()/sched_record_stop()/
* sched_record_write_events()/sched_record_abort_on_error() -- all of
* which mutate the same fields this function reads, with no other
* synchronization. This function therefore takes `sched->record_lock`
* (scheduler.h's field comments) around every read of those fields,
* INCLUDING the copy of `record_file_path` described next.
*
* `reply->file` is a fresh, caller-owned heap copy of
* `sched->record_file_path` (via strdup(), taken while still holding
* `sched->record_lock` -- copying the bytes before the lock is released is
* what makes this safe), NOT a borrowed alias of the live pointer. Unlike
* every other capnp encode call in this codebase, this reply crosses a
* function-return boundary before it is ever encoded, so a borrowed
* pointer into `sched`'s live, mutable state cannot be handed back safely
* -- the driver thread could free() it the instant `record_lock` is
* released, well before the caller's eventual xcecli_reply_schedadmin()
* call. The caller MUST free(reply->file) once it is done (NULL when not
* recording -- nothing to free).
*
* Returns 0 on success, populating every field of `*reply`
* (`reply->ok = true` unconditionally). Returns -EINVAL if `sched` or
* `reply` is NULL; `*reply` is left untouched in that case. Returns
* -ENOMEM if the internal strdup() of `record_file_path` fails; `*reply`
* is left zeroed (via the initial memset()) in that case -- this is the
* one way reading status can now itself fail, a direct consequence of no
* longer handing back a borrowed pointer.
*/
extern int sched_record_status(xce_scheduler_t* sched,
sched_admin_reply_t* reply);
#endif