feat(scheduler-core): implement scheduler-side admin request handler (task 9.4)
Implements the scheduler's first request-reply opcode (OPCode_schedAdmin). sched_handle_admin_request() (handler.c) decodes SchedAdminRequest and dispatches by action: recordStatus answers synchronously (pure read); recordStart/recordStop are deferred onto the existing pend_updates queue (XCE_SCHED_OP_ADMIN_RECORD) since sched_record_start()'s snapshot call requires the session_active==0 precondition only guaranteed on the driver thread, not the async request-handling thread. driver.c's xce_sched_driver_translate_pending_events() performs the deferred action and sends the reply via a new xcecli_reply_schedadmin() (client.c/client.h), correlated by seq copied before the original request is freed. Adversarial review found a genuine cross-thread race: sched_record_status() read driver-owned recording state (including a pointer the driver thread frees) from the vtask thread with no lock -- the codebase's first such race, missed by ASan (doesn't detect races). Fixed with a dedicated record_lock scoped strictly to in-memory field access, never spanning file I/O; sched_record_status() now strdup()s under the lock instead of returning a borrowed live pointer. Verified with a real two-thread lock-contention test (independently confirmed non-vacuous by removing the lock and observing the test fail) plus a deterministic use-after-free reproduction. 26/26 scheduler suite, clang-format clean.
This commit is contained in:
parent
7494542a60
commit
e819a50600
19 changed files with 1761 additions and 115 deletions
|
|
@ -170,6 +170,12 @@ extern int sched_validate_record_filename(const char* record_dir,
|
|||
*
|
||||
* `record_file` may be NULL or empty to auto-generate a timestamped name
|
||||
* (`scheduler-replay-<UTC timestamp>.replay`) beneath `sched->record_dir`.
|
||||
*
|
||||
* Concurrency (task 9.4 fix round): step 4's field commit takes
|
||||
* `sched->record_lock` (scheduler.h's field comments) -- see
|
||||
* sched_record_status()'s doc comment below for the cross-thread reader
|
||||
* this protects against. The lock is held only around the five assignments
|
||||
* themselves, never across steps 1-3's file I/O.
|
||||
*/
|
||||
extern int sched_record_start(xce_scheduler_t* sched, const char* record_file);
|
||||
|
||||
|
|
@ -192,6 +198,13 @@ extern int sched_record_start(xce_scheduler_t* sched, const char* record_file);
|
|||
* marker failed to write (state is still fully cleared either way, per the
|
||||
* error-handling clause above -- the negative return is diagnostic only,
|
||||
* not a signal that recording is still active).
|
||||
*
|
||||
* Concurrency (task 9.4 fix round): the fflush()+fclose()+state-clear
|
||||
* sequence above takes `sched->record_lock` (scheduler.h's field
|
||||
* comments) around the state-clear portion only -- see
|
||||
* sched_record_status()'s doc comment below for the cross-thread reader
|
||||
* this protects against. In particular, freeing `record_file_path` and
|
||||
* setting it NULL happens atomically from that reader's point of view.
|
||||
*/
|
||||
extern int sched_record_stop(xce_scheduler_t* sched);
|
||||
|
||||
|
|
@ -225,6 +238,13 @@ extern int sched_record_stop(xce_scheduler_t* sched);
|
|||
* rationale. A checkpoint write failure disables recording via
|
||||
* sched_record_abort_on_error(), the exact same path as any other
|
||||
* recording write failure -- never special-cased.
|
||||
*
|
||||
* Concurrency (task 9.4 fix round): the per-event increment of
|
||||
* `sched->record_event_count` takes `sched->record_lock` (scheduler.h's
|
||||
* field comments), scoped to just the increment -- see
|
||||
* sched_record_status()'s doc comment below for the cross-thread reader
|
||||
* this protects against. `record_snapshot_interval` is driver-thread-only
|
||||
* and not protected by this lock (scheduler.h's field comment).
|
||||
*/
|
||||
extern int sched_record_write_events(xce_scheduler_t* sched,
|
||||
xce_core_event_t* events, int n);
|
||||
|
|
@ -283,4 +303,48 @@ extern int sched_record_session_end(xce_scheduler_t* sched,
|
|||
sched_record_session_ctx_t* ctx,
|
||||
uint64_t end_ts);
|
||||
|
||||
/*
|
||||
* sched_record_status - Req 5 AC 10 (scheduler-core spec, task 9.4,
|
||||
* "Reply field semantics by action" -- the `recordStatus` column). A pure
|
||||
* read of `sched`'s CURRENT recording-state fields into a
|
||||
* `sched_admin_reply_t` the caller (handler.c's scheduler_req_handler())
|
||||
* sends back over the wire immediately -- no file I/O, no state mutation,
|
||||
* and (unlike sched_record_start()/sched_record_stop()) no session_active
|
||||
* dependency, so it is safe to answer directly from the request-handling
|
||||
* thread rather than deferring through the driver-thread pend_updates
|
||||
* queue the way recordStart/recordStop must.
|
||||
*
|
||||
* Concurrency (task 9.4 fix round): this function is called from a vtask
|
||||
* worker pthread (handler.c's scheduler_req_handler(), invoked with no
|
||||
* lock held -- lib/xcectx.c), which runs concurrently with the
|
||||
* single-threaded driver loop's sched_record_start()/sched_record_stop()/
|
||||
* sched_record_write_events()/sched_record_abort_on_error() -- all of
|
||||
* which mutate the same fields this function reads, with no other
|
||||
* synchronization. This function therefore takes `sched->record_lock`
|
||||
* (scheduler.h's field comments) around every read of those fields,
|
||||
* INCLUDING the copy of `record_file_path` described next.
|
||||
*
|
||||
* `reply->file` is a fresh, caller-owned heap copy of
|
||||
* `sched->record_file_path` (via strdup(), taken while still holding
|
||||
* `sched->record_lock` -- copying the bytes before the lock is released is
|
||||
* what makes this safe), NOT a borrowed alias of the live pointer. Unlike
|
||||
* every other capnp encode call in this codebase, this reply crosses a
|
||||
* function-return boundary before it is ever encoded, so a borrowed
|
||||
* pointer into `sched`'s live, mutable state cannot be handed back safely
|
||||
* -- the driver thread could free() it the instant `record_lock` is
|
||||
* released, well before the caller's eventual xcecli_reply_schedadmin()
|
||||
* call. The caller MUST free(reply->file) once it is done (NULL when not
|
||||
* recording -- nothing to free).
|
||||
*
|
||||
* Returns 0 on success, populating every field of `*reply`
|
||||
* (`reply->ok = true` unconditionally). Returns -EINVAL if `sched` or
|
||||
* `reply` is NULL; `*reply` is left untouched in that case. Returns
|
||||
* -ENOMEM if the internal strdup() of `record_file_path` fails; `*reply`
|
||||
* is left zeroed (via the initial memset()) in that case -- this is the
|
||||
* one way reading status can now itself fail, a direct consequence of no
|
||||
* longer handing back a borrowed pointer.
|
||||
*/
|
||||
extern int sched_record_status(xce_scheduler_t* sched,
|
||||
sched_admin_reply_t* reply);
|
||||
|
||||
#endif
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue