Commit graph

3 commits

Author SHA1 Message Date
lei.chen
f3e510a548 test(scheduler): G-5 JOB_RUSAGE dequeue routing — routes to sched_apply_job_rusage
Closes the G-5 test gap mirroring the G-8 routing test: the rusage notify
handler enqueues XCE_SCHED_OP_JOB_RUSAGE and sched_update() must dispatch it to
sched_apply_job_rusage(), storing the report on the matching job (bug-765 G-5
remediation, production fix already in 93124c5). Driving the real sched_update()
dequeue loop with a recording stub, asserts the request reaches
sched_apply_job_rusage and the job's rusages field is set.

11/11 sched_redispatch_test PASS; full scheduler suite 6/6 OK. Live: job ran to
DONE on the deployed scheduler (G-3 leak fix in place), zero ASAN errors.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-14 02:52:09 +00:00
lei.chen
200303d8fe test(scheduler): G-8 dequeue routing — UPDATE_JOB_FIELDS → in-place, bulk → no damage
Closes the msg-gap G-8 test gap the research flagged ("handler→dequeue ROUTING
is verified by code review, NOT an automated integration test"). The
remediation (UPDATE_JOB_FIELDS op routed to sched_update_existing_job) is
already in the code (93124c5); these tests lock the routing:

- test_update_fields_routes_to_inplace_path: driving the real sched_update()
  dequeue loop, an UPDATE_JOB_FIELDS request reaches sched_update_existing_job
  and mutates the existing job in place (same object, runtime state kept).
- test_bulk_update_existing_job_no_damage: a re-pushed dataset row for an
  existing jobid is EEXIST-dropped by key_add (via the real sched_add_job) —
  the live job keeps identity/state/runtime state.

10/10 sched_redispatch_test PASS; full scheduler suite 6/6 OK. Live cluster:
job PEND→RUN→DONE via notify updates, no WARN, no duplicate instance.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-14 02:08:49 +00:00
lei.chen
ee7dee0ed4 fix(scheduler): stop re-dispatching running jobs (bug-1498) + resval double-free (bug-1497)
bug-1498 (re-dispatch): sched_job_is_ready() was a stub returning 1, so any
job lingering in the fcfs pendJobs queue was always elected. A dispatched job
was never marked non-PEND (the controller defers its schedStartJob notify to
the not-yet-implemented apply pump, controller/job.c), so fcfs_update_job
re-enqueued it on every job update and the election thread re-dispatched the
running job — the worker got conflicting signals and the job was SIGTERM'd.

- sched_job_is_ready: elect only PEND, non-suspended jobs
- sched_commit_decision (dispatch): mark the job JOB_STAT_RUN on dispatch so
  it leaves the PEND set and is no longer re-electable
- fcfs_update_job: dedup via ordq_peek before enqueue

bug-1497 (double-free): sched_update's resval paths transferred ownership of
rv->val to sharedInst/w->resvals but left the request pointer set, so
sched_pend_update_killer double-freed it -> use-after-free in
build_host_eval_ctx. Clear rv->val = NULL on transfer.

Tests: new scheduler/testsuite/test_redispatch.c (8 cases): sched_job_is_ready
PEND gating + dispatch-commit marks the job RUN. Full scheduler meson suite
green. Live-verified: a `sleep 60` job is elected exactly once, runs the full
duration and exits 0 (`completed successfully`); before the fix it was
re-dispatched at ~11s and killed (SIGTERM 143).

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-04 01:20:38 +00:00