--- id: "bug-1624" status: "fixed" status_updated: "2026-09-24" status_note: "" component: "scheduler" severity: "unknown" timestamp: "2026-08-22T19:53:00Z" last_seen: "2026-09-19T09:00:00Z" occurrences: 2 files: ["src/scheduler/src/scheduler.c", "src/scheduler/src/core_session.c", "src/scheduler/src/resource.c"] tags: ["scheduler", "resource-accounting", "over-commit", "double-booking", "normal-mode", "sched_account_jobs", "JOB_IS_RUN", "race-condition"] related: ["bug-1611", "bug-1613"] fix_commits: ["8607348c", "d9aedc4b"] legacy: true --- # bug-1624: None yet observed at runtime -- found via code trace while answering an architectural question… ## Description None yet observed at runtime -- found via code trace while answering an architectural question about job RUN-state boundaries and over/under-commit safety. ## Environment _Not recorded (migrated from buglog.json)._ ## Reproduction _Not recorded (migrated from buglog.json)._ ## Root cause sched_account_jobs() (scheduler.c:1258-1292) re-derives core->sharedInUse/worker->resInUse every round from scratch, but gates on JOB_IS_RUN(job->state) before checking job->accounts. A job that has been DISPATCHED (job->accounts set synchronously at commit-decision time via sched_commit_job_alloc(), scheduler.c:1446) but not yet confirmed RUN by the worker is still job->state==PEND, so it is skipped by this resum. Since xce_core_begin_session() (core_session.c:66-108) unconditionally zeroes sharedInUse/resInUse every round (normal-mode cadence: 1s, main.c:453-460), that job's charge -- correct within the round it was dispatched -- is wiped at the very next round and not restored until the RUN confirmation is processed. The admission check other jobs go through (sched_check_host_satisfy_/sched_check_shared_satisfy_, resource.c:252-370, called from seqjob_allocator.c:130) reads exactly this same resInUse/sharedInUse, with no separate reserved/committed pool (xce_sched_job_t has only state/allocs/accounts, sched/types.h:96-116) -- so once wiped, the resource looks free again and can be granted to a second job, a genuine over-commit/double-booking risk under any dispatch-to-worker-confirmation latency exceeding ~1 round (ordinary for real worker RPC + fork/exec). ## Fix FIXED (2026-08-23, scheduler-dispatch-commit tasks 2.1 + 3.1). Scheduler side (task 2.1, commit 8607348c): sched_commit_decision()'s DISPATCH branch now sets job->state = JOB_STAT_RUN synchronously, in the same operation that already sets job->accounts/job->allocs -- no separate fix to sched_account_jobs()'s JOB_IS_RUN gate was needed; the job simply reaches RUN from the round it's dispatched, so the existing gate naturally includes it every subsequent round regardless of confirmation latency. Controller side (task 3.1, commit d9aedc4b): dispatch_a_job_() gates the job.allocs commit + JOB_STAT_RUN transition together on xce_startjob_req() succeeding, keeping both sides consistent. Regression-tested end-to-end: test_dispatch_commit_survives_several_rounds_before_confirmation and test_dispatch_commit_multi_round_gap_matches_same_round_confirmation (src/scheduler/testsuite/test_scheduler_dispatch_commit.c, task 5.1) prove resInUse survives a 5-round gap with no confirmation event, and match a same-round-confirmation baseline; RED-phase independently verified via a real revert/rebuild/rerun of the fix by both the task 5.1 implementer and its reviewer. Full HA soak (sequential + chaotic, 10 rounds each) confirms no new job loss or divergence, task 5.4. ## Verification _Not recorded (migrated from buglog.json)._ ## References - Related: bug-1611, bug-1613