lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(scheduler): PEND reason classification — main_reason + limit/queue/worker reasons (POC "why is my job waiting")

Browse source 
P0 (done, cluster-verified):
- main_reason was always 0: add_job_preason only set it when BOTH host and res
  were NULL, but the allocator always attaches host/res detail rows. Now the
  first detail add wins as main_reason, so the API's main_reason_name is
  meaningful (was "unknown(0)"). Verified live: PEND jobs now report
  main_reason=3/match_fail + per-host reason_name.
- limit plugin attach: host-based quota hit (hg1_slots_per_host) -> LIMIT_HOST
  + limit name; global quota hit (ug1_slots, ...) -> LIMIT_GLOBAL + limit name.
  Previously both blocks were silent (job just stayed PEND).
- Extended the PREASON_* enum (was 3 codes) with LSF-style classes:
  LIMIT_GLOBAL/LIMIT_HOST/QUEUE_CLOSED/QUEUE_DRAINING/TIME_WINDOW/HOST_UNAVAIL.
  apiserver pendReasonName maps them; host/res detail rows now carry reason_name.

P1 (implemented):
- queue status: sched_queue_is_ready no longer a stub — closed/inactivated
  queues are not electable; the fcfs election attaches QUEUE_CLOSED /
  QUEUE_DRAINING to every PEND job in a skipped queue.
- worker filter: sched_try_alloc_on_host_ skips UNAVAIL/CLOSED workers and
  attaches HOST_UNAVAIL instead of trying to allocate on them.

Unit tests: preason suite updated for the main_reason change + 3 new tests
(limit host/global, first-add-wins); 295 lib tests pass under ASAN.
apiserver backend tests pass.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
lei.chen 2026-08-14 09:43:21 +00:00
parent 62778930e2
commit 4e2ac861f4
10 changed files with 211 additions and 23 deletions
Download patch file Download diff file Expand all files Collapse all files

15
src/include/preason.h
View file

@ -4,9 +4,18 @@
#include "xce/types.h"
#define PREASON_BAD_RESREQ (1)
#define PREASON_NO_RESOURCE (2)
#define PREASON_MATCH_FAIL (3)
#define PREASON_BAD_RESREQ (1)
#define PREASON_NO_RESOURCE (2)
#define PREASON_MATCH_FAIL (3)
/* LSF-style PEND reason classes beyond plain capacity / host-match failure.
* Codes 4+ feed both main_reason and the host/res detail rows; the API layer
* maps each to a human-readable name (pendReasonName in apiserver). */
#define PREASON_LIMIT_GLOBAL (4) /* global quota hit (ug1_slots, ug2_slots_per_user, ...) */
#define PREASON_LIMIT_HOST (5) /* host-based quota hit (hg1_slots_per_host) */
#define PREASON_QUEUE_CLOSED (6) /* queue closed/disabled, not accepting jobs */
#define PREASON_QUEUE_DRAINING (7) /* queue draining (no new jobs until empty) */
#define PREASON_TIME_WINDOW (8) /* outside the queue's dispatch_window */
#define PREASON_HOST_UNAVAIL (9) /* host down/unavailable, skipped for allocation */
extern void erase_job_preason(xce_job_preason_t*);
extern int add_job_preason(xce_job_preason_t*, int, char*, char*);