lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(scheduler): PEND reason classification — main_reason + limit/queue/worker reasons (POC "why is my job waiting")

Browse source 
P0 (done, cluster-verified):
- main_reason was always 0: add_job_preason only set it when BOTH host and res
  were NULL, but the allocator always attaches host/res detail rows. Now the
  first detail add wins as main_reason, so the API's main_reason_name is
  meaningful (was "unknown(0)"). Verified live: PEND jobs now report
  main_reason=3/match_fail + per-host reason_name.
- limit plugin attach: host-based quota hit (hg1_slots_per_host) -> LIMIT_HOST
  + limit name; global quota hit (ug1_slots, ...) -> LIMIT_GLOBAL + limit name.
  Previously both blocks were silent (job just stayed PEND).
- Extended the PREASON_* enum (was 3 codes) with LSF-style classes:
  LIMIT_GLOBAL/LIMIT_HOST/QUEUE_CLOSED/QUEUE_DRAINING/TIME_WINDOW/HOST_UNAVAIL.
  apiserver pendReasonName maps them; host/res detail rows now carry reason_name.

P1 (implemented):
- queue status: sched_queue_is_ready no longer a stub — closed/inactivated
  queues are not electable; the fcfs election attaches QUEUE_CLOSED /
  QUEUE_DRAINING to every PEND job in a skipped queue.
- worker filter: sched_try_alloc_on_host_ skips UNAVAIL/CLOSED workers and
  attaches HOST_UNAVAIL instead of trying to allocate on them.

Unit tests: preason suite updated for the main_reason change + 3 new tests
(limit host/global, first-add-wins); 295 lib tests pass under ASAN.
apiserver backend tests pass.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
lei.chen 2026-08-14 09:43:21 +00:00
parent 62778930e2
commit 4e2ac861f4
10 changed files with 211 additions and 23 deletions
Download patch file Download diff file Expand all files Collapse all files

30
src/frontend/apiserver/backend/convert_pure.go
View file

@ -161,15 +161,17 @@ type rawSubmitData struct {
// rawHostReason mirrors the C xce_host_reason_t (one host-level PEND reason).
type rawHostReason struct {
Host string `json:"host"`
Reason int `json:"reason"`
Host string `json:"host"`
Reason int `json:"reason"`
ReasonName string `json:"reason_name"`
}
// rawResReason mirrors the C xce_res_reason_t (one resource-level PEND reason).
type rawResReason struct {
Resource string `json:"resource"`
Host string `json:"host"`
Reason int `json:"reason"`
Resource string `json:"resource"`
Host string `json:"host"`
Reason int `json:"reason"`
ReasonName string `json:"reason_name"`
}
// rawPendReason mirrors the C xce_job_preason_t attached to a PEND job by the
@ -184,9 +186,9 @@ type rawPendReason struct {
}
// pendReasonName maps a PREASON_* main_reason code to a readable name. The
// codes are the PEND reason bitfield of xce_job_preason_t (PREASON_BAD_RESREQ/
// NO_RESOURCE/MATCH_FAIL, src/include/preason.h). Unknown codes are reported
// verbatim so a new scheduler reason never silently disappears.
// codes are the PEND reason bitfield of xce_job_preason_t (src/include/
// preason.h). Unknown codes are reported verbatim so a new scheduler reason
// never silently disappears.
func pendReasonName(code int) string {
switch code {
case 1: // PREASON_BAD_RESREQ
@ -195,6 +197,18 @@ func pendReasonName(code int) string {
return "no_resource"
case 3: // PREASON_MATCH_FAIL
return "match_fail"
case 4: // PREASON_LIMIT_GLOBAL
return "limit_global"
case 5: // PREASON_LIMIT_HOST
return "limit_host"
case 6: // PREASON_QUEUE_CLOSED
return "queue_closed"
case 7: // PREASON_QUEUE_DRAINING
return "queue_draining"
case 8: // PREASON_TIME_WINDOW
return "time_window"
case 9: // PREASON_HOST_UNAVAIL
return "host_unavail"
default:
return fmt.Sprintf("unknown(%d)", code)
}