lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(scheduler): put_resvals treats resource idx 0 as invalid → SEGV

Browse source 
put_resvals (resource.c) used `idxs[i] <= 0` to skip "invalid" resource
indexes, but index 0 IS valid — it's the first resource in the table
(e.g. "slots"). For idx 0 the val was never allocated (xvals[i] stayed
NULL) and the write loop then SEGV'd. bug-1495.

Fix: skip only negative indexes (`idxs[i] < 0`), and NULL-guard the
write loop defensively.

Verified end-to-end: a locally-built scheduler now connects to the t1
controller and dispatches all 5 test jobs to node t1 (exec_hosts
populated, jobs run and exit). This was the final blocker for job
scheduling (bug-1494: all jobs PEND).

Also: add the t1 host definition to hosts.yaml (worker join was rejected
with -EINVAL because no matching hostconfig existed for hostname "t1").

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
lei.chen 2026-08-03 06:53:14 +00:00
parent 62dee0edc1
commit 07ab05671d
4 changed files with 57 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

26
.wolf/buglog.json
View file

@ -14610,18 +14610,38 @@
"error_message": "All jobs stay PEND \u2014 scheduler never dispatches. New jobs (submitted while scheduler running) also stay PEND; scheduler log has 0 job/alloc lines.",
"file": "src/controller/sched.c, src/configs/resources.yaml",
"root_cause": "WORKER-HARDCODED LOAD INDICES missing from resources.yaml (worker/src/resource_sync.c init_builtin_load_indices registers r15m/ut/pg/ls/it/swp/io; resources.yaml only had r1m/r15s/mem/tmp) \u2192 controller \"update_resval: Success\" ERROR + scheduler \"unknown resource in per-worker resvals update\". Fixed resources, rebuilt cluster.bin (a7824e0586d0), restarted controller+scheduler+apiserver \u2014 jobs STILL PEND. NEW scheduler (292698) went completely silent (no sched_update, no job log), unlike the old scheduler which got worker resvals every 15s. tcpdump earlier proved controller DOES push a 11720B payload (jobs) to scheduler on join. So the remaining suspect is the scheduler\u2192controller join/connection path after restart, not resource mismatch. NOT ROOT-CAUSED.",
"fix": "NOT FIXED. Partial: added 7 missing resources to resources.yaml + rebuilt cluster.bin (a7824e0586d0) + restarted components. Remaining: investigate why the restarted scheduler does not connect/join the controller (new scheduler silent).",
"fix": "FIXED (via bug-1495 root cause): put_resvals used idxs[i] \"<= 0\" which treated the VALID first resource index 0 (slots) as invalid, skipping its val fill and then SEGV on the write. Changed to \"idx < 0\" + NULL-guard on the write loop. After fix, a locally-built scheduler connected to t1 controller and dispatched all 5 jobs to node t1 (exec_hosts populated); all jobs ran and exited (status EXIT). Full scheduling chain verified working.",
"tags": [
"scheduler",
"pend",
"dispatch",
"join",
"open"
"fixed"
],
"related_bugs": [
"bug-1492"
],
"occurrences": 1,
"last_seen": "2026-08-02"
"last_seen": "2026-08-03"
},
{
"id": "bug-1495",
"timestamp": "2026-08-03",
"error_message": "scheduler segfault during job scheduling: SEGV write to 0x8 in put_resvals (scheduler/src/resource.c:90) when committing a job allocation",
"file": "src/scheduler/src/resource.c",
"root_cause": "put_resvals (resource.c:89-91) writes xvals[i]->u.fvalue unconditionally, but the first loop only fills xvals[i] for idxs[i] > 0. When accounts[i]->residx is invalid (<=0, i.e. the resource name was NOT found in sched->resmgr at resource.c:422-431, so residx stays 0/uninitialized), the second loop dereferences a NULL xvals[i] -> SEGV. Stack: sched_allocator -> commit_decision -> sched_commit_decision -> sched_commit_job_alloc -> sched_charge_account -> sched_update_account -> sched_update_res_accounts -> put_resvals. Root trigger: a job/account references a resource name absent from the loaded resources.yaml (resource-convention mismatch between job and scheduler config).",
"fix": "FIXED: resource.c put_resvals idxs[i] \"<= 0\" -> \"< 0\" (idx 0 = valid, e.g. slots), plus NULL-guard on xvals[i] write loop. Verified: scheduler no longer SEGVs and dispatched all 5 test jobs to t1 (exec_hosts populated, jobs EXIT).",
"tags": [
"scheduler",
"segfault",
"resource",
"account",
"fixed"
],
"related_bugs": [
"bug-1494"
],
"occurrences": 1,
"last_seen": "2026-08-03"
}
]