fix(configs): add worker-builtin load indices missing from resources.yaml

worker/src/resource_sync.c init_builtin_load_indices registers 11 load
indices (r15s/r1m/r15m/ut/pg/ls/it/swp/mem/tmp/io) but resources.yaml only
defined r1m/r15s/mem/tmp. The 7 missing ones (r15m/ut/pg/ls/it/swp/io) caused
controller "update_resval: Success" ERRORs and scheduler "unknown resource in
per-worker resvals update" warnings, leaving the worker resource view
incomplete. Add the 7 definitions matching the existing r1m/r15s pattern.

Docs: log bug-1494 (all jobs PEND, scheduler never dispatches; after this
config fix jobs still PEND — remaining suspect is the scheduler→controller
join path after restart), update design doc + verification manual known-issue
tables and root CLAUDE.md cluster-verification status.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
lei.chen 2026-08-02 07:36:03 +00:00
parent 226259a52b
commit 62dee0edc1
6 changed files with 130 additions and 5 deletions

View file

@ -14564,5 +14564,64 @@
],
"occurrences": 4,
"last_seen": "2026-08-02"
},
{
"id": "bug-1492",
"timestamp": "2026-08-02T12:20:00+08:00",
"error_message": "scheduler_list_nodes/cluster_metrics memory_bytes_total=8224821248000000 (~8.2 PB) \u2014 unrealistically large; expected ~8.2 GB",
"file": "src/frontend/schedulerclient/httpclient.go",
"root_cause": "PENDING USER JUDGMENT. C hostattrs_t.maxmem (src/include/xce/types.h:191) is a double carrying 8224821248 in live data. If that value is already bytes (~8.2GB), schedulerclient's mbToBytes(1e6) multiply in ListNodes makes it 8.2 PB. Could also be debug-info or another cause. User will decide next step.",
"fix": "NOT DECIDED \u2014 awaiting user judgment on whether this is a real bug, debug artifact, or other cause.",
"tags": [
"schedulerclient",
"memory-units",
"host",
"observation"
],
"related_bugs": [
"bug-1491"
],
"occurrences": 1,
"last_seen": "2026-08-02T12:20:00+08:00"
},
{
"id": "bug-1493",
"timestamp": "2026-08-02T12:35:00+08:00",
"error_message": "scheduler_get_job returns the first job in the full list regardless of requested ID. Requested 73929981243162624 (sleep666) and 73929907581747200 (verify3) both returned verify1 (73929906127372288).",
"file": "src/frontend/apiserver/controllers/jobs.go (GetJobByIDs) + src/frontend/schedulerclient/httpclient.go GetJob",
"root_cause": "GET /xce/v1/jobs/:jobids calls GetJobByIDs which builds job_id eq filter, but live response returns ALL jobs (count:5) \u2014 the filter is not applied by the QueryData/store layer. schedulerclient.GetJob then takes jobs[0], masking it as 'returns wrong job'. Filter application on job_id appears broken in the query layer.",
"fix": "NOT YET FIXED \u2014 awaiting user judgment. Likely fix in query/store layer to honor job_id filter, or client-side post-filter by ID.",
"tags": [
"apiserver",
"schedulerclient",
"get-job",
"filter",
"query-layer"
],
"related_bugs": [
"bug-1491"
],
"occurrences": 2,
"last_seen": "2026-08-02T12:35:00+08:00"
},
{
"id": "bug-1494",
"timestamp": "2026-08-02",
"error_message": "All jobs stay PEND \u2014 scheduler never dispatches. New jobs (submitted while scheduler running) also stay PEND; scheduler log has 0 job/alloc lines.",
"file": "src/controller/sched.c, src/configs/resources.yaml",
"root_cause": "WORKER-HARDCODED LOAD INDICES missing from resources.yaml (worker/src/resource_sync.c init_builtin_load_indices registers r15m/ut/pg/ls/it/swp/io; resources.yaml only had r1m/r15s/mem/tmp) \u2192 controller \"update_resval: Success\" ERROR + scheduler \"unknown resource in per-worker resvals update\". Fixed resources, rebuilt cluster.bin (a7824e0586d0), restarted controller+scheduler+apiserver \u2014 jobs STILL PEND. NEW scheduler (292698) went completely silent (no sched_update, no job log), unlike the old scheduler which got worker resvals every 15s. tcpdump earlier proved controller DOES push a 11720B payload (jobs) to scheduler on join. So the remaining suspect is the scheduler\u2192controller join/connection path after restart, not resource mismatch. NOT ROOT-CAUSED.",
"fix": "NOT FIXED. Partial: added 7 missing resources to resources.yaml + rebuilt cluster.bin (a7824e0586d0) + restarted components. Remaining: investigate why the restarted scheduler does not connect/join the controller (new scheduler silent).",
"tags": [
"scheduler",
"pend",
"dispatch",
"join",
"open"
],
"related_bugs": [
"bug-1492"
],
"occurrences": 1,
"last_seen": "2026-08-02"
}
]