forked from lei.chen/myxce
add run scripts
This commit is contained in:
parent
bbe8ed3335
commit
68ab5a071a
3 changed files with 38 additions and 0 deletions
|
|
@ -15037,5 +15037,27 @@
|
|||
"related_bugs": [],
|
||||
"occurrences": 1,
|
||||
"last_seen": "2026-08-13"
|
||||
},
|
||||
{
|
||||
"id": "bug-1514",
|
||||
"timestamp": "2026-08-21",
|
||||
"error_message": "换 branch 后全部作业再次 PEND 不调度:scheduler 持续 'unsatisfy_unmet resource 0 (slots) shortfall 1.000000',resval -n slots -m test01 返回 0。与 bug-1494 同症状。",
|
||||
"file": "src/configs/hosts.yaml + deploy/run/controller/ctrl-1/conf/cluster.bin.<tag>",
|
||||
"root_cause": "操作时序:hosts.yaml 15:27:08 加了 test01 hostconfig(max_slot: 8,用于修复 bug-1494 的 slots=0),但 cluster.bin 是 15:24:25 构建的——不含 test01 段。controller 加载旧 cluster.bin 后 worker test01 join 落到 default hostconfig(max_slot: -1 → apply_join_state 回落 w->host.attrs->maxcpus,worker 上报 maxcpus 为垃圾/0)→ slots=0 → scheduler 总 slots=0 → 全部 PEND。换 branch 重建二进制后只做了 config deploy(旧 hosts.yaml),未注意 hosts.yaml 工作区未提交修改未进 cluster.bin。",
|
||||
"fix": "重新 ./all.sh config deploy(用最新 hosts.yaml 重建 cluster.bin.d7c0dc804fc8)并重启 controller;之后 restart worker 让其重新 join(新 hostconfig max_slot:8 → slots=8);restart apiserver(重启 controller 时 apiserver ASAN 'Joining already joined thread' 崩溃)。验证:resval slots@test01=8.0,新作业 RUN,之前 PEND 的 sleep 221/222 也被补调度 RUN。教训:改 src/configs/*.yaml 后必须重新 config deploy,cluster.bin 是编译产物不会自动跟随源。",
|
||||
"tags": [
|
||||
"pend",
|
||||
"slots-zero",
|
||||
"cluster-bin-stale",
|
||||
"config-deploy",
|
||||
"operational",
|
||||
"bug-1494-symptom",
|
||||
"fixed"
|
||||
],
|
||||
"related_bugs": [
|
||||
"bug-1494"
|
||||
],
|
||||
"occurrences": 1,
|
||||
"last_seen": "2026-08-21"
|
||||
}
|
||||
]
|
||||
|
|
@ -7399,3 +7399,9 @@
|
|||
| 02:1x | Added human-readable jobid (job_id_human/human_id) to apiserver responses (backend.FormatJobID reuse); renamed SSE perm event:subscribe->event:view; updated docs (left_scheduler_work, design doc v1.3, cerebrum) | src/frontend/apiserver, src/frontend/schedulerclient, docs | verified live (submit + get_job return 1-0-... form); all tests pass; pushed | ~90k tok |
|
||||
| 06:21 | bug-1512 多 worker 过派发修复:handler.c code<0 才 re-PEND;24 作业 8/8/8 全 DONE;单测+提交 e4aaef4 | handler.c test_handlers.c | fixed+verified | ~2k |
|
||||
| 09:1x | PEND reason 全链路补全(bug-1513):controller SETPREASON hook 推送 + apiserver 转换层填充 + JOB_DEFAULT_COLUMN;集群验证 resreq type==nosuchhost → host_reasons [{t1,3},{t2,3},{t3,3}];提交 ac6131d | apiserver.c convert.go convert_pure.go convert_test.go models/jobs.go | verified | ~40k |
|
||||
|
||||
## Session: 2026-08-21 15:3x
|
||||
|
||||
| Time | Action | File(s) | Outcome | ~Tokens |
|
||||
|------|--------|---------|---------|--------|
|
||||
| 15:3x | 修复换 branch 后全部作业 PEND(bug-1514,bug-1494 症状复发):根因=hosts.yaml 15:27 加的 test01 hostconfig(max_slot:8) 未重新 config deploy(cluster.bin 15:24 构建不含),worker join 落 default(max_slot:-1→回落 maxcpus 垃圾值)→slots=0。重新 config deploy(cluster.bin.d7c0dc804fc8)+restart worker+restart apiserver(controller 重启时 apiserver ASan 崩溃)→ slots@test01=8.0,PEND 作业全部补调度 RUN。教训:改 src/configs/*.yaml 后必须重新 config deploy,cluster.bin 是编译产物不自动跟随源 | src/configs/hosts.yaml, all.sh, .wolf/buglog.json | fixed+verified | ~40k |
|
||||
|
|
|
|||
|
|
@ -54,6 +54,16 @@ hosts:
|
|||
model: "x86_64-linux-gnu"
|
||||
max_slot: 8
|
||||
|
||||
# test01: the local single-node worker (xcecd reports gethostname()=test01).
|
||||
# Without an explicit hostconfig the worker falls back to "default"
|
||||
# (max_slot: -1 -> auto-detect from worker attrs), which has left slots
|
||||
# unset (resvals empty) so the scheduler sees slots total=0 and every job
|
||||
# stays PEND (bug-1494). Give it a concrete max_slot like t1/t2/t3.
|
||||
- hostname: test01
|
||||
type: "x86_64-linux-gnu"
|
||||
model: "x86_64-linux-gnu"
|
||||
max_slot: 8
|
||||
|
||||
|
||||
hostgroups:
|
||||
- groupname: hg1
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue