forked from lei.chen/myxce
docs(buglog): record bug-1503 — scheduler-before-worker join leaves slots unknown
Job PEND root cause when the scheduler registered before the worker: xcesched_push_workers only fires on scheduler join (sched.c on_scheduler_join), so a later worker join's slots (set_host_slots from hconfig->mxj=8) is never re-pushed to the scheduler -> allocation unsatisfied (slots total 0) -> PEND. Operational fix (no code change): start controller -> worker -> scheduler so the scheduler's join-time schedworkers push carries the worker's slots. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
parent
2fa9bedcd3
commit
60ebf4af72
2 changed files with 22 additions and 0 deletions
|
|
@ -14776,5 +14776,26 @@
|
|||
"related_bugs": [],
|
||||
"occurrences": 3,
|
||||
"last_seen": "2026-08-04"
|
||||
},
|
||||
{
|
||||
"id": "bug-1503",
|
||||
"timestamp": "2026-08-05",
|
||||
"error_message": "\u4f5c\u4e1a\u63d0\u4ea4\u540e\u4e00\u76f4 PEND \u4e0d\u8c03\u5ea6\uff0cscheduler \u65e5\u5fd7 unsatisfy_ resource 0 (slots) total 0.000000",
|
||||
"file": "src/controller/sched.c (on_scheduler_join / xcesched_push_workers) + worker.c (apply_join_state)",
|
||||
"root_cause": "\u65f6\u5e8f\u95ee\u9898\uff1axcesched_push_workers \u53ea\u5728 scheduler join \u65f6\u89e6\u53d1\uff08sched.c:668\uff09\u3002\u82e5 scheduler \u5148\u4e8e worker join\uff0c\u4e4b\u540e worker \u52a0\u5165\u65f6\uff08controller \u7528 set_host_slots \u8bbe\u7f6e slots=hconfig->mxj=8\uff09\uff0ccontroller \u4e0d\u4f1a\u91cd\u63a8 schedworkers \u7ed9 scheduler\uff0cscheduler \u7684 worker slots \u4fdd\u6301 0 -> \u5206\u914d\u5931\u8d25 -> PEND\u3002sched_worker_change \u7684 WORKER_JOIN hook \u63a8\u9001\u673a\u5236\u5b58\u5728\uff0c\u4f46\u5728 controller PRIMARY \u5207\u6362\u671f\u95f4 scheduler \u8fde\u63a5\u672a\u5efa\u7acb\u65f6\u4e0d\u4f1a\u89e6\u53d1\u3002\u914d\u7f6e\u672c\u8eab\u6b63\u786e\uff08t1 mxj=8\uff0cDIAG \u786e\u8ba4 apply_join_state nslots=8\uff09\u3002",
|
||||
"fix": "\u65e0\u4ee3\u7801\u6539\u52a8\uff08\u64cd\u4f5c\u65f6\u5e8f\uff09\uff1a\u6309 controller -> worker -> scheduler -> apiserver \u987a\u5e8f\u542f\u52a8\uff0c\u786e\u4fdd scheduler \u5728 worker join \u4e4b\u540e\u6ce8\u518c\uff0cjoin \u65f6\u6536\u5230\u542b slots \u7684 schedworkers \u63a8\u9001\u3002\u91cd\u542f scheduler\uff08worker \u5df2\u6ce8\u518c\uff09\u540e\u4f5c\u4e1a\u7acb\u5373\u6d3e\u53d1\u3002",
|
||||
"tags": [
|
||||
"scheduler",
|
||||
"pending",
|
||||
"slots",
|
||||
"ordering",
|
||||
"worker-join",
|
||||
"dispatch"
|
||||
],
|
||||
"related_bugs": [
|
||||
"bug-1502"
|
||||
],
|
||||
"occurrences": 1,
|
||||
"last_seen": "2026-08-05"
|
||||
}
|
||||
]
|
||||
|
|
@ -7369,3 +7369,4 @@
|
|||
| 13:50 | 看门狗改为纯日志模式(不自动重启): 移到 /home/dankeer/bin/frpc-watchdog.sh + crontab @reboot 开机自启; 只在隧道状态变化时写日志(DOWN抓快照/恢复记录/持续DOWN每5min) | /home/dankeer/bin/frpc-watchdog.sh | 部署完成, 用户人工监视 | ~200 |
|
||||
| 05:50 | Session end: 9 writes across 5 files (frpcdbg_mon.sh, frpc_prod.toml, frpc_mon.sh, frpc_watchdog.sh, frpc-watchdog.sh) | 2 reads | ~2219 tok |
|
||||
| 09:39 | 修复 controller 反复崩溃 bug-1502: persist_job_hook_fun 把 RUSAGE_UPDATE hook 的第2参 (xce_job_rusages_t*) 当 job_t* 用, 读 offset 56 越界 56字节结构 -> ASAN heap-buffer-overflow. 加专用 case 从 jobmgr 取当前 state | src/controller/persist.c, testsuite/test_persist_hooks.c | RED(旧代码 stack-buffer-overflow)/GREEN(修复后 PASS), 全套458测试通过 | ~350 |
|
||||
| 01:08 | 验证完成: bug-1502(controller ASAN崩溃)修复生产生效(sleep 666跑满666秒, controller 0崩溃); 作业PENDING根因=配置正确但scheduler先于worker join拿不到slots(操作时序, bug-1503); 干净controller部署, 集群全链路(apiserver提交->scheduler->controller->worker)冒烟通过 | persist.c, sched.c, worker.c | 验证通过 | ~300 |
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue