From 60ebf4af722d063d0875c1b6bfd9e014172d4f41 Mon Sep 17 00:00:00 2001 From: "lei.chen" Date: Wed, 5 Aug 2026 01:09:02 +0000 Subject: [PATCH] =?UTF-8?q?docs(buglog):=20record=20bug-1503=20=E2=80=94?= =?UTF-8?q?=20scheduler-before-worker=20join=20leaves=20slots=20unknown?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Job PEND root cause when the scheduler registered before the worker: xcesched_push_workers only fires on scheduler join (sched.c on_scheduler_join), so a later worker join's slots (set_host_slots from hconfig->mxj=8) is never re-pushed to the scheduler -> allocation unsatisfied (slots total 0) -> PEND. Operational fix (no code change): start controller -> worker -> scheduler so the scheduler's join-time schedworkers push carries the worker's slots. Co-Authored-By: Claude --- .wolf/buglog.json | 21 +++++++++++++++++++++ .wolf/memory.md | 1 + 2 files changed, 22 insertions(+) diff --git a/.wolf/buglog.json b/.wolf/buglog.json index 75b2d8b..8e0357d 100644 --- a/.wolf/buglog.json +++ b/.wolf/buglog.json @@ -14776,5 +14776,26 @@ "related_bugs": [], "occurrences": 3, "last_seen": "2026-08-04" + }, + { + "id": "bug-1503", + "timestamp": "2026-08-05", + "error_message": "\u4f5c\u4e1a\u63d0\u4ea4\u540e\u4e00\u76f4 PEND \u4e0d\u8c03\u5ea6\uff0cscheduler \u65e5\u5fd7 unsatisfy_ resource 0 (slots) total 0.000000", + "file": "src/controller/sched.c (on_scheduler_join / xcesched_push_workers) + worker.c (apply_join_state)", + "root_cause": "\u65f6\u5e8f\u95ee\u9898\uff1axcesched_push_workers \u53ea\u5728 scheduler join \u65f6\u89e6\u53d1\uff08sched.c:668\uff09\u3002\u82e5 scheduler \u5148\u4e8e worker join\uff0c\u4e4b\u540e worker \u52a0\u5165\u65f6\uff08controller \u7528 set_host_slots \u8bbe\u7f6e slots=hconfig->mxj=8\uff09\uff0ccontroller \u4e0d\u4f1a\u91cd\u63a8 schedworkers \u7ed9 scheduler\uff0cscheduler \u7684 worker slots \u4fdd\u6301 0 -> \u5206\u914d\u5931\u8d25 -> PEND\u3002sched_worker_change \u7684 WORKER_JOIN hook \u63a8\u9001\u673a\u5236\u5b58\u5728\uff0c\u4f46\u5728 controller PRIMARY \u5207\u6362\u671f\u95f4 scheduler \u8fde\u63a5\u672a\u5efa\u7acb\u65f6\u4e0d\u4f1a\u89e6\u53d1\u3002\u914d\u7f6e\u672c\u8eab\u6b63\u786e\uff08t1 mxj=8\uff0cDIAG \u786e\u8ba4 apply_join_state nslots=8\uff09\u3002", + "fix": "\u65e0\u4ee3\u7801\u6539\u52a8\uff08\u64cd\u4f5c\u65f6\u5e8f\uff09\uff1a\u6309 controller -> worker -> scheduler -> apiserver \u987a\u5e8f\u542f\u52a8\uff0c\u786e\u4fdd scheduler \u5728 worker join \u4e4b\u540e\u6ce8\u518c\uff0cjoin \u65f6\u6536\u5230\u542b slots \u7684 schedworkers \u63a8\u9001\u3002\u91cd\u542f scheduler\uff08worker \u5df2\u6ce8\u518c\uff09\u540e\u4f5c\u4e1a\u7acb\u5373\u6d3e\u53d1\u3002", + "tags": [ + "scheduler", + "pending", + "slots", + "ordering", + "worker-join", + "dispatch" + ], + "related_bugs": [ + "bug-1502" + ], + "occurrences": 1, + "last_seen": "2026-08-05" } ] \ No newline at end of file diff --git a/.wolf/memory.md b/.wolf/memory.md index 62261e6..5868d98 100644 --- a/.wolf/memory.md +++ b/.wolf/memory.md @@ -7369,3 +7369,4 @@ | 13:50 | 看门狗改为纯日志模式(不自动重启): 移到 /home/dankeer/bin/frpc-watchdog.sh + crontab @reboot 开机自启; 只在隧道状态变化时写日志(DOWN抓快照/恢复记录/持续DOWN每5min) | /home/dankeer/bin/frpc-watchdog.sh | 部署完成, 用户人工监视 | ~200 | | 05:50 | Session end: 9 writes across 5 files (frpcdbg_mon.sh, frpc_prod.toml, frpc_mon.sh, frpc_watchdog.sh, frpc-watchdog.sh) | 2 reads | ~2219 tok | | 09:39 | 修复 controller 反复崩溃 bug-1502: persist_job_hook_fun 把 RUSAGE_UPDATE hook 的第2参 (xce_job_rusages_t*) 当 job_t* 用, 读 offset 56 越界 56字节结构 -> ASAN heap-buffer-overflow. 加专用 case 从 jobmgr 取当前 state | src/controller/persist.c, testsuite/test_persist_hooks.c | RED(旧代码 stack-buffer-overflow)/GREEN(修复后 PASS), 全套458测试通过 | ~350 | +| 01:08 | 验证完成: bug-1502(controller ASAN崩溃)修复生产生效(sleep 666跑满666秒, controller 0崩溃); 作业PENDING根因=配置正确但scheduler先于worker join拿不到slots(操作时序, bug-1503); 干净controller部署, 集群全链路(apiserver提交->scheduler->controller->worker)冒烟通过 | persist.c, sched.c, worker.c | 验证通过 | ~300 |