sched_publish_retry_loop() calls sched_check_pending_rejoin() after every successful publish -- inside the driver's decision-drain loop -- so after a client-library reconnect the scheduler re-joins mid-round. The controller answers every join (on_scheduler_join(), controller/sched.c) by re-pushing config, workers and jobs. handler.c applied the config push synchronously on the handler thread: sched_init_config() -> sched_init_queues() -> builtin_fcfs on_queue_update(), which nulls the picker cursor when it reinserts the queue the cursor rests on. next_job() then returned NULL and the round ended with the rest of the queue never considered. Evidence (scheduler-replay-20260923-085356.replay, temporary event-stream diagnostics, reverted): 9 recorded rounds end after 11-12 decisions; every one is followed by a workersPush + full 50-job jobsPush (the join's resync, reporting just-dispatched jobs back as PEND), every full-length dispatching round has none, and the config push never appears because it bypassed the pending-update queue the recorder writes from -- which is also why replay could not reproduce the truncation. Fix (defer to the round edge, same direction as bug-1813): handler.c now calls new sched_stash_config(), parking the Conf4Sched in new xce_scheduler_t.pending_config under pending_config_lock (an older unapplied push is freed). main.c's loop calls new sched_apply_pending_config() before each round, running the unchanged sched_init_config() strictly between sessions. sched_fini() frees a never-applied push. Tests: - test_handlers.c: test_config_push_is_stashed_not_applied_on_handler_thread -- RED (sched_init_config called on the handler thread), GREEN after. - new test_config_push_deferred.c (sched_config_push_deferred_test): real scheduler_req_handler() + real builtin_fcfs mid-session -- RED (next_job() returned NULL right after the push), GREEN (session walks the remaining jobs; configver unchanged until applied once between rounds). - test_admin_request.c / test_job_array_expand.c / test_job_array_wire_e2e.c #include handler.c directly and get a link stub for sched_stash_config(). Full scheduler suite 87/87 (asan). Not addressed: config pushes are still not written to a recording (a replay spanning a config change won't see it); the ~100s reconnect churn itself (see bug-1812/1813). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .claude | ||
| .kiro | ||
| .wolf | ||
| docs/superpowers/plans | ||
| issues | ||
| raft | ||
| reports | ||
| scripts | ||
| src | ||
| .clang-format | ||
| .gitignore | ||
| .mcp.json | ||
| .opencode.json | ||
| CHANGES.md | ||
| CLAUDE.md | ||
| README.md | ||
1. commit 规则
- 每个人 fork 自己的 branch, 从自己的 branch 做 PR,review 完再 commit
2. code format
2.1. C
推荐使用 clang-format,公共配置放在源码里面 xce/.clang-format
(理论上直接使用源码里面 xce/.clang-format 文件格式不会变)
2.1.1. vscode 可以在配置中加上如下内容
"C_Cpp.clang_format_style": "WebKit",
"C_Cpp.clang_format_fallbackStyle": "WebKit",
2.1.2. vim & emacs
clang-format -style="{BasedOnStyle: webkit}" -i XX.c
2.2. golang
使用默认 gofmt
3. build environment and build (example of ubuntu)
3.1 build environment
- Install nix package management tool
sh <(curl -L https://nixos.org/nix/install) --daemon
- Edit /etc/nix/nix.conf to make sure
experimental-features = nix-command flakes
sandbox = true
sandbox-fallback = false
substituters = https://mirrors.ustc.edu.cn/nix-channels/store https://cache.nixos.org https://cache.nixos.org/
trusted-users = root <your account>
make sure that you have restarted nix-daemon service after you have changed /etc/nix/nix.conf
- Install socat and config ssh (for clone by ssh protocol)
nix profile add nixpkgs#socat
.ssh/config
Host gitea.shenrs.eu
HostName gitea.shenrs.eu
IdentityFile ~/.ssh/id_ed25519
ProxyCommand socat - openssl-connect:gitea.shenrs.eu:2022,snihost=gitea.shenrs.eu,verify=0
3.2 checkout source code
git clone ssh://forgejo@gitea.shenrs.eu/rshen/xce.git
cd xce
(参考第 1 节,fork 自己的 branch 再提交 PR)
3.3 build (for package and release)
cd xce/src
# for build debug version
nix build .#xcecore_debug
# for normal build
nix build .#xcecore
You can find the generated binaries at =result= directory
构建现在是**完全 hermetic(离线)**的,无需再加
--option sandbox false。 所有 Go 依赖都已 vendored(go mod vendor,提交到各模块的vendor/目录), Python 组件xceadmin也通过pip install --no-build-isolation --no-index从 nix 环境安装,构建阶段不再访问网络。因此在开启 sandbox(sandbox = true) 的机器上也能直接构建,不会再出现向goproxy.cn/ PyPI 请求而 DNS 解析失败的问题。
3.3.1 更新 Go 依赖(vendor 维护)
Go 模块(frontend/apiserver、frontend/xcecli、
autoscaler/providers/baidu/src/bcctool)的依赖以 vendor/ 目录的形式提交进仓库,
构建时 tools/go_build 使用 GOFLAGS=-mod=vendor、GOPROXY=off 完全离线编译。
当修改了某个模块的 go.mod / go.sum(新增、升级、删除依赖)后,需要在
nix develop 环境内重新生成对应模块的 vendor/ 并提交:
cd xce/src
nix develop
# 以 apiserver 为例,其他两个模块同理
cd frontend/apiserver
go mod tidy
go mod vendor
cd -
# vendor/ 必须被 git 跟踪,否则 flake 构建(只读取 git 已跟踪/暂存的文件)看不到它
git add frontend/apiserver/go.mod frontend/apiserver/go.sum frontend/apiserver/vendor
注意:
nix build从 git 树取源码,未被 git add 的vendor/文件不会进入构建, 会导致-mod=vendor报inconsistent vendoring而失败。改完依赖记得git add vendor/。
3.4 build (for daily develop)
日常开发推荐直接使用 src/tools/xce-build.sh(若在 Claude Code 中开发,对应的是 /xce-build skill),
它会执行 meson setup / meson compile,并在 asan profile 下跑测试。该脚本会自动进入 nix develop
环境(脚本开头检测 IN_NIX_SHELL,未设置时自动用 nix develop 重新执行自身),无需手动
nix develop 或额外包一层,直接跑即可:
cd xce/src
tools/xce-build.sh # 全部组件 + 全部测试 (asan profile, 输出到 xce.build/)
tools/xce-build.sh controller # 只构建/测试 controller (lib + controller + xce_test)
tools/xce-build.sh worker # 只构建 worker
tools/xce-build.sh scheduler # 只构建/测试 scheduler
tools/xce-build.sh apiserver # 只构建/测试 apiserver + xcecli (Go)
(如果已经身处 nix develop 交互式 shell 内运行以上命令,脚本会检测到 IN_NIX_SHELL 已设置,跳过重新
进入的步骤,直接执行构建。)
支持三种 profile(tools/xce-build.sh [component] [profile]):
asan(默认):debug + AddressSanitizer,输出目录xce.build/,会运行测试debug:-O0 -g3无 sanitizer,输出目录build.dbg/,仅编译,用于 gdb 调试release:-O3无 sanitizer,输出目录build.release/,仅编译,用于 soak/性能测试
注意:单组件构建默认会先 rm -rf 对应的 build 目录再重新配置,若同一目录下已有其他组件的产物会被清掉;
需要在同一目录下累积多个组件时加 --incremental(增量 reconfigure,不清空目录)。
也可以手动执行底层的 nix + meson 命令(xce-build.sh 实际上就是对下面步骤的封装):
cd xce/src
nix develop
# using meson to generated build configuration (debug and address sanitize)
meson setup xce.build --buildtype=debug -Db_sanitize=address
# run meson to build
meson compile -C xce.build
# The generated files will be in xce.build/
3.5 build AppImage package
cd xce/src
nix build .#xce.appImage
This will create an all-in-one xce.AppImage (available at ./result) which contains everything for package delivery.
4. misc
4.1. Windows remap ctrl and alt key
create a reg file, e.g. remapkey.reg
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Keyboard Layout]
"Scancode Map"=hex:00,00,00,00,00,00,00,00,05,00,00,00,38,00,1d,00,1d,00,38,00,38,e0,1d,e0,1d,e0,38,e0,00,00,00,00
then double click remapkey.reg to import it. Restart OS.