No description
Rongsong Shen
1ce007814d
Root cause: peer_ctrl_msg_handler held ep->dispatch_inflight incremented across the entire peer_ctrl_dispatch_to_pool call, including its inline fallback (the common cold-start shape when ha->pool isn't wired yet). That inline body can reach raft_cb_become_follower, which -- now that cluster_init_phase2 ran synchronously earlier in the same call chain and wired the pool -- tears down its own loser vote-ep (the very ep whose reply triggered this call) via the deferred path (vote_ep_teardown_async), whose endpoint_mark_tearing_down spins waiting for dispatch_inflight==0 on that SAME ep. That count is held only by this exact call, on this exact thread, and can't reach 0 until the spin itself returns -- a permanent self-deadlock. Confirmed live via a gdb thread backtrace: added a GDB_WRAP debug seam to run-ha-soak.sh (launches a named node directly under `gdb --batch` so it becomes ptrace-legal as gdb's own direct child under yama scope 1, bypassing a live-attach restriction), then SIGINT to gdb once the log went stale for a bounded interrupt + full backtrace. The frame dump showed pool=0x0 at peer_ctrl_dispatch_to_pool, and the same ep pointer in the nested vote_ep_teardown_async -> endpoint_mark_tearing_down frame, spinning on sched_yield(). Fix: moved the dispatch_inflight release into peer_ctrl_dispatch_to_pool itself, at the exact right point per exit path -- released BEFORE calling the inline body on every fallback path (pool==NULL, OOM, enqueue failure), since io_worker is single-threaded and the inline path never enqueues anything (no pool-worker race left to protect against once inline execution is committed to); released only AFTER the non-blocking enqueue completes on the success path, preserving the original protection for the genuinely concurrent pool-worker case. New unit test test_vep_self_teardown_does_not_deadlock uses a bounded pthread_timedjoin_np so a regression fails loud instead of hanging the suite. RED confirmed via a temporary manual revert (ETIMEDOUT) before restoring the fix. Live-verified with both flags ON: --force-snapshot soak (the exact repro shape) completes without hanging across repeated runs, confirms real snapshot installs, zero divergence, zero job loss; a 6-round --chaotic soak also passed clean; full controller suite green throughout. g_drive_async_snapshot_offload and g_drive_async_peerctrl_dispatch_enabled now default ON in production, closing raft-drive-async tasks 3.1 and 3.2. g_drive_async_configpush_offload stays OFF -- a distinct, still-open hazard in push_config_transport_only's endpoint handling (task 3.3, unrelated to this bug). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014dZVSx3CdBjys9BB6hqpxP |
||
|---|---|---|
| .claude | ||
| .kiro | ||
| .wolf | ||
| docs/superpowers/plans | ||
| ha/data | ||
| raft | ||
| src | ||
| .clang-format | ||
| .gitignore | ||
| .mcp.json | ||
| .opencode.json | ||
| CLAUDE.md | ||
| package-lock.json | ||
| package.json | ||
| README.md | ||
| uninstall-code-review-graph.sh | ||
1. commit 规则
- 每个人 fork 自己的 branch， 从自己的 branch 做 PR，review 完再 commit
2. code format
2.1. C
推荐使用 clang-format，公共配置放在源码里面 xce/.clang-format
（理论上直接使用源码里面 xce/.clang-format 文件格式不会变）
2.1.1. vscode 可以在配置中加上如下内容
"C_Cpp.clang_format_style": "WebKit",
"C_Cpp.clang_format_fallbackStyle": "WebKit",
2.1.2. vim & emacs
clang-format -style="{BasedOnStyle: webkit}" -i XX.c
2.2. golang
使用默认 gofmt
3. build environment and build (example of ubuntu)
3.1 build environment
- Install nix package management tool
sh <(curl -L https://nixos.org/nix/install) --daemon
- Edit /etc/nix/nix.conf to make sure
experimental-features = nix-command flakes
sandbox = true
sandbox-fallback = false
substituters = https://mirrors.ustc.edu.cn/nix-channels/store https://cache.nixos.org https://cache.nixos.org/
trusted-users = root <your account>
make sure that you have restarted nix-daemon service after you have changed /etc/nix/nix.conf
- Install socat and config ssh (for clone by ssh protocol)
nix profile add nixpkgs#socat
.ssh/config
Host gitea.shenrs.eu
HostName gitea.shenrs.eu
IdentityFile ~/.ssh/id_ed25519
ProxyCommand socat - openssl-connect:gitea.shenrs.eu:2022,snihost=gitea.shenrs.eu,verify=0
3.2 checkout source code
3.3 build (for package and release)
cd xce/src
# for build debug version
nix build --option sandbox false .#xcecore_debug
# for normal build
nix build --option sandbox false .#xcecore
You can find the generated binaries at =result= directory
3.4 build (for daily develop)
cd xce/src
nix develop
# using meson to generated build configuration (debug and address sanitize)
meson setup build --buildtype=debug -Db_sanitize=address
# run meson to build
meson compile -C build
# The generated files will be in build/
3.5 build AppImage package
cd xce/src
nix bundle --bundler github:ralismark/nix-appimage --option sandbox false .#xce
This will create all in one AppImage file which contains everything for package delivery.
4. misc
4.1. Windows remap ctrl and alt key
create a reg file, e.g. remapkey.reg
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Keyboard Layout]
"Scancode Map"=hex:00,00,00,00,00,00,00,00,05,00,00,00,38,00,1d,00,1d,00,38,00,38,e0,1d,e0,1d,e0,38,e0,00,00,00,00
then double click remapkey.reg to import it. Restart OS.