No description
Rongsong Shen
ce8df99037
A restarted standby could get permanently stuck unable to catch up, with the primary re-streaming the identical doomed AppendEntries range every heartbeat with zero progress -- discovered via a live 3-node config-upgrade-snapshot soak. Liveness-only; no safety invariant (Election Safety, Leader Completeness, State Machine Safety, Commit Durability) was ever violated. Three-layer root cause, all now closed: 1. ha_stream_wal_to_standby (config_repl.c) used to skip advancing its prev_streamed_lsn/prev_streamed_term tracker across WAL_CHECKPOINT entries (a local-only, never-replicated marker that still consumes a real LSN slot). Now advances across them too, while still never shipping a catchup entry for the checkpoint itself. 2. ha_apply_term_entry_via_library (ha.c) now JIT-resyncs ha->raft.log_len from persist_durable_floor() before consuming an AE, to correct for persist_write_checkpoint_marker advancing current_lsn without going through raft_log_append/record_entry_term (pre-existing gap, bug-1466, not itself fixed here). 3. The actual root cause, found via real instrumentation after layers 1-2 alone still left a regression test red: record_entry_term's array-growth realloc zero-filled new slots, silently reclassifying a checkpoint's deliberately-never-recorded gap slot from "absent" to "present, term=0/WAL_LEGACY_TERM" the instant any later LSN got recorded -- a time-dependent read where a permanent fact was assumed. Fixed with a TERM_INDEX_UNSET (UINT64_MAX) sentinel that growth now fills gap slots with instead of zero, so entry_term_locked can distinguish "never explicitly written" from "explicitly recorded as legacy term 0" regardless of later array growth. The same determinism fix exposed a second, independent instance of the identical wedge via the snapshot-install catch-up path instead of WAL-streaming catch-up: persist_reconcile_checkpoint_term recorded a wire checkpoint_term of 0 as a raw, explicit write (bypassing the growth-fill sentinel entirely), so a snapshot-installed standby read the checkpoint LSN back as WAL_LEGACY_TERM(1) forever while the leader now correctly read its own checkpoint LSN as absent(0) -- the same permanent logOk mismatch, a different code path. Fixed by substituting TERM_INDEX_UNSET for a wire value of exactly 0 before recording; a wire 0 is unambiguous (persist_entry_term's own contract never returns raw 0 for a present entry, so 0 can only mean genuine absence). Independently reviewed across three rounds: sentinel mechanism verified via real RED->GREEN revert-and-rebuild; a documentation claim that config_repl.c/ha.c were "comment-only" corrected; the snapshot-path gap independently traced, confirmed real via its own RED->GREEN cycle, and fixed; two stale doc comments (persist.h's public contract, ha.c's call-site rationale) still describing the disproven pre-fix legacy-0 convention corrected. Suite: 1028 -> 1035 tests, 0 failures. |
||
|---|---|---|
| .claude | ||
| .kiro | ||
| .wolf | ||
| docs/superpowers/plans | ||
| raft | ||
| reports | ||
| scripts | ||
| src | ||
| .clang-format | ||
| .gitignore | ||
| .mcp.json | ||
| .opencode.json | ||
| CLAUDE.md | ||
| README.md | ||
1. commit 规则
- 每个人 fork 自己的 branch， 从自己的 branch 做 PR，review 完再 commit
2. code format
2.1. C
推荐使用 clang-format，公共配置放在源码里面 xce/.clang-format
（理论上直接使用源码里面 xce/.clang-format 文件格式不会变）
2.1.1. vscode 可以在配置中加上如下内容
"C_Cpp.clang_format_style": "WebKit",
"C_Cpp.clang_format_fallbackStyle": "WebKit",
2.1.2. vim & emacs
clang-format -style="{BasedOnStyle: webkit}" -i XX.c
2.2. golang
使用默认 gofmt
3. build environment and build (example of ubuntu)
3.1 build environment
- Install nix package management tool
sh <(curl -L https://nixos.org/nix/install) --daemon
- Edit /etc/nix/nix.conf to make sure
experimental-features = nix-command flakes
sandbox = true
sandbox-fallback = false
substituters = https://mirrors.ustc.edu.cn/nix-channels/store https://cache.nixos.org https://cache.nixos.org/
trusted-users = root <your account>
make sure that you have restarted nix-daemon service after you have changed /etc/nix/nix.conf
- Install socat and config ssh (for clone by ssh protocol)
nix profile add nixpkgs#socat
.ssh/config
Host gitea.shenrs.eu
HostName gitea.shenrs.eu
IdentityFile ~/.ssh/id_ed25519
ProxyCommand socat - openssl-connect:gitea.shenrs.eu:2022,snihost=gitea.shenrs.eu,verify=0
3.2 checkout source code
3.3 build (for package and release)
cd xce/src
# for build debug version
nix build --option sandbox false .#xcecore_debug
# for normal build
nix build --option sandbox false .#xcecore
You can find the generated binaries at =result= directory
3.4 build (for daily develop)
cd xce/src
nix develop
# using meson to generated build configuration (debug and address sanitize)
meson setup build --buildtype=debug -Db_sanitize=address
# run meson to build
meson compile -C build
# The generated files will be in build/
3.5 build AppImage package
cd xce/src
nix bundle --bundler github:ralismark/nix-appimage --option sandbox false .#xce
This will create all in one AppImage file which contains everything for package delivery.
4. misc
4.1. Windows remap ctrl and alt key
create a reg file, e.g. remapkey.reg
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Keyboard Layout]
"Scancode Map"=hex:00,00,00,00,00,00,00,00,05,00,00,00,38,00,1d,00,1d,00,38,00,38,e0,1d,e0,1d,e0,38,e0,00,00,00,00
then double click remapkey.reg to import it. Restart OS.