No description
Find a file
Rongsong Shen 40196be2e5 perf(raft-apply-decoupling): fix O(n^2) apply-pump WAL rescan, measure throughput (task 7.4)
Measured sustained worker-fact throughput against the pre-feature baseline.
Commit throughput is unregressed (~2.5% delta, within noise) and no per-fact
synchronous commit barrier exists, confirming Req 11.4's core claim. But
apply/observable throughput -- the new hop this spec introduces -- regressed
56% with clearly superlinear scaling (2x volume -> 5.2x elapsed).

Root cause: apply_pump_read_entry has no memory across calls -- every LSN
lookup re-lists WAL segments and rescans its segment from byte 0 -- while its
only caller applies committed entries one at a time in strict ascending
order, turning a sequential access pattern into O(n^2) total work.

Fixed with a minimal segment-read cache on persist_t (fd + offset + expected
next LSN, guarded by the existing apply_pump_lock): a fast path reads the
next expected LSN directly at a remembered offset, falling back unchanged to
full rediscovery on any miss. Correctness never depends on the cache, only
performance does. Invalidated on init, fini, recovery, and WAL truncation --
the four points a stale cursor could otherwise misread from.

Two review rounds: round 1 caught a missing lock guard in the recovery
invalidation path and an inaccurate risk claim about the cache/trim
interaction (a real, bounded behavior change deferred to the existing
APPLY_DIVERGENT fence, not "no worse than before") -- both corrected and
re-verified in round 2. Post-fix apply-throughput scales sublinearly (1.46x
elapsed for 2x volume) and exceeds the pre-feature baseline. Zero new test
failures across the pump, correctness-verification, recovery, and truncation
groups.

An intermittent, pre-existing client hang in the synchronous jobsub path and
a trim-floor/apply-lag architectural gap were surfaced but are explicitly
out of scope for this fix -- flagged for separate follow-up.
2026-07-07 07:14:15 +00:00
.claude fix(raft-apply-decoupling): kill-mid-fact-stream soak + close bug-1416 boot-time TOCTOU (task 7.2) 2026-07-07 03:03:40 +00:00
.kiro perf(raft-apply-decoupling): fix O(n^2) apply-pump WAL rescan, measure throughput (task 7.4) 2026-07-07 07:14:15 +00:00
.wolf fix(raft-apply-decoupling): kill-mid-fact-stream soak + close bug-1416 boot-time TOCTOU (task 7.2) 2026-07-07 03:03:40 +00:00
docs/superpowers/plans claude skills 2026-05-29 15:00:06 +08:00
ha/data chore: checkpoint all prior work for clean state 2026-06-04 13:04:16 +00:00
raft chore: checkpoint all prior work for clean state 2026-06-04 13:04:16 +00:00
reports fix(raft-apply-decoupling): kill-mid-fact-stream soak + close bug-1416 boot-time TOCTOU (task 7.2) 2026-07-07 03:03:40 +00:00
src perf(raft-apply-decoupling): fix O(n^2) apply-pump WAL rescan, measure throughput (task 7.4) 2026-07-07 07:14:15 +00:00
.clang-format reformat code and job array 2025-11-07 16:14:27 +08:00
.gitignore chore: clean buglog.json junk/duplicates, gitignore scratch soak output 2026-07-03 03:24:19 +00:00
.mcp.json checkpoint: bug-851 root cause + raft TLA+ conformance spec family 2026-06-21 14:02:15 +00:00
.opencode.json checkpoint: bug-851 root cause + raft TLA+ conformance spec family 2026-06-21 14:02:15 +00:00
CLAUDE.md chore: baseline snapshot — prior-session work + ha-consistency-invariants spec family 2026-06-11 13:21:50 +00:00
README.md save works 2026-03-03 16:49:26 +08:00
uninstall-code-review-graph.sh checkpoint: bug-851 root cause + raft TLA+ conformance spec family 2026-06-21 14:02:15 +00:00

1. commit 规则

  • 每个人 fork 自己的 branch, 从自己的 branch 做 PR,review 完再 commit

2. code format

2.1. C

推荐使用 clang-format,公共配置放在源码里面 xce/.clang-format

(理论上直接使用源码里面 xce/.clang-format 文件格式不会变)

2.1.1. vscode 可以在配置中加上如下内容

"C_Cpp.clang_format_style": "WebKit",
"C_Cpp.clang_format_fallbackStyle": "WebKit",

2.1.2. vim & emacs

clang-format -style="{BasedOnStyle: webkit}" -i XX.c

2.2. golang

使用默认 gofmt

3. build environment and build (example of ubuntu)

3.1 build environment

  • Install nix package management tool
sh <(curl -L https://nixos.org/nix/install) --daemon
  • Edit /etc/nix/nix.conf to make sure
experimental-features = nix-command flakes
sandbox = true
sandbox-fallback = false
substituters = https://mirrors.ustc.edu.cn/nix-channels/store https://cache.nixos.org https://cache.nixos.org/
trusted-users = root <your account>

make sure that you have restarted nix-daemon service after you have changed /etc/nix/nix.conf

  • Install socat and config ssh (for clone by ssh protocol)
nix profile add nixpkgs#socat

.ssh/config

Host gitea.shenrs.eu
	HostName gitea.shenrs.eu
	IdentityFile ~/.ssh/id_ed25519
	ProxyCommand socat - openssl-connect:gitea.shenrs.eu:2022,snihost=gitea.shenrs.eu,verify=0

3.2 checkout source code

3.3 build (for package and release)

cd xce/src

# for build debug version
nix build --option sandbox false .#xcecore_debug

# for normal build
nix build --option sandbox false .#xcecore

You can find the generated binaries at =result= directory

3.4 build (for daily develop)

cd xce/src
nix develop

# using meson to generated build configuration (debug and address sanitize)

meson setup build --buildtype=debug -Db_sanitize=address

# run meson to build
meson compile -C  build

# The generated files will be in build/

3.5 build AppImage package

cd xce/src
nix bundle --bundler github:ralismark/nix-appimage --option sandbox false .#xce

This will create all in one AppImage file which contains everything for package delivery.

4. misc

4.1. Windows remap ctrl and alt key

create a reg file, e.g. remapkey.reg

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Keyboard Layout]
"Scancode Map"=hex:00,00,00,00,00,00,00,00,05,00,00,00,38,00,1d,00,1d,00,38,00,38,e0,1d,e0,1d,e0,38,e0,00,00,00,00

then double click remapkey.reg to import it. Restart OS.