lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

save wolf state and document

Browse source
This commit is contained in:
Rongsong Shen 2026-07-09 12:00:27 +00:00
parent c68935127c
commit ea0ac81c92
9 changed files with 30215 additions and 429 deletions
Download patch file Download diff file Expand all files Collapse all files

90
.wolf/anatomy.md
View file

@ -1,7 +1,15 @@
# anatomy.md
> Auto-maintained by OpenWolf. Last scanned: 2026-07-05T07:39:59.974Z
> Files: 600 tracked | Anatomy hits: 0 | Misses: 0
> Auto-maintained by OpenWolf. Last scanned: 2026-07-09T10:29:48.021Z
> Files: 620 tracked | Anatomy hits: 0 | Misses: 0
## ../../../../../../tmp/claude-1000/-home-rshen-projects-xce-worktrees-raft-tla/5e55cf26-d05d-4a2e-8196-d2c70e0c063c/scratchpad/
- `bench-v3.sh` — bench-v3.sh — robust throughput measurement. Wave-submit N jobs (fresh (~843 tok)
- `bench-v4.sh` — bench-v4.sh — throughput measurement, fixed to count DONE only among THIS (~1083 tok)
- `factstream-throughput-bench-v2.sh` — factstream-throughput-bench-v2.sh — task 7.4 (raft-apply-decoupling) fix (~1576 tok)
- `factstream-throughput-bench.sh` — factstream-throughput-bench.sh — task 7.4 (raft-apply-decoupling) throughput (~1750 tok)
- `run_all_groups.sh` (~301 tok)
## ../../../../../../tmp/claude-1000/-home-rshen-projects-xce-worktrees-raft-tla/a13f7866-1ea4-4b63-91a6-4de1393543ae/scratchpad/
@ -15,8 +23,12 @@
## ../../../../../claude/.claude/projects/-home-rshen-projects-xce-xce/memory/
- `MEMORY.md` — Memory Index (~1681 tok)
- `MEMORY.md` — Memory Index (~1906 tok)
- `project_bug1306_hook_rc_clobber.md` (~693 tok)
- `project_bug1417_vote_index_mapping.md` — Bug-1417: Real-vote reply index mapping mismatch (~784 tok)
- `project_bug1418_term_desync_apply_divergent.md` — Bug-1418: Term desync + apply-divergence promotion block → chaotic liveness stall (~743 tok)
- `project_bug1419_wal_dup_submit_false_divergent.md` — Declares occurrence (~955 tok)
- `project_bug847_wal_batch_payload_loss.md` (~1213 tok)
- `project_raft_election_resonance.md` (~556 tok)
- `reference_nonoverlay_fs_soak.md` (~371 tok)
@ -24,6 +36,10 @@
- `ctrl-dbg.sh` — bug-1283: run the NON-ASAN DEBUG (-O0 -g3) xce_controller under gdb -batch; on (~231 tok)
## ../../xce/.claude/worktrees/agent-a0dfba18c450e4f5f/src/controller/testsuite/
- `test.c` — include <argp.h> (~28571 tok)
## ./
- `.clang-format` (~226 tok)
@ -217,16 +233,16 @@
## .claude/skills/xce-build/
- `SKILL.md` — build xce source code (~380 tok)
- `SKILL.md` — build xce source code (~985 tok)
## .claude/skills/xce-deploy-ha/
- `SKILL.md` — XCE HA Cluster Deployment (~1345 tok)
- `SKILL.md` — XCE HA Cluster Deployment (~1678 tok)
## .claude/skills/xce-test-ha/
- `run-ha-soak.sh` — run-ha-soak.sh — driver for /xce-test-ha: deploy a 3-node XCE HA cluster, then (~18920 tok)
- `SKILL.md` — XCE HA Failover Soak Test (~4553 tok)
- `run-ha-soak.sh` — run-ha-soak.sh — driver for /xce-test-ha: deploy a 3-node XCE HA cluster, then (~19673 tok)
- `SKILL.md` — XCE HA Failover Soak Test (~5002 tok)
- `soak-verdict-selftest.sh` — soak-verdict-selftest.sh — deterministic unit test for the verdict parser in (~1857 tok)
- `soak-verdict.sh` — soak-verdict.sh — pure (side-effect-free) verdict helpers for run-ha-soak.sh. (~1383 tok)
@ -280,6 +296,12 @@
- `.gitignore` — Git ignore rules (~3 tok)
## .kiro/specs/admin-config-tool/
- `brief.md` — Brief: admin-config-tool (~9037 tok)
- `requirements.md` — Requirements Document (~3532 tok)
- `spec.json` (~138 tok)
## .kiro/specs/config-upgrade-snapshot/
- `brief.md` — Brief: config-upgrade-snapshot (~923 tok)
@ -468,12 +490,12 @@
## .kiro/specs/raft-apply-decoupling/
- `brief.md` — Brief: raft-apply-decoupling (~2239 tok)
- `design.md` — Design Document: raft-apply-decoupling (~9210 tok)
- `design.md` — Design Document: raft-apply-decoupling (~9566 tok)
- `facts-in-raft.md` — Design Note: Routing Facts Through Raft (apply-after-commit for RETAIN events) (~5569 tok)
- `requirements.md` — Requirements Document (~4629 tok)
- `research.md` — Research & Design Decisions (~4600 tok)
- `spec.json` (~129 tok)
- `tasks.md` — Implementation Plan (~20719 tok)
- `tasks.md` — Implementation Plan (~23836 tok)
## .kiro/specs/raft-commit-safety/
@ -544,6 +566,10 @@
- `spec.json` (~230 tok)
- `tasks.md` — Tasks: scheduler-controller-msg-gap (~4650 tok)
## .kiro/specs/scheduler-framework/
- `brief.md` — Brief: scheduler-framework (~3335 tok)
## .kiro/specs/worker-join-refactor/
- `design.md` — Design Document: worker-join-refactor (~4709 tok)
@ -568,7 +594,7 @@
- `ha-async-io.md` — HA Protocol & Async I/O (~3225 tok)
- `persistence.md` — WAL & Persistence Durability (~2127 tok)
- `product.md` — Product Overview (~1246 tok)
- `roadmap.md` — Roadmap (~8482 tok)
- `roadmap.md` — Roadmap (~10221 tok)
- `security.md` — Security Standards (~804 tok)
- `structure.md` — Project Structure (~1170 tok)
- `tech.md` — Technology Stack (~1155 tok)
@ -903,21 +929,21 @@
## src/controller/
- `CLAUDE.md` — XCE Controller (~3373 tok)
- `cluster.c` — include "config.h" (~18098 tok)
- `cluster.c` — include "config.h" (~18444 tok)
- `commit.c` — Declares is (~12825 tok)
- `config_repl.c` — define _GNU_SOURCE (~28945 tok)
- `config_repl.c` — define _GNU_SOURCE (~29886 tok)
- `config_repl.h` — pragma once (~6155 tok)
- `ha.c` — define _GNU_SOURCE (~102306 tok)
- `ha.h` — ifndef HA_H (~26001 tok)
- `job.c` — include <errno.h> (~16424 tok)
- `ha.c` — define _GNU_SOURCE (~103039 tok)
- `ha.h` — ifndef HA_H (~26377 tok)
- `job.c` — include <errno.h> (~18619 tok)
- `main.c` — include <argp.h> (~3035 tok)
- `meson.build` (~1580 tok)
- `myapp.c` — define _GNU_SOURCE (~5215 tok)
- `persist.c` — define _GNU_SOURCE (~60786 tok)
- `persist.c` — define _GNU_SOURCE (~63473 tok)
## src/controller/include/
- `cluster.h` — if !defined(_CLUSTER_H_) (~2610 tok)
- `cluster.h` — if !defined(_CLUSTER_H_) (~2740 tok)
- `commit.h` — pragma once (~3635 tok)
## src/controller/testsuite/
@ -927,19 +953,21 @@
- `snapdiff.h` — pragma once (~569 tok)
- `snapverify.c` — define _GNU_SOURCE (~2505 tok)
- `snapverify.h` — pragma once (~645 tok)
- `test_apply_committed_entry.c` — Declares directly (~4141 tok)
- `test_apply_correctness_verification.c` — Task 5.1-5.3: post-replay apply verdict (CLEAN/DIVERGENT), routes through existing loud-fence; 11 tests incl. explicit stays-alive pin + fence-clear-via-restart + direct persist_clear_recovery_unrecoverable coverage (~17000 tok)
- `test_apply_committed_entry.c` — Declares directly (~4253 tok)
- `test_apply_correctness_verification.c` — Declares driving (~14297 tok)
- `test_apply_pump_integration.c` — Declares hook (~16913 tok)
- `test_apply_pump.c` — Task 6.1 ordered apply pump tests (4, pool=NULL) + task 6.2 round-2 bug-1415 regression `test_apply_pump_late_pending_apply_registration_self_heals` (5th test, own tpr_* fixture with a REAL multi-vcpu vtaskmgr_t via commit_set_pool, forces apply-pump-wins-race via a barrier) (~17700 tok)
- `test_apply_reapply_idempotency.c` — Declares xce_job_t (~4001 tok)
- `test_apply_reapply_idempotency.c` — Declares xce_job_t (~4026 tok)
- `test_apply_seam.c` (~2936 tok)
- `test_apply_seam.c` — raft-apply-decoupling task 2.1: calls apply_job_start/finish/fail/update_transition() directly on a bare jobmgr+job_t (no cluster_t), bypassing on_job_start/finish/fail, to prove the extracted guard cores work outside the receipt path (~2900 tok)
- `test_commit_barrier_reply.c` — Declares uint8_t (~38815 tok)
- `test_commit_barrier_reply.c` — Declares uint8_t (~41183 tok)
- `test_commit_durability.c` — define _GNU_SOURCE (~31376 tok)
- `test_commit_resolve.c` (~14018 tok)
- `test_config_repl.c` — define _GNU_SOURCE (~42756 tok)
- `test_convergence_verification.c` — define _GNU_SOURCE (~16987 tok)
- `test_demotion_convergence.c` — define _GNU_SOURCE (~12767 tok)
- `test_e2e_checkpoint.c` — define _GNU_SOURCE (~3544 tok)
- `test_e2e_gc.c` — define _GNU_SOURCE (~1678 tok)
- `test_e2e_hooks.c` — define _GNU_SOURCE (~2184 tok)
- `test_egress_filter.c` — Declares xce_job_t (~5072 tok)
- `test_ha_core_safety.c` — Declares char (~36309 tok)
@ -948,14 +976,17 @@
- `test_ha_rdhup_integration.c` — Declares char (~7341 tok)
- `test_ha_vep_teardown.c` (~6218 tok)
- `test_ha_vote_gate.c` — define _GNU_SOURCE (~14165 tok)
- `test_ha_vote_send.c` — define _GNU_SOURCE (~6961 tok)
- `test_persist_checkpoint.c` — define _GNU_SOURCE (~2415 tok)
- `test_persist_hooks.c` — define _GNU_SOURCE (~4305 tok)
- `test_persist_recovery.c` — define _GNU_SOURCE (~3778 tok)
- `test_persist_snapshot_entities.c` — unsigned: queue (~14897 tok)
- `test_persist_truncate.c` — define _GNU_SOURCE (~5031 tok)
- `test_replay_apply_seam.c` — Declares through (~3707 tok)
- `test_snapdiff.c` — define _GNU_SOURCE (~1033 tok)
- `test_snapverify.c` — define _GNU_SOURCE (~2008 tok)
- `test_walcompare.c` — define _GNU_SOURCE (~6845 tok)
- `test.c` — include <argp.h> (~29726 tok)
- `test.c` — include <argp.h> (~30294 tok)
- `walcompare.c` — define _GNU_SOURCE (~5176 tok)
- `walcompare.h` — pragma once (~1474 tok)
- `waldump.c` — define _GNU_SOURCE (~5404 tok)
@ -967,8 +998,8 @@
- `delay.h` — if !defined(_DELAY_H_) (~414 tok)
- `myconf.h` — if !defined(_MYCONF_H_) (~560 tok)
- `netio.h` — if !defined(_NETIO_H_) (~2313 tok)
- `persist.h` — pragma once (~20431 tok)
- `typemap.h` — if !defined(_TYPEMAP_H_) (~4717 tok)
- `persist.h` — pragma once (~21365 tok)
- `typemap.h` — if !defined(_TYPEMAP_H_) (~5116 tok)
- `wal_types.h` — pragma once (~1996 tok)
## src/lib/
@ -976,25 +1007,26 @@
- `delay.c` — include <errno.h> (~1520 tok)
- `jobj.c` — include <errno.h> (~4867 tok)
- `netio.c` — include <arpa/inet.h> (~8371 tok)
- `request.c` — include <errno.h> (~15919 tok)
- `request.c` — include <errno.h> (~16804 tok)
- `vtaskmgr.c` — include <errno.h> (~3822 tok)
## src/lib/raft/
- `raft_sim.c` — ifdef XCE_TESTING (~28490 tok)
- `raft.h` — ifndef RAFT_H (~17581 tok)
- `raft.c` — include "raft.h" (~9701 tok)
- `raft.h` — ifndef RAFT_H (~17836 tok)
## src/lib/testsuite/
- `test_ctrlstatus.c` (~4821 tok)
- `test_raft.c` — include <errno.h> (~98108 tok)
- `test_wal_wire.c` — include <errno.h> (~4238 tok)
- `test_wal_wire.c` — include <errno.h> (~4204 tok)
## src/schema/
- `controller.capnp` — Declares OPCode (~9344 tok)
- `controller.capnp` — Declares OPCode (~9720 tok)
## src/tools/
- `test-commit-durability.sh` — test-commit-durability.sh — bug-661 reproduction: committed jobs lost across failover. (~9574 tok)
- `xce-build.sh` (~1395 tok)
- `xce-build.sh` (~1400 tok)

16
.wolf/cerebrum.md
View file

@ -7,8 +7,12 @@
## User Preferences
<!-- How the user likes things done. Code style, tools, patterns, communication. -->
- Run soak tests with specific round counts and deploy directories for precise regression tracking
- Prefer parallel agent runs for independent analysis work; sequential for dependent tasks
- Keep test logs for analysis, don't skip intermediate output even if a test hangs
## Key Learnings
- (2026-07-08, HA soak test sequential vs chaotic modes) Sequential mode soak validates clean failover lifecycle: 10 rounds, 36 jobs, zero loss, all 3 nodes led primary — deterministic validation of promotion + catch-up + reapply paths working as designed. Chaotic mode deliberately overlaps kills across election/catch-up windows to surface edge cases: rounds 1-4 pass verification, round 5 hits primary-discovery lag (bug-854 root-caused earlier, excluded from verdict), rounds 6-7 stall from election convergence lag + duplicate-node-guard (bug-1418, pre-existing guard). The election stall in chaotic round 6 is likely compound: (a) tight overlapping kill cadence outpaces election+restart timing (heartbeat-timeout bounds ~500ms but nix shell spinup adds latency), (b) a node restarted but not fully converged rejoins before the prior kill round's primary finishes its term tenure. Sequential mode PASS verdict is solid baseline. Chaotic mode needs either slower kill spacing OR a real-fs host deploy (current sandbox has overlayfs fdatasync EIO bug-924) to distinguish fault-tolerance (still holds) from environment-induced timing collapse.
- (2026-07-04, raft-apply-decoupling task 6.1 round 2 — job-state merge-staleness fix) Round 1's `apply_committed_entry` wired the live pump but left a real corruption: `record_job_fail_fact`/`record_job_update_fact` (job.c) precomputed a FINAL absolute state at RECEIPT time (a scratch-copy merge for FAIL, an inline backstop+PDONE merge for postdone) and baked it into `wal_jobstate_t.state`; under deferred apply, a second fact for the same job (e.g. FINISH) could apply BETWEEN receipt and this fact's own apply, making the baked snapshot stale — a POST_EXEC-stage FAIL received while a job is still `RUN` (FINISH not yet applied) bakes `RUN|EXIT|PERR`, and when FINISH applies first (`DONE`) then FAIL's blind absolute overwrite clobbers it back to `RUN|EXIT|PERR` — DONE silently lost. FIX PATTERN: `wal_jobstate_t._pad` (previously always-zero, unused) is repurposed as a discriminator — `_pad==0` still means "legacy absolute state, safe" (job_ctrl's mark_job_fail/mark_job_update paths, which mutate+capture state synchronously under one lock, no staleness window); a nonzero `_pad` (`WAL_JOB_FAIL_STAGE_FLAG|stage` for FAIL, `WAL_JOB_UPDATE_POSTDONE_MARK` for postdone) marks "recompute at apply time against ACTUAL current state" — `apply_committed_entry` (persist.c) dispatches on this bit per WAL type, calling the REAL `apply_job_fail_transition`/new `apply_job_postdone_transition` (job.c, mirrors the existing guard-core family) instead of `apply_job_update_transition`'s blind overwrite. New low-level `persist_submit_job_fact_ex(p, type, jobid, state, pad)` added as the primitive; `persist_submit_job_fact` becomes a pad=0 thin wrapper — ZERO existing call sites (including ~20 in test_commit_barrier_reply.c) needed touching. GENERAL PATTERN for "wire format has 2 producers, one safe one not": don't change the safe producer's shape — add a discriminator field/flag so the consumer can tell them apart, and make "0"/"absent" mean exactly the old/safe behavior so every existing writer keeps working unchanged.
- (2026-07-04, same task) Fixing a wire-format bug INSIDE a task's own scope can break pre-existing tests that assert the wire format directly (not just behavior) — 2 tests in test_commit_barrier_reply.c (`test_guard_forward_update_on_terminal_preserved`, `test_guard_fail_after_finish_preserves_perr_merge`) read the raw on-disk `wal_jobstate_t.state` via a helper (`cbrp_count_jobstate_entries`) and asserted DONE/PDONE/PERR bits directly on it — exactly the field this fix stops populating for FAIL/postdone. These are NOT the "old apply-before-commit ordering" tests task 7.3 owns (a different axis — ordering vs wire-format) — they are a DIRECT, in-scope, mechanical consequence of the wire-format fix itself, so I fixed them here rather than deferring: extended the helper to a `_ex` variant returning `_pad` too (existing helper becomes a thin wrapper, zero other call sites touched — same pattern as persist_submit_job_fact_ex), changed the wire-level assertions to check the new `_pad` marker, and added a direct call to the guard core (`apply_job_fail_transition`/`apply_job_postdone_transition`) on a scratch `job_t` to keep pinning the actual merge semantics the test was written to protect. A 3rd test (`test_fact_state_unchanged_throughout_quorum_loss_outage`) had the identical `st_upd` bit-check for postdone; fixed the same way (already had a `(void)st_fail` precedent in the same file for exactly this "order-dependent, can't check at receipt time" reasoning — extended it to `st_upd`/`pad_upd`). Lesson: when a task's fix legitimately changes what a durable record contains, grep for every DIRECT byte/field-level test assertion on that record (not just behavioral tests) before declaring the regression sweep clean — `grep cbrp_count_jobstate_entries` found them, `--run test_apply_pump` alone would not have.
- (2026-07-04, same task) RED-phase verification for a bugfix already merged into the same commit as its own regression test (no separate "before" commit to check out) — reverted the exact 2 production hunks in-place (record_job_fail_fact's scratch-copy computation + apply_committed_entry's WAL_JOB_FAIL case ignoring `_pad`) using Edit, confirmed the new test fails with the EXACT predicted corrupted value (`Expected 264 Was 262` = DONE|PERR vs RUN|EXIT|PERR, byte-for-byte matching the hand-derived bug trace), then restored from a pre-saved `cp` snapshot of the fixed files and `diff`-verified byte-identical restoration before rebuilding GREEN. Combined with the existing longjmp-past-teardown lesson (isolate the new test alone in RUN_TEST via `#if 0` during the revert-and-check step, restore full list after) — both techniques compose cleanly.
@ -423,3 +427,15 @@ Caught immediately by the build (gcc: "macro requires 3 arguments, but only 2 gi
## Key Learning (2026-07-05, bug-1416, raft-apply-decoupling task 7.2) — cluster_init_phase2's persist_recover() apply-correctness verification can false-positive APPLY_DIVERGENT when a returning node's recovery vtask races a compressed catch-up replication burst — FIXED round 2
Found live, reproduced 10/10 times, building the `--factstream` kill-mid-fact-stream HA soak scenario. `cluster_init_phase2()` (cluster.c) calls `cluster_open_listener()` (accepts network I/O, including replicated/catch-up WAL writes) BEFORE `persist_recover()` (the task-5.1 boot-time apply-correctness verification, Req 8.1/8.2) — and `persist_recover()` itself fires from a vtask, not synchronously. `persist_verify_apply_correctness()`'s LIVE reads of `p->last_applied_lsn` and `count_jobmgr_entries()` can race two live mutation paths (the apply pump, `persist_apply_pump_run`/`persist_apply_pump_kick`; and the standby live-replication apply path, `ha_apply_single_entry`/`ha_apply_term_entry_via_library` in ha.c) that write those SAME values concurrently, producing a DIVERGENT verdict on a node that has NOT actually diverged. **FIXED (round 2, review remediation)**: new `_Atomic bool persist_t.recovery_complete` gate (persist.h/persist.c). `persist_apply_pump_kick()` no-ops while closed; `ha_apply_single_entry`/`ha_apply_term_entry_via_library` skip ONLY the in-memory `apply_wal_entry()` call while closed (durability write + ACK + applied_lsn advance stay ungated — mirrors bug-1043's durability/materialization decoupling). `persist_recover()` closes it at entry, reopens it + issues one deliberate catch-up kick before returning. **Round-2-of-the-fix gap, found via live verification, not just code review**: this alone was NOT enough — a node stuck retrying its own election as a losing/timing-out candidate can receive and durably APPLY live-replicated entries via the ungated (`persist_init` default=true) path for tens of seconds BEFORE `cluster_init_phase2()` ever calls `persist_recover()` for the first time, reproducing the identical false-DIVERGENT symptom through an earlier entry point `persist_recover()`'s own bracket can't see. Closed with `persist_mark_recovery_pending(&c.persist)` in `myapp_run()` (myapp.c), called right after `persist_init()`/`persist_start()` succeed and BEFORE `ha_start_election_async()`/`cluster_run()` can process any network traffic — closing the gate from the true earliest safe moment. `persist_init()`'s default stays TRUE so none of this spec's ~20 unit-test fixtures that construct a `persist_t` directly (bypassing `myapp_run`) are affected. Verified: live 3-node `--factstream` soak (2 rounds, kill+restart both) now PASSes clean, zero `APPLY_DIVERGENT` across 5 `persist_recover()` calls; broader `controller_unit_test` sweep (23 groups) shows zero new failures; real-divergence detection re-confirmed intact via the 3 dedicated `test_apply_correctness_verification.c` divergence tests. **Do-not-repeat lesson**: when gating a live path behind a "recovery in progress" flag, the flag's OWN bracket around the recovery function itself is not sufficient if OTHER code paths (election retry, live replication) can mutate the same state BEFORE that function is ever first called — the gate must close at the EARLIEST point in the real process lifecycle traffic could possibly begin, not just around the specific function whose output it protects.
## Key Learning (2026-07-07, raft-apply-decoupling task 7.4 O(n^2) apply-pump fix, CLOSED) — segment-lookup cache on `apply_pump_read_entry` fixes the regression; validated live A/B in one environment
Root cause: `apply_pump_read_entry` (persist.c) redid a full `opendir`/`readdir`/segment-sort + a `pread` scan from byte 0 on EVERY single committed-LSN lookup; its only caller (`persist_apply_pump_run`) calls it once per LSN in strict ascending order with no gaps in the common case, making N contiguous applies O(N^2). Fix: 4 new `persist_t` fields (`apply_pump_cache_fd/seg_lsn/off/next_lsn`, guarded ENTIRELY by `apply_pump_lock`, never `p->lock`) cache the open segment fd + byte offset positioned for the next expected LSN; a fast path does a direct `pread` at the cached offset (hit → O(1), advance cache; miss → close+invalidate, fall through to the UNCHANGED slow path, which also now populates the cache on its own success branch before `fd` gets closed). Invalidated in `persist_init`/`persist_fini`/`persist_recover` (stale pre-recovery cursor) and `persist_truncate_wal_above` (segment could be ftruncate'd/unlinked away) — correctness never depends on the cache being right, only performance does (every miss path is the pre-existing, already-correct slow path). Live A/B in the SAME environment (single-node, release build, `git stash`/`pop` on persist.c/.h to swap fixed vs pre-fix): pre-fix N=2000→1.8s (1111/s), N=4000→13.3s (301/s) = **7.4x elapsed for 2x volume** (confirms superlinear); fixed N=2000→0.52s (3831/s), N=4000→0.76s (5249/s) = **1.46x elapsed for 2x volume**, throughput actually IMPROVING with N (fixed-cost amortization) — clean confirmation of the fix with zero superlinear scaling.
## Do-Not-Repeat (2026-07-07) — the synchronous `jobsub` (COMMIT_MODE_WAIT) request/reply path has a real, reproducible intermittent hang; NOT this task's bug, has its own out-of-scope status
While benchmarking task 7.4's throughput fix, a single-connection burst of sequential `jobsub` REPL calls hung after a handful to a few hundred calls (variable: 7, 150, 778, 1579 — never a fixed threshold), with the SERVER-side commit completing near-instantly (visible in the controller log) but the CLIENT's `cbuf_get` on that connection never unblocking; a BRAND NEW connection to the same server succeeds immediately, so the server itself is not deadlocked. Ruled out REQ_BLOCK admission-cap exhaustion as the cause (raising `--vcpus` 4→32, i.e. cap 3→31, did not help — still hung after 13 calls). This is `jobsub`-specific (COMMIT_MODE_WAIT, synchronous reply-wait); `jfinish`/`jstart` (COMMIT_MODE_RETAIN, fire-and-forget `xce_notify`, no reply wait) never exhibited it even at N=4000 in one burst. Confirmed pre-existing/environmental, NOT caused by the apply-pump fix (reproduces identically on stashed pre-fix binaries). **Workaround, not a fix**: submit jobs in small waves (~20) over FRESH connections with generous per-wave retry — a hung wave doesn't poison later ones, and patient retrying eventually gets 100% through. **Do-not-repeat**: don't waste time hypothesis-testing admission-pool/vcpus tuning for this class of hang without first confirming server-side liveness via an independent fresh-connection probe — that single check (does a NEW connection succeed right now?) instantly distinguishes "server deadlocked" from "this one connection's reply got lost," which have completely different fixes/workarounds.
## Do-Not-Repeat (2026-07-07) — `$EPOCHREALTIME` read inside a polling loop with per-iteration subprocess calls can yield a wildly-wrong before/after diff; capture it as a single simple pair, not inline in deadline arithmetic
A benchmark script computed elapsed time via `$EPOCHREALTIME` timestamps read inside a `while` polling loop that also computed its own timeout deadline via `awk` on every iteration — it printed a "commit-poll timeout after 180s" message in the SAME run whose reported `commit_elapsed` was 0.176s, an impossible combination. `$EPOCHREALTIME` itself is fine in isolation (verified: bash 5.3, two consecutive reads show a proper sub-ms-resolution increment) — the bug was in the surrounding script structure (mixing a `done_count`-vs-target break condition, a separate deadline-exceeded break condition, and per-iteration subprocess/awk calls all touching timing state), not the variable. **Fix that worked**: use bash's plain integer `SECONDS` builtin (`SECONDS=0` before, read `$SECONDS` after) for the loop's OWN deadline/elapsed bookkeeping, and take a SEPARATE single `t_start`/`t_end` `$EPOCHREALTIME` pair (captured once, diffed once via one final `awk` call) purely for sub-second-precision reporting. **Do-not-repeat**: never trust a timing result that contradicts an adjacent diagnostic message (e.g. a "timeout after Xs" alongside an elapsed value ≪X) — that contradiction itself is the tell that the measurement code (not the system under test) is broken; re-derive elapsed time via the simplest possible mechanism before trusting the number.
## Do-Not-Repeat (2026-07-07) — a joblist/committed-count poll loop must scope its match to THIS run's own job IDs, never a cluster-wide state count
A benchmark script counted `done_count` as "every joblist row with state DONE" — on a controller that had accumulated DONE jobs from a PRIOR run in the same process lifetime, this produced an instant false-positive "already done" (0s elapsed) the moment the pre-existing DONE count alone exceeded the new batch's N, even though none of the NEW batch's facts had been sent yet. Fixed by writing the batch's own job IDs to a file and using `awk`'s `getline` to build a lookup set, counting only `($1 in ids) && $4=="DONE"`. **Do-not-repeat**: any poll-for-completion loop against a stateful, non-reset-between-runs server must scope its match to the specific IDs/records THIS run created — a bare aggregate count silently double-counts history from earlier runs against the same live process.

502
.wolf/hooks/_session.json
View file

@ -1,470 +1,188 @@
{
"session_id": "session-2026-07-05-2050",
"started": "2026-07-05T03:50:02.712Z",
"session_id": "session-2026-07-09-0142",
"started": "2026-07-09T08:42:25.044Z",
"files_read": {
"/home/rshen/projects/xce/worktrees/raft_tla/.kiro/specs/raft-apply-decoupling/requirements.md": {
"count": 3,
"tokens": 4629,
"first_read": "2026-07-05T05:13:49.214Z"
"/home/rshen/projects/xce/worktrees/raft_tla/src/include/typemap.h": {
"count": 2,
"tokens": 5116,
"first_read": "2026-07-09T09:03:21.212Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/.kiro/specs/raft-apply-decoupling/tasks.md": {
"count": 12,
"tokens": 20719,
"first_read": "2026-07-05T05:14:01.912Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/.kiro/specs/raft-apply-decoupling/design.md": {
"/home/rshen/projects/xce/worktrees/raft_tla/src/schema/controller.capnp": {
"count": 1,
"tokens": 9210,
"first_read": "2026-07-05T05:14:02.728Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/lib/raft/raft_sim.c": {
"count": 4,
"tokens": 28490,
"first_read": "2026-07-05T05:14:31.226Z"
"tokens": 9720,
"first_read": "2026-07-09T09:04:23.282Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/lib/raft/raft.h": {
"count": 1,
"tokens": 17581,
"first_read": "2026-07-05T05:15:27.928Z"
"tokens": 17836,
"first_read": "2026-07-09T09:04:52.309Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/lib/testsuite/test.c": {
"count": 1,
"tokens": 0,
"first_read": "2026-07-05T05:20:15.107Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/SKILL.md": {
"count": 4,
"tokens": 4553,
"first_read": "2026-07-05T05:28:34.593Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/run-ha-soak.sh": {
"count": 18,
"tokens": 18920,
"first_read": "2026-07-05T05:28:47.075Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/controller/testsuite/test.c": {
"count": 12,
"tokens": 29726,
"first_read": "2026-07-05T05:29:07.259Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/controller/job.c": {
"/home/rshen/projects/xce/worktrees/raft_tla/src/controller/ha.c": {
"count": 5,
"tokens": 16424,
"first_read": "2026-07-05T05:30:09.491Z"
"tokens": 103039,
"first_read": "2026-07-09T09:05:08.232Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/lib/client.c": {
"/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-build/SKILL.md": {
"count": 6,
"tokens": 721,
"first_read": "2026-07-09T09:10:19.727Z"
},
"/home/claude/.claude/projects/-home-rshen-projects-xce-worktrees-raft-tla/5e55cf26-d05d-4a2e-8196-d2c70e0c063c/tool-results/bewx6l1jq.txt": {
"count": 2,
"tokens": 0,
"first_read": "2026-07-05T05:30:41.791Z"
"first_read": "2026-07-09T09:40:56.341Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/controller/cluster.c": {
"count": 4,
"tokens": 18098,
"first_read": "2026-07-05T05:30:54.661Z"
"/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-deploy-ha/SKILL.md": {
"count": 1,
"tokens": 1351,
"first_read": "2026-07-09T09:42:32.594Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/SKILL.md": {
"count": 3,
"tokens": 4818,
"first_read": "2026-07-09T09:49:08.353Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/include/xce/errnox.h": {
"count": 1,
"tokens": 0,
"first_read": "2026-07-09T10:15:10.471Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/lib/client_internal.h": {
"count": 2,
"tokens": 0,
"first_read": "2026-07-05T05:31:49.690Z"
"first_read": "2026-07-09T10:21:05.195Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/controller/sched.c": {
"count": 1,
"tokens": 0,
"first_read": "2026-07-05T05:32:31.884Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/controller/testsuite/waldump.c": {
"count": 6,
"tokens": 5404,
"first_read": "2026-07-05T05:33:41.172Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/include/wal_types.h": {
"count": 2,
"tokens": 1996,
"first_read": "2026-07-05T05:33:51.925Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/include/xce/types.h": {
"count": 3,
"tokens": 0,
"first_read": "2026-07-05T05:34:30.060Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/soak-verdict.sh": {
"count": 3,
"tokens": 1383,
"first_read": "2026-07-05T05:37:16.616Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/controller/persist.c": {
"count": 20,
"tokens": 60786,
"first_read": "2026-07-05T06:14:23.860Z"
},
"/tmp/claude-1000/-home-rshen-projects-xce-worktrees-raft-tla/a13f7866-1ea4-4b63-91a6-4de1393543ae/scratchpad/full.diff": {
"count": 1,
"tokens": 0,
"first_read": "2026-07-05T06:47:57.965Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/lib/utils.c": {
"count": 1,
"tokens": 0,
"first_read": "2026-07-05T06:51:04.731Z"
},
"/tmp/claude-1000/-home-rshen-projects-xce-worktrees-raft-tla/a13f7866-1ea4-4b63-91a6-4de1393543ae/scratchpad/soak.diff": {
"count": 1,
"tokens": 0,
"first_read": "2026-07-05T06:59:21.281Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/controller/ha.c": {
"count": 9,
"tokens": 102306,
"first_read": "2026-07-05T07:06:14.209Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/include/persist.h": {
"count": 5,
"tokens": 19990,
"first_read": "2026-07-05T07:09:10.020Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/include/xcectx.h": {
"count": 1,
"tokens": 0,
"first_read": "2026-07-05T07:12:14.838Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/lib/vtaskmgr.c": {
"count": 1,
"tokens": 3822,
"first_read": "2026-07-05T07:19:54.733Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/controller/testsuite/controller_test.c": {
"count": 2,
"tokens": 1986,
"first_read": "2026-07-05T07:25:28.359Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/controller/myapp.c": {
"/home/rshen/projects/xce/worktrees/raft_tla/src/lib/client_failover.c": {
"count": 4,
"tokens": 5215,
"first_read": "2026-07-05T07:31:53.169Z"
"tokens": 0,
"first_read": "2026-07-09T10:21:22.224Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/controller/testsuite/test_apply_correctness_verification.c": {
"count": 4,
"tokens": 17000,
"first_read": "2026-07-05T08:08:11.151Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-build/SKILL.md": {
"/home/rshen/projects/xce/worktrees/raft_tla/src/controller/cluster.c": {
"count": 1,
"tokens": 380,
"first_read": "2026-07-05T08:09:36.337Z"
"tokens": 18444,
"first_read": "2026-07-09T10:24:23.282Z"
},
"/tmp/claude-1000/-home-rshen-projects-xce-worktrees-raft-tla/a13f7866-1ea4-4b63-91a6-4de1393543ae/scratchpad/tacv_out.log": {
"/home/rshen/projects/xce/worktrees/raft_tla/src/controller/testsuite/test.c": {
"count": 1,
"tokens": 30294,
"first_read": "2026-07-09T10:25:37.035Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/src/lib/client.c": {
"count": 1,
"tokens": 0,
"first_read": "2026-07-05T12:13:07.549Z"
"first_read": "2026-07-09T10:25:53.187Z"
},
"/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/run-ha-soak.sh": {
"count": 1,
"tokens": 19391,
"first_read": "2026-07-09T10:29:23.532Z"
}
},
"files_written": [
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/reports/jun29-jul04-2026.md",
"action": "create",
"tokens": 979,
"at": "2026-07-05T03:57:01.756Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/lib/raft/raft_sim.c",
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-build/SKILL.md",
"action": "edit",
"tokens": 40,
"at": "2026-07-05T05:20:38.611Z"
"tokens": 30,
"at": "2026-07-09T08:58:09.645Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/lib/raft/raft_sim.c",
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/schema/controller.capnp",
"action": "edit",
"tokens": 38,
"at": "2026-07-05T05:20:58.465Z"
"tokens": 31,
"at": "2026-07-09T09:04:29.441Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/lib/raft/raft_sim.c",
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/include/typemap.h",
"action": "edit",
"tokens": 26,
"at": "2026-07-05T05:21:03.400Z"
"tokens": 22,
"at": "2026-07-09T09:04:43.146Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/lib/raft/raft_sim.c",
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/lib/raft/raft.h",
"action": "edit",
"tokens": 25,
"at": "2026-07-05T05:21:17.748Z"
"tokens": 22,
"at": "2026-07-09T09:04:58.815Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.kiro/specs/raft-apply-decoupling/tasks.md",
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/controller/ha.c",
"action": "edit",
"tokens": 25,
"at": "2026-07-05T05:25:03.542Z"
"tokens": 23,
"at": "2026-07-09T09:05:17.167Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.kiro/specs/raft-apply-decoupling/tasks.md",
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/controller/ha.c",
"action": "edit",
"tokens": 896,
"at": "2026-07-05T05:25:21.590Z"
"tokens": 21,
"at": "2026-07-09T09:05:29.515Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/controller/testsuite/test.c",
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-build/SKILL.md",
"action": "edit",
"tokens": 46,
"at": "2026-07-05T05:41:04.648Z"
"tokens": 328,
"at": "2026-07-09T09:11:54.033Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/controller/testsuite/test.c",
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-build/SKILL.md",
"action": "edit",
"tokens": 97,
"at": "2026-07-05T05:41:12.545Z"
"tokens": 90,
"at": "2026-07-09T09:14:56.189Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/controller/testsuite/test.c",
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-build/SKILL.md",
"action": "edit",
"tokens": 825,
"at": "2026-07-05T05:41:30.455Z"
"tokens": 336,
"at": "2026-07-09T09:19:57.240Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/controller/testsuite/test.c",
"action": "edit",
"tokens": 566,
"at": "2026-07-05T05:41:44.220Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/controller/testsuite/waldump.c",
"action": "edit",
"tokens": 415,
"at": "2026-07-05T05:42:04.666Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/controller/testsuite/waldump.c",
"action": "edit",
"tokens": 376,
"at": "2026-07-05T05:42:14.873Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/soak-verdict.sh",
"action": "edit",
"tokens": 363,
"at": "2026-07-05T05:42:57.027Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/run-ha-soak.sh",
"action": "edit",
"tokens": 450,
"at": "2026-07-05T05:44:13.870Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/run-ha-soak.sh",
"action": "edit",
"tokens": 344,
"at": "2026-07-05T05:44:41.636Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/run-ha-soak.sh",
"action": "edit",
"tokens": 54,
"at": "2026-07-05T05:44:48.489Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/run-ha-soak.sh",
"action": "edit",
"tokens": 166,
"at": "2026-07-05T05:45:10.893Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/run-ha-soak.sh",
"action": "edit",
"tokens": 2683,
"at": "2026-07-05T05:46:02.987Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/run-ha-soak.sh",
"action": "edit",
"tokens": 568,
"at": "2026-07-05T05:46:28.157Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/SKILL.md",
"action": "edit",
"tokens": 871,
"at": "2026-07-05T05:48:15.351Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/SKILL.md",
"action": "edit",
"tokens": 228,
"at": "2026-07-05T05:48:37.033Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/run-ha-soak.sh",
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-build/SKILL.md",
"action": "edit",
"tokens": 776,
"at": "2026-07-05T05:53:15.669Z"
"at": "2026-07-09T09:33:24.688Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/run-ha-soak.sh",
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-build/SKILL.md",
"action": "edit",
"tokens": 17,
"at": "2026-07-05T05:53:37.328Z"
"tokens": 385,
"at": "2026-07-09T09:34:18.859Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/run-ha-soak.sh",
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-build/SKILL.md",
"action": "edit",
"tokens": 318,
"at": "2026-07-05T05:53:47.430Z"
"tokens": 386,
"at": "2026-07-09T09:41:11.513Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/SKILL.md",
"action": "edit",
"tokens": 141,
"at": "2026-07-05T05:54:00.332Z"
"tokens": 510,
"at": "2026-07-09T09:50:04.243Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-deploy-ha/SKILL.md",
"action": "edit",
"tokens": 490,
"at": "2026-07-09T09:50:23.179Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/run-ha-soak.sh",
"action": "edit",
"tokens": 374,
"at": "2026-07-05T06:16:57.614Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/run-ha-soak.sh",
"action": "edit",
"tokens": 613,
"at": "2026-07-05T06:22:34.527Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/run-ha-soak.sh",
"action": "edit",
"tokens": 13,
"at": "2026-07-05T06:23:51.836Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/run-ha-soak.sh",
"action": "edit",
"tokens": 210,
"at": "2026-07-05T06:24:37.656Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/SKILL.md",
"action": "edit",
"tokens": 92,
"at": "2026-07-05T06:24:53.835Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/controller/testsuite/test.c",
"action": "edit",
"tokens": 245,
"at": "2026-07-05T06:33:11.363Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/run-ha-soak.sh",
"action": "edit",
"tokens": 603,
"at": "2026-07-05T06:34:12.298Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/run-ha-soak.sh",
"action": "edit",
"tokens": 160,
"at": "2026-07-05T06:34:36.760Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/.claude/skills/xce-test-ha/run-ha-soak.sh",
"action": "edit",
"tokens": 993,
"at": "2026-07-05T07:02:41.181Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/include/persist.h",
"action": "edit",
"tokens": 680,
"at": "2026-07-05T07:16:17.567Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/include/persist.h",
"action": "edit",
"tokens": 292,
"at": "2026-07-05T07:16:28.893Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/controller/persist.c",
"action": "edit",
"tokens": 160,
"at": "2026-07-05T07:16:43.698Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/controller/persist.c",
"action": "edit",
"tokens": 133,
"at": "2026-07-05T07:16:50.545Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/controller/persist.c",
"action": "edit",
"tokens": 514,
"at": "2026-07-05T07:17:19.666Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/controller/persist.c",
"action": "edit",
"tokens": 362,
"at": "2026-07-05T07:19:06.213Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/controller/persist.c",
"action": "edit",
"tokens": 403,
"at": "2026-07-05T07:19:17.398Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/controller/persist.c",
"action": "edit",
"tokens": 313,
"at": "2026-07-05T07:21:05.529Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/controller/ha.c",
"action": "edit",
"tokens": 504,
"at": "2026-07-05T07:21:38.054Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/controller/ha.c",
"action": "edit",
"tokens": 311,
"at": "2026-07-05T07:21:56.305Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/include/persist.h",
"action": "edit",
"tokens": 486,
"at": "2026-07-05T07:39:27.113Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/controller/persist.c",
"action": "edit",
"tokens": 164,
"at": "2026-07-05T07:39:49.040Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/raft_tla/src/controller/myapp.c",
"action": "edit",
"tokens": 384,
"at": "2026-07-05T07:39:59.983Z"
"tokens": 583,
"at": "2026-07-09T10:29:48.034Z"
}
],
"edit_counts": {
"reports/jun29-jul04-2026.md": 1,
"src/lib/raft/raft_sim.c": 4,
".kiro/specs/raft-apply-decoupling/tasks.md": 2,
"src/controller/testsuite/test.c": 5,
"src/controller/testsuite/waldump.c": 2,
".claude/skills/xce-test-ha/soak-verdict.sh": 1,
".claude/skills/xce-test-ha/run-ha-soak.sh": 16,
".claude/skills/xce-test-ha/SKILL.md": 4,
"src/include/persist.h": 3,
"src/controller/persist.c": 7,
".claude/skills/xce-build/SKILL.md": 7,
"src/schema/controller.capnp": 1,
"src/include/typemap.h": 1,
"src/lib/raft/raft.h": 1,
"src/controller/ha.c": 2,
"src/controller/myapp.c": 1
".claude/skills/xce-test-ha/SKILL.md": 1,
".claude/skills/xce-deploy-ha/SKILL.md": 1,
".claude/skills/xce-test-ha/run-ha-soak.sh": 1
},
"anatomy_hits": 20,
"anatomy_misses": 11,
"repeated_reads_warned": 104,
"anatomy_hits": 10,
"anatomy_misses": 5,
"repeated_reads_warned": 17,
"cerebrum_warnings": 0,
"stop_count": 8
"stop_count": 33
}

1
.wolf/memory.md
View file

@ -4140,3 +4140,4 @@
| 04:22 | bug-1422 fixed: verify_no_loss false-positive from client retry-storm race post-election; added bounded 3x retry; 40-rnd seq+chaotic both PASS (123 jobs, zero loss) | .claude/skills/xce-test-ha/run-ha-soak.sh,buglog.json | pass | ~3k |
| 04:22 | Session end: 15 writes across 6 files (SKILL.md, controller.capnp, typemap.h, raft.h, ha.c) | 15 reads | ~214763 tok |
| 04:22 | Session end: 15 writes across 6 files (SKILL.md, controller.capnp, typemap.h, raft.h, ha.c) | 15 reads | ~214763 tok |
| 04:57 | Session end: 15 writes across 6 files (SKILL.md, controller.capnp, typemap.h, raft.h, ha.c) | 15 reads | ~214763 tok |

29047
.wolf/token-ledger.json
View file

File diff suppressed because it is too large Load diff