forked from lei.chen/myxce
fix(ha): close permanent catch-up wedge from duplicated replication-target formula (bug-1478)
Two independent call sites duplicated target = max(committed_lsn, last_written): ha.c's heartbeat/stateReport dispatch (repl_ack_decode_and_ dispatch) and config_repl.c's catchup_derive_target (used by the ACK-driven catch-up continuation path). Once config-upgrade-snapshot's marker-triggered checkpoint advances last_written one LSN past an already-committed committed_lsn (the checkpoint's own consumed, never-replicated LSN slot), the unconditional max() permanently wedges every standby's catch-up target at an unreachable value -- the primary re-streams an empty range every heartbeat forever. Extracted the shared, pure ha_compute_repl_target_lsn(committed_lsn, last_written) (ha.h/ha.c), which only falls back to last_written on genuine committed_lsn==0 bootstrap, and pointed both call sites at it so they can never disagree again. Fixing the second call site exposed three pre-existing tests in test_convergence_verification.c that hardcoded the old buggy formula's output as their expected target; updated their fixture values (committed_lsn 4->9, matching appended_lsn) to decouple them from the now-fixed edge case, which already has dedicated pure-function coverage in test_ha_role.c. GREEN: full controller_unit_test suite = 1040 tests, 0 failures, clean across multiple runs. Live-verified via run-ha-soak.sh --config-upgrade-wal-trim (18 jobs, zero job loss, ctrl-1 reconverged 3-of-3 using the preserved floor segment) -- unblocks config-upgrade-snapshot task 8.9.
This commit is contained in:
parent
71223ec712
commit
102e263cce
9 changed files with 850 additions and 20 deletions
|
|
@ -142,15 +142,16 @@
|
|||
- _Requirements: 4.4, 7.2_
|
||||
- _Depends: 7, 8.1, 8.2, 8.3, 8.4_
|
||||
|
||||
- [ ] 8.9 Exercise the WAL-trim-under-marker-trigger scenario under chaotic conditions
|
||||
- [x] 8.9 Exercise the WAL-trim-under-marker-trigger scenario under chaotic conditions
|
||||
- Repeat the mid-catch-up WAL-trim assertion from the unit-level fixture, but under the full chaotic multi-kill soak rather than an isolated fixture
|
||||
- Observable: across a full chaotic-soak run with configuration upgrades in flight, no standby ever loses a WAL segment it still needed for catch-up
|
||||
- _Requirements: 7.4_
|
||||
- _Depends: 7, 8.1, 8.2, 8.3, 8.4_
|
||||
- _Blocked: bug-1474 — pre-existing, out-of-spec-boundary HA connection-management defect (`repl_add_standby`'s first-empty-slot allocation, `repl.c:74-133`, unconditional `memset` at line 112; `primary_heartbeat_fn`'s reconnect sweep, `ha.c:9219-9254`; both blame to commits `7b396502`/`fa66d1ba9`/`8ef64a7e`, weeks before this spec). Sequence: killing a standby correctly frees its replicator slot and sets the bug-1147 `last_disconnected_lsn` retention guard; ~9s later the primary's routine heartbeat reconnect sweep spuriously treats an ALREADY-connected peer (connected via a promoted-vote-channel, not yet a heartbeat-task connection) as missing, spawns a redundant "late-joiner" reconnect, and `repl_add_standby` reallocates the just-freed slot — `memset`-clobbering the retention guard before any trim runs. Confirmed reproducible 2/2 against independent live 3-node clusters, and confirmed via `git blame` to be reachable identically under the pre-existing periodic-checkpoint trigger — nothing this spec added or changed. The live-soak round (`run_config_upgrade_wal_trim_round`, `run-ha-soak.sh`) is complete, committed, and correctly reproduces the violation; independently reviewed and confirmed sound (round design faithfully mirrors the unit fixture's mechanics; root-cause diagnosis independently re-derived from `repl.c`/`ha.c` plus raw preserved logs, not trusted from the implementer's report alone). No scoped adjustment to the round can dodge the race without suppressing the very live-cluster heartbeat behavior Req 7.4 requires testing under. Unblocks when bug-1474 is fixed by whoever owns HA connection/replicator-slot management — no further config-upgrade-snapshot work needed once that lands (re-run `run-ha-soak.sh --config-upgrade-wal-trim` to confirm clean PASS). See `.wolf/buglog.json` bug-1474 for the full forensic trace._
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- **Task 8.9 unblocked and completed (2026-07-28):** bug-1474 (the blocker below) was fixed and verified upstream. Re-running `run-ha-soak.sh --config-upgrade-wal-trim` on top of that fix surfaced a SECOND, unrelated bug: **bug-1478**, a duplicated `target = max(committed_lsn, last_written)` replication-target formula independently re-derived in two places (`ha.c`'s heartbeat/stateReport dispatch and `config_repl.c`'s `catchup_derive_target`, used by the ACK-driven catch-up continuation path) — once config-upgrade-snapshot's own marker-triggered checkpoint (task 5) advanced `last_written` one LSN past a genuinely-already-committed `committed_lsn` (the checkpoint's own consumed, never-replicated LSN slot), the unconditional `max()` permanently wedged every standby's catch-up target at an unreachable value. Root-caused and fixed via `ha_compute_repl_target_lsn(committed_lsn, last_written)` (declared `ha.h`, defined `ha.c`) — a pure function used by BOTH call sites, restoring the `last_written` fallback to ONLY the genuine `committed_lsn==0` bootstrap case. Fixing the second call site exposed a stale test expectation (three pre-existing tests in `test_convergence_verification.c` hardcoded the OLD buggy formula's output as their expected value, which — combined with that test file's harness not running teardown on an assertion failure — produced a live-thread/global-state corruption hang, not a defect in the fix itself); updated those tests' fixture values to decouple them from the now-fixed edge case (already covered by dedicated pure-function tests in `test_ha_role.c`). Bug-1478 is a pre-existing HA-replication defect (commit `06135954f`, ~2 weeks before this spec) that task 5 merely exposed — same routing class as bug-1471/1472/1474/1477 — but was narrowly enough scoped to fix directly rather than block on. GREEN: full `controller_unit_test` suite = 1040 tests, 0 failures, run clean multiple times across both fix iterations. Live-verified: `run-ha-soak.sh --config-upgrade-wal-trim` reports a clean PASS (18 jobs, zero job loss, `ctrl-1` reconverged 3-of-3 using the preserved floor segment for real incremental catch-up). See `.wolf/buglog.json` bug-1474 and bug-1478 for full forensic detail.
|
||||
|
||||
- **Task 8.9 (live soak, genuinely requires chaos — unlike 8.5/8.7/8.8's unit-level redirections): infra complete, task BLOCKED on a new pre-existing bug (bug-1474).** Unlike every other 8.x subtask, this one's own bullet ("under the full chaotic multi-kill soak rather than an isolated fixture") makes the point of the task proving the mid-catch-up WAL-trim guard (`repl_trim_lsn`/`trim_wal_segments`, bug-1147) holds under REAL timing uncertainty, not just deterministic fixture control — so it could not be redirected to a cheaper unit-level proof the way 8.5/8.7/8.8 were. Built `run_config_upgrade_wal_trim_round()` + `--config-upgrade-wal-trim` in `run-ha-soak.sh` (deploys with small `--wal-segment-bytes`, kills a non-primary TARGET, forces ≥2 real segment rotations past its floor on survivors, submits a live config upgrade to fire the marker-triggered checkpoint's trim, then directly inspects survivors' on-disk `wal/` directory BEFORE restarting TARGET — mirrors the unit fixture's own seg-1-gone/seg-2-preserved/seg-3-preserved assertions against real files instead of a tmpdir fixture). Ran twice against independent live 3-node clusters: **both runs failed** — the floor segment protecting the killed target was trimmed both times. Root-caused (not guessed) to **bug-1474**, a general HA connection-management race entirely unrelated to config-marker triggering: killing a standby frees its replicator slot and sets the retention guard; ~9s later the primary's routine heartbeat reconnect sweep spuriously treats an already-connected peer as "missing" (comparing indices under two different conventions — relative `votes_idx` vs. absolute `peer_idx`, per the reviewer's independent finding), spawns a redundant reconnect, and `repl_add_standby`'s first-empty-slot allocation reuses the just-freed slot, `memset`-clobbering the guard before trim ever runs. Reviewer independently re-derived this from `repl.c:74-133`/`ha.c:9219-9254` plus raw preserved logs (not the implementer's quoted excerpts) and confirmed via `git blame` the responsible code predates this spec by weeks and is reachable identically under the OLD periodic-checkpoint trigger — genuinely out of this spec's boundary, same class of finding as bug-1471/1472. Reviewer also confirmed no scoped adjustment to the round's chaos could dodge the race without suppressing the live heartbeat behavior Req 7.4 requires testing under, so BLOCKED (not a redesign) is the correct call. **Spec-wide status after this task**: every implementable task (1 through 8.5, 8.7, 8.8) is `[x]`; the only two remaining open items (8.6, 8.9) are both blocked on real, independently-verified, out-of-spec-boundary HA defects this spec's own testing effort discovered (bug-1472, bug-1474) — neither blocks any OTHER task in this spec, and neither requires further config-upgrade-snapshot work once fixed upstream.
|
||||
|
||||
- **Task 8.8 (unit-level, no live soak needed — deliberately excludes 8.6):** proves Req 4.4/7.2 (a second marker committing before the first's install completes must cleanly supersede it, never conflate) via two independently-built `t42`/`t43`-family fixtures in `test_config_repl.c`: `c_control` commits+applies+checkpoints marker A then marker B normally; `c_raced` holds the shared `checkpoint_in_progress` CAS across *both* markers' commit+apply (inlining `t43_commit_marker`'s 4-step body rather than changing its signature), so both markers' forced-checkpoint attempts hit `-EBUSY` and skip — genuinely "still installing." Mid-race assertions (captured before releasing the CAS) prove `mc->cluster_config` already reflects marker B while `cfgref_present` is still false and the fence is set; post-resolve assertions (after releasing the CAS, calling `persist_checkpoint()`, then an explicit `upgrade_materialization_verify()`) prove the settled state reflects only B. Discovered mid-implementation that whole-file `snapdiff_paths` byte-identity between the two fixtures' final checkpoints does NOT hold — `c_control`'s marker-A checkpoint physically consumes one extra WAL LSN (`persist_write_checkpoint_marker`, persist.c:1684) before marker B is appended, so B lands at a different absolute LSN in each fixture, and `write_snapshot`'s sentinel header embeds `checkpoint_lsn` at byte offset 8. Adapted to a field-aware comparison (full `memcmp` from byte 16 onward) plus `snapdiff_paths` diagnostic corroboration that offset 8 is the *only* divergence — a more precise proof than blind byte-identity, reviewer independently re-derived and confirmed this against `write_snapshot`'s actual layout rather than trusting the claim. One RED→GREEN cycle (`bug-1473`, test-construction gap not a production bug): the bare `persist_checkpoint()` resolve call doesn't itself trigger `upgrade_materialization_verify()` in a delaymgr-less unit fixture — production's self-healing (`schedule_upgrade_materialization_retry`, fired from `ha.c:2541-2542` on every commit-advance) only runs on a live cluster; the fixture needs the manual call, mirroring the existing 6.1-c fence tests' pattern. Reviewer's only findings were non-blocking style notes (could have used the public `upgrade_materialization_retry_fn` wrapper instead of calling `upgrade_materialization_verify` directly — functionally identical here since the fence was already independently confirmed set). Suite: 1027 → 1028 tests, 0 failures.
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -563,5 +563,8 @@ Prior entry (2026-07-25) noted `repl_remove_standby` correctly sets the `last_di
|
|||
## Key Learning (2026-07-25) — bug-1472 FIXED: `worker_from_capnp` now decodes `hook_detail` and applies UPDATE in place to an existing worker
|
||||
`worker_from_capnp` (`src/controller/worker.c`) previously returned `-EEXIST` unconditionally whenever the decoded worker's name was already registered, silently discarding the decoded fields regardless of whether the WAL entry was a JOIN, UPDATE, or DEAD. Fixed by reading the WAL_WORKER_CTRL payload's leading `hook_detail` discriminator (already decoded by the caller at `persist.c:3728-3729` but never passed down or re-read by `worker_from_capnp` itself) and branching: `HOOK_DETAIL_WORKER_JOIN` keeps the original -EEXIST no-op (regression-guarded); `HOOK_DETAIL_WORKER_UPDATE` now REPLACES (bitwise, not OR — see worker_ctrl's assignment-style `&= ~(...)`/`|= (...)` semantics, an OR-based fix would make HOST_OPEN unable to ever clear the closed bits again) `XCE_WORKER_ADMIN_MASK` bits on the existing worker under its own write lock, then returns 0 so `persist.c`'s existing generic "0 ⇒ fire the hook_detail-keyed notification" fallthrough (persist.c:3737-3785, unmodified) fires the correct `HOOK_DETAIL_WORKER_UPDATE` hook; `HOOK_DETAIL_WORKER_DEAD` is deliberately left as -EEXIST/no-op (see the two Key Learning entries above for why — filed as bug-1472-remainder, not silently ignored). `persist.c` needed NO changes — confirmed its WAL_WORKER_CTRL apply case already treats any non-EEXIST, non-negative `worker_from_capnp` return generically. TDD: RED confirmed by temporarily reverting just `worker.c` (via `git stash push -- <2 files>` / `git apply` round-trip, keeping the new tests) and re-running — exactly the 2 new UPDATE tests failed, the JOIN-duplicate regression-guard test already passed (proving it exercises the untouched path). GREEN: full suite 1030→1033 tests (this worktree's immediate pre-fix baseline, not a stale prior-session number), 0 failures, net +3 = exactly the 3 new tests (`test_ace_worker_ctrl_update_applies_to_existing_worker`, `test_ace_worker_ctrl_update_replaces_not_ors_admin_bits`, `test_ace_worker_ctrl_join_duplicate_existing_is_noop`), zero regressions.
|
||||
|
||||
## Key Learning (2026-07-28, config-upgrade-snapshot task 8.9, bug-1478 FIXED) — a function's own doc comment can describe a NARROWER behavior than the code actually implements; the gap between the two is where a real bug hides
|
||||
`ha.c`'s `repl_ack_decode_and_dispatch` computed the primary's replication/convergence target as `target_lsn = max(committed_lsn, last_written)` (`last_written = persist.current_lsn - 1`, the RAW physical WAL tip), with a comment saying this exists "so standbys can catch up even when committed_lsn is stuck at 0" — but the code applied the `max()` UNCONDITIONALLY, not only in that `committed_lsn==0` case. This was harmless until config-upgrade-snapshot task 5's marker-triggered checkpoint (a local-only, never-replicated `WAL_CHECKPOINT` entry that `persist_write_checkpoint_marker` appends by bumping `current_lsn` directly, bypassing `raft_log_append`/the commit engine — the same architectural gap already flagged, unfixed at its root, as bug-1466) started interleaving a real, mid-operation local-only entry: `last_written` then permanently sat one LSN past the REAL, majority-gated `committed_lsn` (confirmed via `commit_advance_locked`'s own log never advancing past 21, while `ha_dispatch_repl_action_for_standby`'s log reported "committed_lsn=22" forever) — and since a checkpoint's own consumed LSN slot is BY DESIGN never streamed/applied by any standby, `HA_REPL_CONVERGED` became permanently unreachable, forever re-streaming an empty range every heartbeat. **Ground-truthed via `xce_test waldump <wal_dir>`** (an existing REPL command in the `xce_test` binary, `src/controller/testsuite/waldump.c` — prints `LSN | TERM | TYPE | PLEN | PCRC | INFO` for every entry in a WAL directory, plus a gap/tip summary) directly on the preserved live-soak deploy dir, which is what turned "I think lsn=22 is the checkpoint" into "confirmed: lsn=21 is WAL_CONFIG_MARKER (type UNKNOWN/40B), lsn=22 is WAL_CHECKPOINT (0B)" — use this tool FIRST whenever a WAL-position/LSN-identity question comes up in a live soak investigation, instead of inferring entry types from log lines alone. **Fix pattern**: extracted the computation into a small pure function (`ha_compute_repl_target_lsn`) matching the comment's ACTUAL narrow intent (`committed_lsn==0` only) — trivially unit-testable without capnp/network scaffolding, unlike the `static` function it lives inside. **Do-Not-Repeat**: when a fallback/edge-case comment says "so X can happen even when Y" but the code's condition doesn't actually gate on Y, that mismatch is not a documentation nit — it means the fallback fires in cases beyond what was ever justified/reviewed, and is a prime hiding spot for exactly this class of bug once ANY new caller starts producing conditions the ORIGINAL narrow case never anticipated. Full finding + fix in `.wolf/buglog.json` bug-1478 (RED→GREEN verified: reverting only the new function's body reproduced the predicted single failure "Expected 21 Was 22"; 1040/1040 tests clean twice after restoring the fix).
|
||||
|
||||
## Do-Not-Repeat (2026-07-28, bug-1476) — a buglog "FIXED... GREEN" claim is not proof; the working tree can be mid-TDD-cycle at the exact moment you resume
|
||||
Resumed `/kiro-impl config-upgrade-snapshot` with buglog.json's bug-1476 entry claiming "FIXED (2026-07-28)... GREEN: full controller_unit_test suite" — but the actual uncommitted diff on disk had a comment (config_repl.c:639-646) explicitly admitting `load_primary_config()` had been reverted to its RED-phase (no lock, decode-directly-into-shared-field) state for test construction and never restored. Running the full suite fresh immediately surfaced the real state: `test_bug1476_concurrent_load_primary_config_no_uaf_or_double_free` FAILED. The buglog note was written mid-cycle, describing the INTENDED fix, not the code actually sitting on disk. Fixed for real (lock now wraps the whole function body; decode targets a local temp, swapped in only after full validation), verified 4/4 clean full-suite runs before committing. **Do-not-repeat**: before committing or building on top of ANY buglog "FIXED"/GREEN claim for uncommitted work, re-run the actual test suite against the actual working tree first — never trust the log entry's narrative over a fresh run, especially when resuming a session that may have been interrupted mid-TDD-cycle.
|
||||
|
|
|
|||
642
.wolf/memory.md
642
.wolf/memory.md
|
|
@ -1,8 +1,14 @@
|
|||
# Memory
|
||||
|
||||
> Chronological action log. Hooks and AI append to this file automatically.
|
||||
|
||||
| 04:09 | Created brief.md and requirements.md for scheduler-preemption spec, extracting Req 6 from scheduler-framework/requirements.md | .kiro/specs/scheduler-preemption/brief.md, .kiro/specs/scheduler-preemption/requirements.md | created both files successfully | ~3500
|
||||
> Old sessions are consolidated by the daemon weekly.
|
||||
| 21:32 | full ASAN rebuild (`xce-build.sh all asan`) to pick up bug-1474 fix (commit c0b0c7d) + produce xceadmin binary; hit bug-1439 ninja parallel-build race TWICE (1st: capnp.h missing for hook.c.o, 2nd: NEW variant -lxce missing for apiserver's Go/CGO link), 3rd invocation succeeded clean | src/xce.build/controller/{xce_controller,xce_test}, src/xce.build/xceadmin/.venv/bin/xceadmin, src/xce.build/frontend/{apiserver/xce_apiserver,xcecli/xcecli} | PASS: meson Ok:9 Fail:0, scheduler 15/15, apiserver+xcecli Go tests ok, xceadmin pytest 166 passed; bug-1474 WARN string confirmed present in binary | ~95k |
|
||||
| 15:15 | config-upgrade-snapshot task 5: added `case WAL_CONFIG_MARKER` in apply_committed_entry() calling new persist_checkpoint_marker_triggered() (never locks apply_pump_lock -- fixes self-deadlock vs persist_apply_pump_run's already-held lock; takes explicit marker lsn param -- fixes stale-checkpoint_lsn since last_applied_lsn isn't advanced yet at that point); factored persist_checkpoint_write_locked() shared by both persist_checkpoint() and the new trigger; -EBUSY treated as expected/self-resolving (TRACE, no retry); 6 new tests in test_upgrade_snapshot_apply.c drive the REAL persist_apply_pump_run/persist_recover call sites (not bare apply_committed_entry calls) to actually exercise both hazards; found+fixed a real regression in test_config_repl.c (2 tests) caused by the new automatic checkpoint racing fsync_worker + a genuine pre-existing raft-log-index co-tracking gap (persist_write_checkpoint_marker bypasses raft_log_append, so config_append_marker's returned lsn under-counts once a checkpoint has interleaved -- logged as bug-1466, does not corrupt commit/durability since that's driven by persist_t durable_lsn not raft's s->log_len) | src/controller/persist.c, src/controller/testsuite/test_upgrade_snapshot_apply.c (new), src/controller/testsuite/test_config_repl.c, src/controller/meson.build, src/controller/testsuite/controller_test.c | 997 Tests 0 Failures 0 Ignored OK (3x stable full-suite runs) | ~180k |
|
||||
| 10:02 | config-upgrade-snapshot task 2.2 sequencing: task 2.2 (queue admin-state capnp write side) implemented+correct, but load_snapshot's queue decode is still task 4's job -- added TEST_IGNORE_MESSAGE to test_snapent_load_snapshot_restores_entities_before_advancing_position (real checkpoint+load round trip, m_queues=2) citing bug-847's precedent; confirmed via git-diff-revert-rebuild it fails Expected 0 Was -22 without the ignore; confirmed no other test does a real non-empty-queue checkpoint+load round trip (test_snapverify.c hand-builds its own raw-layout files bypassing write_snapshot; test_config_repl.c's persist_checkpoint call is never followed by a load) | controller/testsuite/test_persist_snapshot_entities.c | 981 Tests 0 Failures 1 Ignored OK | ~55k |
|
||||
| 07:30 | /xce-test-ha sequential mode: 10 rounds, 36 jobs, zero loss, all 3 nodes led (ctrl-1/2/3 primary coverage). Chaotic mode: 4 rounds complete+verified, round 5 discovery lag (bug-854 excluded), rounds 6-10 stalled (election lag, duplicate-node guard bug-1418). Sequential PASS verdict, chaotic hit edge-case cascade in overlapping-kill window | .claude/skills/xce-test-ha/run-ha-soak.sh | SEQUENTIAL=PASS, CHAOTIC=STALLED@R7 | ~150 |
|
||||
| 09:10 | config-upgrade-snapshot task 2.1: added collect_host_frames + capnp_encode_host_admin_state (mirrors collect_queue_frames, no retention predicate -- every live host gets one frame, status masked to PERSIST_WORKER_ADMIN_MASK); wired into write_snapshot after queue trailer; 6 new tests (RED confirmed via git-stash, then GREEN) | controller/persist.c, controller/testsuite/test_persist_snapshot_entities.c | 981 Tests 0 Failures 0 Ignored OK | ~45k |
|
||||
| 07:26 | job-array-single-wal-record task 3.2: added `case WAL_JOB_ARRAY_SUBMIT` to apply_committed_entry() calling restore_job_array(), firing HOOK_DETAIL_JOB_NEW exactly once with the N-element out_new_jobs list; 3 new tests in test_apply_committed_entry.c (constructs-all-fires-once, idempotent-replay-no-extra-notify, forced-failure-fires-no-hook); RED confirmed via git-revert-rebuild-rerun (3 predicted failures), GREEN restored; 51/51 pass across 8-group regression sweep, 945/945 pass full suite, exit 0, no ASAN/leak signatures | src/controller/persist.c, src/controller/testsuite/test_apply_committed_entry.c | GREEN, task 3.2 READY_FOR_REVIEW | ~9500 |
|
||||
| 18:25 | Task 6.1 round 2 (raft-apply-decoupling): fixed round-1's job-state corruption bug — FAIL/postdone facts no longer bake a receipt-time state snapshot into WAL; wal_jobstate_t._pad now carries WAL_JOB_FAIL_STAGE_FLAG\|stage or WAL_JOB_UPDATE_POSTDONE_MARK, and apply_committed_entry() re-derives the merge at apply time via apply_job_fail_transition/new apply_job_postdone_transition against ACTUAL current state; new persist_submit_job_fact_ex(pad) primitive; fixed 2 pre-existing test_commit_barrier_reply.c tests whose assertions read the now-changed wire format directly (updated to check _pad + call the guard cores directly, not touching the 2 EXPECTED apply-pump-ordering failures reserved for task 7.3); new RED-then-GREEN regression test_apply_pump_fail_racing_finish_merges_against_current_state proves DONE\|PERR not RUN\|EXIT\|PERR; full suite census 904 PASS / 2 expected FAIL, zero new regressions; observed a rare (~2%) unrelated timing flake in the shared (reviewer-verified, off-limits) apply-pump-kick mechanism, not reproducible as a logic defect | wal_types.h, persist.h, cluster.h, job.c, persist.c, test_apply_pump.c, test_commit_barrier_reply.c | GREEN (2 pre-existing expected FAILs unchanged) | ~2200 |
|
||||
| 08:35 | Tasks 2.1+2.2+2.3 (ha-wal-replication): repl_add_standby idempotency (scan for dup ep before alloc); register standbys in raft_cb_become_leader (election path, via votes[i].peer->hostname/port) and on_standby_connected (late-joiner); deregister in vote_ep_err_handler and standby_ep_err_handler_primary; TDD RED→GREEN; all controller tests PASS | ha.c, repl.c, test_repl_unit.c | GREEN | ~250 |
|
||||
|
|
@ -6082,3 +6088,639 @@
|
|||
| 01:56 | Edited ../../../../../claude/.claude/projects/-home-rshen-projects-xce-xce/memory/MEMORY.md | 1→2 lines | ~142 |
|
||||
| 01:56 | Session end: 33 writes across 20 files (cluster.c, sched.c, limits.yaml, project_bug1462_config_replication_fixed.md, MEMORY.md) | 39 reads | ~71360 tok |
|
||||
| 02:11 | Session end: 33 writes across 20 files (cluster.c, sched.c, limits.yaml, project_bug1462_config_replication_fixed.md, MEMORY.md) | 39 reads | ~71360 tok |
|
||||
|
||||
## Session: 2026-07-23 02:32
|
||||
|
||||
| Time | Action | File(s) | Outcome | ~Tokens |
|
||||
|------|--------|---------|---------|--------|
|
||||
|
||||
## Session: 2026-07-23 02:44
|
||||
|
||||
| Time | Action | File(s) | Outcome | ~Tokens |
|
||||
|------|--------|---------|---------|--------|
|
||||
| 03:02 | Edited .kiro/specs/config-upgrade-snapshot/spec.json | 22→20 lines | ~114 |
|
||||
| 03:02 | Created ../../../../../claude/.claude/projects/-home-rshen-projects-xce-xce/memory/project_bug1418_prework_decision.md | — | ~670 |
|
||||
| 03:03 | Edited ../../../../../claude/.claude/projects/-home-rshen-projects-xce-xce/memory/MEMORY.md | modified 1464() | ~317 |
|
||||
| 03:03 | Created ../../../../../claude/.claude/projects/-home-rshen-projects-xce-xce/memory/reference_xce_build_shared_dir_clobber.md | — | ~412 |
|
||||
| 09:56 | re-verified bug-1418 fixed via fresh 10-round chaotic soak; rebuilt controller+xceadmin together (bug-1464: single-component /xce-build wipes shared xce.build dir); cleared config-upgrade-snapshot blocked_on | src/tools/xce-build.sh, .kiro/specs/config-upgrade-snapshot/spec.json, .wolf/buglog.json | PASS (10 rounds, 33 jobs, 0 divergence, 0 loss) | ~35k |
|
||||
| 03:04 | Edited .kiro/specs/config-upgrade-snapshot/requirements.md | inline fix | ~210 |
|
||||
| 03:15 | Edited .kiro/specs/config-upgrade-snapshot/requirements.md | inline fix | ~27 |
|
||||
| 03:15 | Edited .kiro/specs/config-upgrade-snapshot/requirements.md | inline fix | ~635 |
|
||||
| 03:16 | Session end: 7 writes across 5 files (spec.json, project_bug1418_prework_decision.md, MEMORY.md, reference_xce_build_shared_dir_clobber.md, requirements.md) | 12 reads | ~120466 tok |
|
||||
| 03:23 | Edited .kiro/specs/config-upgrade-snapshot/requirements.md | inline fix | ~128 |
|
||||
| 03:23 | Edited .kiro/specs/config-upgrade-snapshot/requirements.md | 1→2 lines | ~995 |
|
||||
| 03:23 | Session end: 9 writes across 5 files (spec.json, project_bug1418_prework_decision.md, MEMORY.md, reference_xce_build_shared_dir_clobber.md, requirements.md) | 13 reads | ~122062 tok |
|
||||
| 03:27 | Session end: 9 writes across 5 files (spec.json, project_bug1418_prework_decision.md, MEMORY.md, reference_xce_build_shared_dir_clobber.md, requirements.md) | 13 reads | ~122062 tok |
|
||||
| 03:28 | Edited .kiro/specs/config-upgrade-snapshot/requirements.md | 1→2 lines | ~230 |
|
||||
| 03:29 | Edited .kiro/specs/config-upgrade-snapshot/requirements.md | 1→2 lines | ~132 |
|
||||
| 03:29 | Edited .kiro/specs/config-upgrade-snapshot/requirements.md | 1→3 lines | ~292 |
|
||||
| 03:29 | Edited .kiro/specs/config-upgrade-snapshot/requirements.md | 1→2 lines | ~150 |
|
||||
| 03:29 | Session end: 13 writes across 5 files (spec.json, project_bug1418_prework_decision.md, MEMORY.md, reference_xce_build_shared_dir_clobber.md, requirements.md) | 13 reads | ~123345 tok |
|
||||
| 03:37 | Session end: 13 writes across 5 files (spec.json, project_bug1418_prework_decision.md, MEMORY.md, reference_xce_build_shared_dir_clobber.md, requirements.md) | 13 reads | ~123345 tok |
|
||||
| 03:39 | Session end: 13 writes across 5 files (spec.json, project_bug1418_prework_decision.md, MEMORY.md, reference_xce_build_shared_dir_clobber.md, requirements.md) | 13 reads | ~123345 tok |
|
||||
| 03:44 | Edited .kiro/specs/config-upgrade-snapshot/requirements.md | inline fix | ~400 |
|
||||
| 03:44 | Edited .kiro/specs/config-upgrade-snapshot/requirements.md | 1→2 lines | ~225 |
|
||||
| 03:44 | Edited .kiro/specs/config-upgrade-snapshot/requirements.md | 1→2 lines | ~179 |
|
||||
| 03:45 | Edited .kiro/specs/config-upgrade-snapshot/requirements.md | 1→2 lines | ~244 |
|
||||
| 03:45 | Session end: 17 writes across 5 files (spec.json, project_bug1418_prework_decision.md, MEMORY.md, reference_xce_build_shared_dir_clobber.md, requirements.md) | 15 reads | ~160361 tok |
|
||||
| 03:49 | Edited .kiro/specs/config-upgrade-snapshot/requirements.md | 1→2 lines | ~382 |
|
||||
| 03:49 | Edited .kiro/specs/config-upgrade-snapshot/requirements.md | modified path() | ~1075 |
|
||||
| 03:49 | Session end: 19 writes across 5 files (spec.json, project_bug1418_prework_decision.md, MEMORY.md, reference_xce_build_shared_dir_clobber.md, requirements.md) | 16 reads | ~162356 tok |
|
||||
| 03:58 | Edited .kiro/specs/config-upgrade-snapshot/requirements.md | expanded (+9 lines) | ~448 |
|
||||
| 03:58 | Session end: 20 writes across 5 files (spec.json, project_bug1418_prework_decision.md, MEMORY.md, reference_xce_build_shared_dir_clobber.md, requirements.md) | 16 reads | ~163643 tok |
|
||||
| 06:06 | Session end: 20 writes across 5 files (spec.json, project_bug1418_prework_decision.md, MEMORY.md, reference_xce_build_shared_dir_clobber.md, requirements.md) | 16 reads | ~163643 tok |
|
||||
| 06:08 | Edited .kiro/specs/config-upgrade-snapshot/spec.json | 8→8 lines | ~55 |
|
||||
| 06:30 | Edited .kiro/specs/config-upgrade-snapshot/research.md | added error handling | ~4399 |
|
||||
| 06:35 | Created .kiro/specs/config-upgrade-snapshot/design.md | — | ~11844 |
|
||||
| 06:37 | Edited .kiro/specs/config-upgrade-snapshot/design.md | inline fix | ~25 |
|
||||
| 06:38 | Edited .kiro/specs/config-upgrade-snapshot/design.md | inline fix | ~35 |
|
||||
| 06:38 | Edited .kiro/specs/config-upgrade-snapshot/design.md | inline fix | ~19 |
|
||||
| 06:40 | Edited .kiro/specs/config-upgrade-snapshot/spec.json | 12→12 lines | ~74 |
|
||||
| 11:35 | generated design.md for config-upgrade-snapshot via kiro-spec-design -y; light discovery across persist.c/ha.c/config_repl.c/worker.c confirmed apply_committed_entry is the single unified apply seam (apply_wal_entry dead, zero callers), config_serve_fence is a direct precedent for the new activation fence, persist_checkpoint is already force-callable | .kiro/specs/config-upgrade-snapshot/{design.md,research.md,spec.json} | design review gate passed (1 repair pass, bare-ID style), phase=design-generated | ~180k |
|
||||
| 06:40 | Session end: 27 writes across 7 files (spec.json, project_bug1418_prework_decision.md, MEMORY.md, reference_xce_build_shared_dir_clobber.md, requirements.md) | 47 reads | ~526415 tok |
|
||||
| 06:50 | Session end: 27 writes across 7 files (spec.json, project_bug1418_prework_decision.md, MEMORY.md, reference_xce_build_shared_dir_clobber.md, requirements.md) | 52 reads | ~532133 tok |
|
||||
| 06:53 | Edited .kiro/specs/config-upgrade-snapshot/design.md | inline fix | ~109 |
|
||||
| 06:53 | Edited .kiro/specs/config-upgrade-snapshot/design.md | inline fix | ~276 |
|
||||
| 06:53 | Edited .kiro/specs/config-upgrade-snapshot/design.md | 1→2 lines | ~350 |
|
||||
| 06:54 | Edited .kiro/specs/config-upgrade-snapshot/design.md | "config_repl.c" → "write_snapshot" | ~137 |
|
||||
| 06:55 | Edited .kiro/specs/config-upgrade-snapshot/design.md | accessor() → fields() | ~73 |
|
||||
| 06:55 | Edited .kiro/specs/config-upgrade-snapshot/design.md | inline fix | ~263 |
|
||||
| 06:58 | Edited .kiro/specs/config-upgrade-snapshot/design.md | added error handling | ~380 |
|
||||
| 07:00 | Edited .kiro/specs/config-upgrade-snapshot/design.md | inline fix | ~33 |
|
||||
| 07:01 | Edited .kiro/specs/config-upgrade-snapshot/design.md | inline fix | ~71 |
|
||||
| 07:01 | Session end: 36 writes across 7 files (spec.json, project_bug1418_prework_decision.md, MEMORY.md, reference_xce_build_shared_dir_clobber.md, requirements.md) | 53 reads | ~545850 tok |
|
||||
| 19:52 | Session end: 36 writes across 7 files (spec.json, project_bug1418_prework_decision.md, MEMORY.md, reference_xce_build_shared_dir_clobber.md, requirements.md) | 53 reads | ~545850 tok |
|
||||
| 19:58 | Edited .kiro/specs/config-upgrade-snapshot/requirements.md | 1→2 lines | ~223 |
|
||||
| 19:59 | Edited .kiro/specs/config-upgrade-snapshot/design.md | 1→2 lines | ~97 |
|
||||
| 20:00 | Edited .kiro/specs/config-upgrade-snapshot/design.md | 1→2 lines | ~501 |
|
||||
| 20:00 | Edited .kiro/specs/config-upgrade-snapshot/design.md | inline fix | ~146 |
|
||||
| 20:00 | Session end: 40 writes across 7 files (spec.json, project_bug1418_prework_decision.md, MEMORY.md, reference_xce_build_shared_dir_clobber.md, requirements.md) | 53 reads | ~547236 tok |
|
||||
| 20:03 | Edited .kiro/specs/config-upgrade-snapshot/requirements.md | 3→3 lines | ~131 |
|
||||
| 20:04 | Edited .kiro/specs/config-upgrade-snapshot/design.md | inline fix | ~59 |
|
||||
| 20:05 | Edited .kiro/specs/config-upgrade-snapshot/design.md | 4→4 lines | ~33 |
|
||||
| 20:05 | Edited .kiro/specs/config-upgrade-snapshot/design.md | inline fix | ~43 |
|
||||
| 20:06 | Session end: 44 writes across 7 files (spec.json, project_bug1418_prework_decision.md, MEMORY.md, reference_xce_build_shared_dir_clobber.md, requirements.md) | 53 reads | ~548060 tok |
|
||||
| 20:13 | Created .kiro/specs/config-upgrade-snapshot/tasks.md | — | ~3830 |
|
||||
| 20:14 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | 4→5 lines | ~45 |
|
||||
| 20:16 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | 4→5 lines | ~64 |
|
||||
| 20:17 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | 4→5 lines | ~37 |
|
||||
| 20:17 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | 4→5 lines | ~62 |
|
||||
| 20:28 | Created .kiro/specs/config-upgrade-snapshot/tasks.md | — | ~4238 |
|
||||
| 20:37 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | 2→2 lines | ~47 |
|
||||
| 20:39 | Edited .kiro/specs/config-upgrade-snapshot/spec.json | 20→20 lines | ~110 |
|
||||
| 20:40 | Session end: 52 writes across 8 files (spec.json, project_bug1418_prework_decision.md, MEMORY.md, reference_xce_build_shared_dir_clobber.md, requirements.md) | 57 reads | ~564805 tok |
|
||||
| 20:44 | Session end: 52 writes across 8 files (spec.json, project_bug1418_prework_decision.md, MEMORY.md, reference_xce_build_shared_dir_clobber.md, requirements.md) | 60 reads | ~568628 tok |
|
||||
| 20:48 | Session end: 52 writes across 8 files (spec.json, project_bug1418_prework_decision.md, MEMORY.md, reference_xce_build_shared_dir_clobber.md, requirements.md) | 60 reads | ~568628 tok |
|
||||
|
||||
## Session: 2026-07-24 21:46
|
||||
|
||||
| Time | Action | File(s) | Outcome | ~Tokens |
|
||||
|------|--------|---------|---------|--------|
|
||||
| 22:16 | Edited src/schema/controller.capnp | modified QueueConf() | ~354 |
|
||||
| 22:16 | Edited src/include/typemap.h | modified struct() | ~302 |
|
||||
| 22:16 | Edited src/controller/persist.c | modified 1() | ~279 |
|
||||
| 22:17 | Edited src/controller/testsuite/test_config_repl.c | modified filler() | ~354 |
|
||||
| 22:17 | Edited src/controller/testsuite/test_config_repl.c | 15→16 lines | ~279 |
|
||||
| 22:18 | Edited src/controller/testsuite/test_config_repl.c | modified checkpoint() | ~299 |
|
||||
| 22:19 | Edited src/controller/testsuite/test_config_repl.c | modified 1() | ~1190 |
|
||||
| 22:20 | Edited src/controller/testsuite/test_config_repl.c | 3→6 lines | ~109 |
|
||||
| 22:21 | Edited src/controller/persist.c | inline fix | ~28 |
|
||||
| 05:29 | xce-build.sh controller: env broken, nix hash mismatch (capnpc-1.0.1 source.drv) recurrence of bug-918, deterministic on retry, no compile reached | .wolf/buglog.json, src/flake.lock | env-blocked, no test results captured | ~300 |
|
||||
| 22:48 | Edited src/controller/persist.c | inline fix | ~10 |
|
||||
| 22:53 | config-upgrade-snapshot task 1 complete: HostAdminState/QueueAdminState capnp structs (mapname'd onto new typemap.h C counterparts, matching capnp_snap_hdr_t precedent), SNAPSHOT_FORMAT_VERSION 1->2, 3 test fixtures updated to fake_fmt=2u, new test_snapshot_recv_old_format_version_rejected_fail_loud (fmt=1u rejected fail-loud) | src/schema/controller.capnp, src/include/typemap.h, src/controller/persist.c, src/controller/testsuite/test_config_repl.c | schema verified via direct capnp/capnpc-c binaries from cached /nix/store (bug-918 recurrence-3 blocked full nix develop/build+link+test; confirmed identical failure in canonical non-worktree repo, not worktree-specific; librtti never built on this machine at all so full link is separately infeasible) | ~9500 |
|
||||
| 23:42 | Edited src/nixSupport/xce.nix | expanded (+11 lines) | ~175 |
|
||||
| 23:56 | Session end: 11 writes across 5 files (controller.capnp, typemap.h, persist.c, test_config_repl.c, xce.nix) | 20 reads | ~296043 tok |
|
||||
| 00:18 | Session end: 11 writes across 5 files (controller.capnp, typemap.h, persist.c, test_config_repl.c, xce.nix) | 20 reads | ~296043 tok |
|
||||
| 00:46 | Session end: 11 writes across 5 files (controller.capnp, typemap.h, persist.c, test_config_repl.c, xce.nix) | 20 reads | ~296043 tok |
|
||||
| 00:49 | Session end: 11 writes across 5 files (controller.capnp, typemap.h, persist.c, test_config_repl.c, xce.nix) | 20 reads | ~296043 tok |
|
||||
| 00:59 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | 1→2 lines | ~48 |
|
||||
| 01:10 | config-upgrade-snapshot task 1 validated: nix build fixed (bug-918 capnpc pin) unblocked full controller_unit_test run; 3 pre-existing FAILs in test_persist_snapshot_entities.c root-caused to stale SNAPENT_TEST_FORMAT_VERSION=1u, fixed to 2u; full suite 978 Tests 0 Failures 0 Ignored OK | controller/persist.c, controller/testsuite/{test_config_repl,test_persist_snapshot_entities}.c, include/typemap.h, schema/controller.capnp, nixSupport/xce.nix | READY_FOR_REVIEW | ~0 |
|
||||
| 01:20 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | 2→2 lines | ~47 |
|
||||
| 01:24 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | inline fix | ~29 |
|
||||
| 01:25 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | modified environment() | ~557 |
|
||||
| 01:27 | Created ../../../../../claude/.claude/projects/-home-rshen-projects-xce-xce/memory/reference_nix_capnpc_pin_fix.md | — | ~486 |
|
||||
| 01:27 | Edited ../../../../../claude/.claude/projects/-home-rshen-projects-xce-xce/memory/MEMORY.md | 3→4 lines | ~79 |
|
||||
| 01:45 | Edited src/controller/persist.c | added 1 condition(s) | ~316 |
|
||||
| 01:45 | Edited src/controller/persist.c | added 9 condition(s) | ~1290 |
|
||||
| 01:46 | Edited src/controller/persist.c | snap_job_frame_t() → snap_host_frame_t() | ~162 |
|
||||
| 01:46 | Edited src/controller/persist.c | modified if() | ~115 |
|
||||
| 01:47 | Edited src/controller/persist.c | added 1 condition(s) | ~165 |
|
||||
| 01:47 | Edited src/controller/persist.c | added 4 condition(s) | ~341 |
|
||||
| 01:49 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | 7→8 lines | ~48 |
|
||||
| 01:49 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | modified snapent_queue_status() | ~402 |
|
||||
| 01:49 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | modified snapent_seed_hosts() | ~359 |
|
||||
| 01:51 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | modified snapent_make() | ~525 |
|
||||
| 01:52 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | added 1 condition(s) | ~61 |
|
||||
| 01:54 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | added 20 condition(s) | ~3372 |
|
||||
| 01:54 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | 2→5 lines | ~96 |
|
||||
| 02:16 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | inline fix | ~17 |
|
||||
| 02:16 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | modified 3() | ~168 |
|
||||
| 02:38 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | 15→18 lines | ~358 |
|
||||
| 02:39 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | modified section() | ~570 |
|
||||
| 02:39 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | modified for() | ~306 |
|
||||
| 02:41 | Edited src/controller/persist.c | added 1 condition(s) | ~386 |
|
||||
| 02:42 | Edited src/controller/persist.c | modified struct() | ~540 |
|
||||
| 02:42 | Edited src/controller/persist.c | strdup() → capnp_encode_queue_admin_state() | ~207 |
|
||||
| 02:43 | Edited src/controller/persist.c | modified trailer() | ~251 |
|
||||
| 02:52 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | modified test_snapent_load_snapshot_restores_entities_before_advancing_position() | ~310 |
|
||||
| 02:58 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | removed 17 lines | ~8 |
|
||||
| 03:19 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | inline fix | ~18 |
|
||||
| 03:20 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | 1→4 lines | ~220 |
|
||||
| 03:44 | Edited src/controller/persist.c | added 15 condition(s) | ~1756 |
|
||||
| 03:44 | Edited src/controller/persist.c | added 6 condition(s) | ~854 |
|
||||
| 03:45 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | 23→26 lines | ~138 |
|
||||
| 03:45 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | modified struct() | ~107 |
|
||||
| 03:46 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | modified cluster() | ~519 |
|
||||
| 03:47 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | added 13 condition(s) | ~955 |
|
||||
| 03:49 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | modified of() | ~2030 |
|
||||
| 03:51 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | 2→5 lines | ~107 |
|
||||
| 03:58 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | modified of() | ~778 |
|
||||
| 11:02 | Task 2.3 (config-upgrade-snapshot): added read_active_config_bytes + config-bytes checkpoint section, inline-duplicated (no config_repl.c call) per design.md precedent | src/controller/persist.c, src/controller/testsuite/test_persist_snapshot_entities.c | full suite 984 Tests 0 Failures 1 Ignored OK | ~9500 |
|
||||
| 04:55 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | inline fix | ~19 |
|
||||
| 04:55 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | modified needs() | ~134 |
|
||||
| 05:11 | Edited src/controller/include/worker.h | 3→4 lines | ~78 |
|
||||
| 05:15 | Created src/controller/testsuite/test_upgrade_snapshot_restore.c | — | ~3581 |
|
||||
| 05:16 | Edited src/controller/testsuite/controller_test.c | modified setUp() | ~34 |
|
||||
| 05:16 | Edited src/controller/testsuite/controller_test.c | 3→4 lines | ~28 |
|
||||
| 05:16 | Edited src/controller/meson.build | 1→2 lines | ~25 |
|
||||
| 05:17 | Edited src/controller/worker.c | added 5 condition(s) | ~1136 |
|
||||
| 05:44 | Edited src/controller/worker.c | modified gap() | ~321 |
|
||||
| 05:47 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | inline fix | ~30 |
|
||||
| 05:48 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | 1→3 lines | ~180 |
|
||||
| 06:07 | Edited src/controller/persist.c | 2→7 lines | ~119 |
|
||||
| 06:08 | Edited src/controller/persist.c | added 6 condition(s) | ~2307 |
|
||||
| 06:08 | Edited src/controller/persist.c | modified returns() | ~439 |
|
||||
| 06:11 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | modified test_snapent_load_snapshot_restores_entities_before_advancing_position() | ~135 |
|
||||
| 06:11 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | modified snapent_find_queue() | ~195 |
|
||||
| 06:13 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | modified Test() | ~2256 |
|
||||
| 06:14 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | expanded (+9 lines) | ~171 |
|
||||
| 06:16 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | inline fix | ~16 |
|
||||
| 06:16 | Edited src/controller/testsuite/test_persist_snapshot_entities.c | 2→3 lines | ~67 |
|
||||
| 06:31 | Edited src/controller/testsuite/test_config_repl.c | modified checkpoint() | ~469 |
|
||||
| 06:33 | Edited src/controller/testsuite/test_config_repl.c | 8→9 lines | ~120 |
|
||||
| 07:07 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | inline fix | ~19 |
|
||||
| 07:07 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | inline fix | ~63 |
|
||||
| 07:33 | Edited src/controller/persist.c | added 3 condition(s) | ~2030 |
|
||||
| 07:36 | Edited src/controller/persist.c | added 2 condition(s) | ~1172 |
|
||||
| 07:47 | Created src/controller/testsuite/test_upgrade_snapshot_apply.c | — | ~5535 |
|
||||
| 07:50 | Edited src/controller/testsuite/test_upgrade_snapshot_apply.c | 4→8 lines | ~100 |
|
||||
| 07:50 | Edited src/controller/meson.build | 1→2 lines | ~25 |
|
||||
| 07:51 | Edited src/controller/testsuite/controller_test.c | 1→2 lines | ~26 |
|
||||
| 07:51 | Edited src/controller/testsuite/controller_test.c | 1→2 lines | ~22 |
|
||||
| 07:58 | Created src/controller/testsuite/test_upgrade_snapshot_apply.c | — | ~6646 |
|
||||
| 08:12 | Edited src/controller/testsuite/test_config_repl.c | added 1 condition(s) | ~516 |
|
||||
| 08:13 | Edited src/controller/testsuite/test_config_repl.c | modified t43_commit_marker() | ~232 |
|
||||
| 08:15 | Edited src/controller/testsuite/test_config_repl.c | expanded (+8 lines) | ~209 |
|
||||
| 08:17 | Edited src/controller/testsuite/test_config_repl.c | expanded (+6 lines) | ~163 |
|
||||
| 08:19 | Edited src/controller/testsuite/test_config_repl.c | expanded (+13 lines) | ~426 |
|
||||
| 08:32 | Edited src/controller/testsuite/test_config_repl.c | modified t43_append_marker_uncommitted() | ~507 |
|
||||
| 08:33 | Edited src/controller/testsuite/test_config_repl.c | reduced (-13 lines) | ~104 |
|
||||
| 08:34 | Edited src/controller/testsuite/test_config_repl.c | reduced (-6 lines) | ~32 |
|
||||
| 09:27 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | inline fix | ~24 |
|
||||
| 09:28 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | modified finding() | ~526 |
|
||||
| 09:52 | Edited src/controller/ha.h | expanded (+38 lines) | ~833 |
|
||||
| 09:52 | Edited src/controller/ha.h | modified upgrade_materialization_verify() | ~547 |
|
||||
| 09:52 | Edited src/controller/ha.c | 2→3 lines | ~20 |
|
||||
| 09:53 | Edited src/controller/ha.c | added 9 condition(s) | ~2592 |
|
||||
| 19:39 | Edited src/controller/testsuite/test_config_repl.c | modified test_upgrade_materialization_verify_no_marker_clears_fence() | ~2975 |
|
||||
| 19:40 | Edited src/controller/testsuite/test_config_repl.c | 2→7 lines | ~146 |
|
||||
| 19:43 | Session end: 100 writes across 17 files (controller.capnp, typemap.h, persist.c, test_config_repl.c, xce.nix) | 48 reads | ~635552 tok |
|
||||
| 19:46 | Edited src/controller/testsuite/test_config_repl.c | 6→11 lines | ~159 |
|
||||
| 19:47 | Edited src/controller/testsuite/test_config_repl.c | 5→8 lines | ~96 |
|
||||
| 19:50 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | inline fix | ~31 |
|
||||
| 19:51 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | 1→3 lines | ~210 |
|
||||
| 20:15 | Edited src/controller/ha.h | modified schedule_upgrade_materialization_retry() | ~308 |
|
||||
| 20:15 | Edited src/controller/ha.c | added 11 condition(s) | ~1450 |
|
||||
| 20:16 | Edited src/controller/ha.c | added 1 condition(s) | ~672 |
|
||||
| 20:16 | Edited src/controller/ha.c | added 1 condition(s) | ~638 |
|
||||
| 20:16 | Edited src/controller/ha.c | added 1 condition(s) | ~263 |
|
||||
| 20:23 | Edited src/controller/ha.h | modified upgrade_materialization_retry_fn() | ~230 |
|
||||
| 20:25 | Edited src/controller/testsuite/test_config_repl.c | modified test_upgrade_materialization_fence_enforced_by_dispatch_gate_and_promotion() | ~4128 |
|
||||
| 20:26 | Edited src/controller/testsuite/test_config_repl.c | modified XCE_TESTING() | ~202 |
|
||||
| 21:12 | Edited src/controller/persist.c | modified and() | ~321 |
|
||||
| 21:12 | Edited src/controller/persist.c | modified and() | ~1068 |
|
||||
| 21:13 | Edited src/controller/myapp.c | added 1 condition(s) | ~790 |
|
||||
| 21:18 | Edited src/controller/testsuite/test_config_repl.c | modified test_wal_config_marker_apply_sets_fence_via_real_checkpoint_miss() | ~1902 |
|
||||
| 21:19 | Edited src/controller/testsuite/test_config_repl.c | modified XCE_TESTING() | ~109 |
|
||||
| 21:29 | Edited src/controller/testsuite/test_config_repl.c | expanded (+23 lines) | ~537 |
|
||||
| 21:56 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | inline fix | ~19 |
|
||||
| 21:57 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | modified rejection() | ~263 |
|
||||
| 22:09 | Edited src/controller/config_repl.c | modified XCE_TESTING() | ~538 |
|
||||
| 22:10 | Edited src/controller/config_repl.c | modified XCE_TESTING() | ~283 |
|
||||
| 22:12 | Edited src/controller/testsuite/test_commit_barrier_rollout.c | modified XCE_TESTING() | ~1796 |
|
||||
| 22:14 | Edited src/controller/testsuite/test_commit_barrier_rollout.c | 2→3 lines | ~59 |
|
||||
| 22:37 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | inline fix | ~30 |
|
||||
| 22:54 | Edited src/controller/persist.c | modified XCE_TESTING() | ~510 |
|
||||
| 22:55 | Edited src/controller/persist.c | added 1 condition(s) | ~189 |
|
||||
| 23:00 | Edited src/controller/testsuite/test_upgrade_snapshot_apply.c | added error handling | ~172 |
|
||||
| 23:00 | Edited src/controller/testsuite/test_upgrade_snapshot_apply.c | added error handling | ~349 |
|
||||
| 23:01 | Edited src/controller/testsuite/test_upgrade_snapshot_apply.c | added error handling | ~2563 |
|
||||
| 23:02 | Edited src/controller/testsuite/test_upgrade_snapshot_apply.c | 2→4 lines | ~64 |
|
||||
| 23:07 | Edited src/controller/testsuite/test_upgrade_snapshot_apply.c | 4→6 lines | ~103 |
|
||||
| 23:09 | Edited src/controller/testsuite/test_upgrade_snapshot_apply.c | 6→4 lines | ~61 |
|
||||
| 23:32 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | inline fix | ~24 |
|
||||
| 23:52 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | inline fix | ~20 |
|
||||
| 23:53 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | 1→3 lines | ~224 |
|
||||
| 00:25 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | inline fix | ~12 |
|
||||
| 00:26 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | 1→3 lines | ~183 |
|
||||
|
||||
## Session: 2026-07-25 00:30
|
||||
|
||||
| Time | Action | File(s) | Outcome | ~Tokens |
|
||||
|------|--------|---------|---------|--------|
|
||||
| 00:56 | Created src/controller/testsuite/configcompare.h | — | ~1107 |
|
||||
| 00:57 | Created src/controller/testsuite/test_configcompare.c | — | ~2076 |
|
||||
| 00:57 | Edited src/controller/meson.build | inline fix | ~44 |
|
||||
| 00:57 | Edited src/controller/meson.build | 11→12 lines | ~80 |
|
||||
| 00:57 | Edited src/controller/meson.build | 5→8 lines | ~135 |
|
||||
| 00:58 | Edited src/controller/meson.build | 4→6 lines | ~50 |
|
||||
| 00:58 | Edited src/controller/testsuite/controller_test.c | 2→3 lines | ~28 |
|
||||
| 00:58 | Edited src/controller/testsuite/controller_test.c | 2→3 lines | ~22 |
|
||||
| 01:02 | Created src/controller/testsuite/configcompare.c | — | ~2122 |
|
||||
| 01:06 | Session end: 9 writes across 5 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 11 reads | ~200437 tok |
|
||||
| 01:07 | Edited src/controller/testsuite/snapverify.h | modified struct() | ~769 |
|
||||
| 01:08 | Edited src/controller/testsuite/snapverify.h | expanded (+6 lines) | ~283 |
|
||||
| 01:08 | Edited src/controller/testsuite/snapverify.c | 4→8 lines | ~148 |
|
||||
| 01:09 | Edited src/controller/testsuite/snapverify.c | added 1 condition(s) | ~154 |
|
||||
| 01:10 | Edited src/controller/testsuite/snapverify.c | added 20 condition(s) | ~1878 |
|
||||
| 01:11 | Edited src/controller/testsuite/snapverify.c | 10→13 lines | ~148 |
|
||||
| 01:14 | Created src/controller/testsuite/test_snapverify.c | — | ~4696 |
|
||||
| 01:24 | Session end: 16 writes across 8 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 11 reads | ~211107 tok |
|
||||
| 01:45 | Session end: 16 writes across 8 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 15 reads | ~231413 tok |
|
||||
| 01:48 | Session end: 16 writes across 8 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 15 reads | ~231413 tok |
|
||||
| 01:49 | Edited src/controller/testsuite/test.c | 2→3 lines | ~72 |
|
||||
| 01:49 | Edited src/controller/testsuite/test.c | 3→4 lines | ~44 |
|
||||
| 01:49 | Edited src/controller/testsuite/test.c | 4→5 lines | ~101 |
|
||||
| 01:50 | Edited src/controller/testsuite/test.c | 4→5 lines | ~104 |
|
||||
| 01:50 | Edited src/controller/testsuite/test.c | added 1 condition(s) | ~299 |
|
||||
| 08:55 | Registered configcompare as xce_test CLI subcommand (reviewer fix, task 8.4) | src/controller/testsuite/test.c | 1022/1022 tests pass; help lists configcompare; IDENTICAL/DIVERGENT smoke-tested via piped stdin, exit 0/1 | ~1800 |
|
||||
| 01:57 | Session end: 21 writes across 9 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 16 reads | ~262456 tok |
|
||||
| 02:04 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | inline fix | ~30 |
|
||||
| 02:05 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | modified with() | ~395 |
|
||||
| 02:20 | Session end: 23 writes across 10 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 17 reads | ~269881 tok |
|
||||
| 02:52 | Session end: 23 writes across 10 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 23 reads | ~279545 tok |
|
||||
| 03:14 | Created src/controller/testsuite/test_bug857_regression.c | — | ~6090 |
|
||||
| 03:15 | Edited src/controller/meson.build | 2→3 lines | ~35 |
|
||||
| 03:16 | Edited src/controller/testsuite/controller_test.c | modified setUp() | ~45 |
|
||||
| 03:16 | Edited src/controller/testsuite/controller_test.c | 4→5 lines | ~37 |
|
||||
| 03:17 | Edited .kiro/specs/config-upgrade-snapshot/design.md | inline fix | ~46 |
|
||||
| 03:18 | Edited .kiro/specs/config-upgrade-snapshot/design.md | modified XCE_TESTING() | ~823 |
|
||||
| 03:23 | Session end: 29 writes across 12 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 24 reads | ~289399 tok |
|
||||
| 03:25 | Session end: 29 writes across 12 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 25 reads | ~295489 tok |
|
||||
| 03:39 | Edited .kiro/specs/config-upgrade-snapshot/design.md | 2→2 lines | ~251 |
|
||||
| 03:41 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | inline fix | ~23 |
|
||||
| 03:42 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | modified XCE_TESTING() | ~845 |
|
||||
| 04:01 | Session end: 32 writes across 12 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 32 reads | ~362257 tok |
|
||||
| 04:28 | Edited src/controller/cluster.c | modified fix() | ~544 |
|
||||
| 04:30 | Edited src/lib/request.c | added 4 condition(s) | ~836 |
|
||||
| 04:32 | Session end: 34 writes across 14 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 34 reads | ~384133 tok |
|
||||
| 04:32 | Edited src/include/request.h | expanded (+13 lines) | ~304 |
|
||||
| 04:32 | Edited src/include/xceapi.h | expanded (+11 lines) | ~248 |
|
||||
| 04:33 | Edited src/lib/xceapi.c | added 4 condition(s) | ~256 |
|
||||
| 04:33 | Edited src/include/client.h | expanded (+8 lines) | ~192 |
|
||||
| 04:36 | Edited src/lib/client.c | added 1 condition(s) | ~262 |
|
||||
| 04:41 | Session end: 39 writes across 19 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 36 reads | ~387384 tok |
|
||||
| 04:41 | Edited src/controller/testsuite/test.c | 1→3 lines | ~32 |
|
||||
| 04:42 | Edited src/controller/testsuite/test.c | 2→3 lines | ~35 |
|
||||
| 04:42 | Edited src/controller/testsuite/test.c | 2→5 lines | ~87 |
|
||||
| 04:43 | Edited src/controller/testsuite/test.c | added 10 condition(s) | ~1079 |
|
||||
| 04:59 | Edited src/controller/testsuite/test.c | added 1 condition(s) | ~184 |
|
||||
| 05:00 | Session end: 44 writes across 19 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 38 reads | ~388902 tok |
|
||||
| 05:03 | Session end: 44 writes across 19 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 38 reads | ~389017 tok |
|
||||
| 05:15 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | modified find_xceadmin() | ~3566 |
|
||||
| 05:16 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | 2→3 lines | ~48 |
|
||||
| 05:17 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | modified 6() | ~423 |
|
||||
| 05:18 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | expanded (+8 lines) | ~247 |
|
||||
| 05:39 | Session end: 48 writes across 20 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 39 reads | ~427537 tok |
|
||||
| 05:46 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | modified PASS() | ~460 |
|
||||
| 05:47 | Edited .claude/skills/xce-test-ha/SKILL.md | inline fix | ~45 |
|
||||
| 05:48 | Edited .claude/skills/xce-test-ha/SKILL.md | 3→5 lines | ~123 |
|
||||
| 05:49 | Edited .claude/skills/xce-test-ha/SKILL.md | modified PASS() | ~1054 |
|
||||
| 06:03 | Session end: 52 writes across 21 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 41 reads | ~430394 tok |
|
||||
| 06:58 | Verified prior agent's task 8.6 work (bug-1468 fix, Parts A-D) coherent via read | cluster.c, request.c, xceapi.c, client.c, test.c | all correct, matches server-side decode expectations | ~15k |
|
||||
| 06:59 | Added --config-upgrade dispatch block (array-soak-style: hard-fail internal, no scan_bug924) | run-ha-soak.sh | inserted after array-chaos-retry block | ~600 |
|
||||
| 06:59 | Documented --config-upgrade mode: env-var table + XCE_HA_MODE list + full subsection | SKILL.md | matches array-chaos-retry depth | ~900 |
|
||||
| 07:00 | Full clean build (ninja -C src/xce.build), verified fresh binaries | xce_controller, xce_test, xceadmin | 0 errors | ~2k |
|
||||
| 07:01 | Ran live --config-upgrade soak: /tmp/xce-ha-configupgrade | run-ha-soak.sh | upgrade genuinely interleaved w/ 3 chaotic kills, committed, 6/6 jobs survived, but snapdiff FAILED: ctrl-1 checkpoint 51569B vs ctrl-2/3 51621B | ~8k |
|
||||
| 07:10 | Root-caused divergence: do_upgrade_config_with_bytes never calls config_activate_committed on originating primary, so c->myconf queue table stays stale until a later event; ctrl-1 killed before that could happen | config_repl.c, ha.c, persist.c (read-only) | logged bug-1469, live empirical repro of bug-857's hypothesis via a 3rd distinct mechanism | ~20k |
|
||||
| 07:11 | Logged bug-1469 to buglog.json (related to bug-857, bug-1466); did NOT fix (out of task 8.6 scope, tasks 1-7 territory) | buglog.json | valid JSON, 788 entries | ~3k |
|
||||
| 06:25 | Session end: 52 writes across 21 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 42 reads | ~547481 tok |
|
||||
| 06:54 | Edited src/controller/testsuite/test_config_repl.c | modified soak() | ~1961 |
|
||||
| 06:55 | Edited src/controller/testsuite/test_config_repl.c | modified XCE_TESTING() | ~83 |
|
||||
| 06:56 | Session end: 54 writes across 22 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 43 reads | ~550527 tok |
|
||||
| 06:57 | Edited src/controller/persist.c | modified and() | ~1627 |
|
||||
| 06:59 | Edited src/controller/persist.c | expanded (+14 lines) | ~253 |
|
||||
| 07:06 | Edited src/controller/persist.c | added 1 condition(s) | ~354 |
|
||||
| 07:24 | Edited src/controller/persist.c | added 1 condition(s) | ~1792 |
|
||||
| 14:26 | Ran full /xce-build controller (nix develop + xce-build.sh); 1st attempt failed on transient meson/ninja capnp-header race (unrelated to code), 2nd clean rebuild compiled clean but controller_unit_test reproducibly FAILED test_commit_advance_retry_clears_push_before_commit_fence (4/4 runs) — logged bug-1470, needs triage before task 8.6 soak | src/xce.build/controller/*, .wolf/buglog.json | build green, 1 test red | ~large |
|
||||
| 07:26 | Edited src/controller/persist.c | removed 82 lines | ~59 |
|
||||
| 07:28 | Session end: 59 writes across 23 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 43 reads | ~559722 tok |
|
||||
| 07:37 | Edited src/controller/testsuite/test_config_repl.c | expanded (+17 lines) | ~750 |
|
||||
| 07:38 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | expanded (+20 lines) | ~834 |
|
||||
| 07:41 | Edited .claude/skills/xce-test-ha/SKILL.md | expanded (+9 lines) | ~257 |
|
||||
| 07:47 | Edited src/controller/persist.c | expanded (+6 lines) | ~317 |
|
||||
| 14:50 | Task 8.6 wrap-up: fixed persist.c config_activate_committed ordering (bug-1469 pt1), diagnosed+documented deep qmgr live-sync gap (bug-1469 pt2, needs owner), diagnosed+documented 1-byte host-frame divergence (bug-1471, needs owner), switched soak's upgrade-target change from add-queue to update-queue to avoid the unrelated gap | persist.c, run-ha-soak.sh, SKILL.md, buglog.json | 3 real bugs found via live soak; 1 fixed+verified, 2 documented for tasks 1-7 owners | ~1200 |
|
||||
| 07:56 | Session end: 63 writes across 23 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 44 reads | ~570970 tok |
|
||||
| 08:05 | Created ../../../../../../tmp/claude-1000/-home-rshen-projects-xce-worktrees-config/d3f40b2e-8c73-4820-8ad3-1400be8761ac/scratchpad/update_bug1471.py | — | ~439 |
|
||||
| 08:12 | Session end: 64 writes across 24 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 44 reads | ~571409 tok |
|
||||
| 08:43 | Session end: 64 writes across 24 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 44 reads | ~571727 tok |
|
||||
| 08:47 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | modified progress() | ~1227 |
|
||||
| 08:48 | Session end: 65 writes across 24 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 44 reads | ~573041 tok |
|
||||
| 09:11 | Session end: 65 writes across 24 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 44 reads | ~573041 tok |
|
||||
| 16:36 | bug-1471 root-caused: byte-precise capnp decode of preserved checkpoints proves the divergent byte is inside HostAdminState.status (0x00500000 CLOSED\|CLOSED_BY_ADMIN on ctrl-1 vs 0x0 on ctrl-2/ctrl-3), not capnp padding (calloc-zeroed, verified via disasm of libCapnC_Runtime.so create_local). True cause: worker_from_capnp (worker.c:827-856, predates spec by ~2.5mo) EEXIST-short-circuits WAL_WORKER_CTRL UPDATE/DEAD applies on standbys, silently dropping admin-status changes for already-joined hosts -- general host-control replication bug, out of config-upgrade-snapshot scope. Filed as bug-1472, cross-referenced from bug-1471. No production code changed (out-of-scope, not patched). | .wolf/buglog.json | root-caused, routed out-of-scope, buglog updated | ~large |
|
||||
| 09:39 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | 6→7 lines | ~482 |
|
||||
| 09:44 | Session end: 66 writes across 24 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 46 reads | ~609994 tok |
|
||||
| 10:09 | Edited src/controller/testsuite/test_upgrade_snapshot_apply.c | 5→8 lines | ~145 |
|
||||
| 10:10 | Edited src/controller/testsuite/test_upgrade_snapshot_apply.c | added 4 condition(s) | ~2731 |
|
||||
| 10:10 | Edited src/controller/testsuite/test_upgrade_snapshot_apply.c | 2→3 lines | ~44 |
|
||||
| 10:16 | Session end: 69 writes across 25 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 48 reads | ~635904 tok |
|
||||
| 10:20 | Edited src/controller/testsuite/test_config_repl.c | 5→9 lines | ~107 |
|
||||
| 10:22 | Edited src/controller/testsuite/test_config_repl.c | added error handling | ~3652 |
|
||||
| 10:22 | Edited src/controller/testsuite/test_config_repl.c | 3→3 lines | ~49 |
|
||||
| 10:23 | Edited src/controller/testsuite/test_config_repl.c | modified XCE_TESTING() | ~127 |
|
||||
| 10:24 | Edited src/controller/testsuite/test_config_repl.c | 3→6 lines | ~97 |
|
||||
| 10:48 | Session end: 74 writes across 25 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 49 reads | ~640223 tok |
|
||||
| 10:53 | Session end: 74 writes across 25 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 49 reads | ~640223 tok |
|
||||
| 10:57 | Session end: 74 writes across 25 files (configcompare.h, test_configcompare.c, meson.build, controller_test.c, configcompare.c) | 49 reads | ~640223 tok |
|
||||
| 11:16 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | inline fix | ~23 |
|
||||
| 11:19 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | modified twice() | ~663 |
|
||||
|
||||
## Session: 2026-07-25 11:26
|
||||
|
||||
| Time | Action | File(s) | Outcome | ~Tokens |
|
||||
|------|--------|---------|---------|--------|
|
||||
| 11:59 | Edited src/controller/testsuite/test_config_repl.c | added 12 condition(s) | ~4947 |
|
||||
| 12:01 | Edited src/controller/testsuite/test_config_repl.c | 2→7 lines | ~133 |
|
||||
| 12:11 | Edited src/controller/testsuite/test_config_repl.c | expanded (+12 lines) | ~455 |
|
||||
| 12:11 | Edited src/controller/testsuite/test_config_repl.c | 4→8 lines | ~127 |
|
||||
| 12:46 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | inline fix | ~31 |
|
||||
| 12:47 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | modified 8() | ~691 |
|
||||
| 13:09 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | modified 9() | ~731 |
|
||||
| 13:11 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | expanded (+15 lines) | ~330 |
|
||||
| 13:12 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | 2→3 lines | ~51 |
|
||||
| 13:12 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | modified 9() | ~369 |
|
||||
| 13:14 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | modified wal_dir() | ~4510 |
|
||||
| 13:15 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | modified PASS() | ~463 |
|
||||
| 13:18 | Edited .claude/skills/xce-test-ha/SKILL.md | added error handling | ~1699 |
|
||||
| 13:19 | Edited .claude/skills/xce-test-ha/SKILL.md | inline fix | ~52 |
|
||||
| 13:19 | Edited .claude/skills/xce-test-ha/SKILL.md | 3→6 lines | ~178 |
|
||||
| 13:35 | Ran live 3-node soak --config-upgrade-wal-trim (2x, reproducible) | .claude/skills/xce-test-ha/run-ha-soak.sh | FAIL both runs — real bug found, not test-script flaw | ~9000 |
|
||||
| 13:50 | Root-caused live-soak failure via ctrl-2.log forensics: late-joiner slot-reuse clobbers bug-1147 disconnect guard | src/controller/ha.c, src/controller/repl.c | root cause confirmed, 2/2 reproducible | ~6500 |
|
||||
| 13:52 | Logged bug-1474 (late-joiner reconnect clobbers disconnected-standby's WAL-trim guard via slot reuse) | .wolf/buglog.json | new entry appended, NOT fixed (out of task 8.9 scope) | ~2100 |
|
||||
| 13:53 | Task 8.9 implementation: run_config_upgrade_wal_trim_round added and verified sound; task BLOCKED on bug-1474 | .claude/skills/xce-test-ha/run-ha-soak.sh, SKILL.md, .wolf/buglog.json, .wolf/cerebrum.md | infra correct, live PASS blocked on separate HA defect | ~1200 |
|
||||
| 14:09 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | 3→4 lines | ~513 |
|
||||
| 14:10 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | modified 9() | ~794 |
|
||||
| 20:04 | Edited src/controller/testsuite/test_repl_unit.c | modified RED() | ~1771 |
|
||||
| 20:04 | Edited src/controller/testsuite/test_repl_unit.c | 3→6 lines | ~89 |
|
||||
| 20:10 | Edited src/controller/repl.c | added 3 condition(s) | ~656 |
|
||||
| 20:12 | Created ../../../../../../tmp/claude-1000/-home-rshen-projects-xce-worktrees-config/d3f40b2e-8c73-4820-8ad3-1400be8761ac/scratchpad/update_bug1474.py | — | ~1227 |
|
||||
| 20:14 | Edited src/controller/worker.c | added 1 condition(s) | ~842 |
|
||||
| 20:15 | Edited src/controller/testsuite/test_apply_committed_entry.c | modified tace_build_worker_ctrl_pkt() | ~2190 |
|
||||
| 20:15 | Edited src/controller/testsuite/test_apply_committed_entry.c | 2→5 lines | ~92 |
|
||||
| 20:16 | bug-1474 FIXED: repl_add_standby two-pass slot allocation (virgin-slot-first, then oldest-guard fallback) — TDD RED (2 new tests fail pre-fix) → GREEN (1030 tests, 0 failures, no regressions vs 1028 baseline) | src/controller/repl.c, src/controller/testsuite/test_repl_unit.c, .wolf/buglog.json | fixed + verified, buglog "fix" field updated (forensic history preserved) | ~large |
|
||||
| 20:29 | bug-1472 FIXED: worker_from_capnp() now decodes hook_detail and, for an already-existing worker, REPLACES (not ORs) XCE_WORKER_ADMIN_MASK bits from the decoded HOOK_DETAIL_WORKER_UPDATE payload instead of unconditionally returning -EEXIST — admin hostctrl -c/-o now propagates to standbys/own-WAL-replay. HOOK_DETAIL_WORKER_JOIN unchanged (-EEXIST no-op, regression-guarded). HOOK_DETAIL_WORKER_DEAD deliberately deferred (investigated: w->host.resvals live resource-value zarray isn't in the Worker capnp wire schema at all — logged as bug-1472-remainder). TDD: RED (2 new UPDATE tests fail against pre-fix code, confirmed) → GREEN (fix applied, full suite 1030→1033 tests, 0 failures, net +3 = exactly the new tests, zero regressions). persist.c untouched (its WAL_WORKER_CTRL apply case already generically handles any non-EEXIST-non-negative return). | src/controller/worker.c, src/controller/testsuite/test_apply_committed_entry.c, .wolf/buglog.json, .wolf/anatomy.md | fixed + verified, buglog bug-1472 "fix" field updated + bug-1472-remainder filed (forensic history preserved) | ~large |
|
||||
| 20:48 | Edited src/controller/repl.c | added 1 condition(s) | ~454 |
|
||||
| 21:03 | bug-1474 review REMEDIATED: reviewer rejected on 1 finding (Pass 2 guard-slot-reuse memset was silent, no WARN, undercutting the fix's own "make silent destruction visible" goal). Added used_fallback_slot tracking + WARN("repl_add_standby: reusing guard-holding slot...") right before the memset, mirroring repl_broadcast's existing WARN convention. No new test needed (log line only) — verified via `strings` on the built binary + both existing bug-1474 tests still passing. Hit one transient shared-build-dir artifact (stale worker.c.o from a concurrent sibling agent's in-flight build caused 2 spurious bug-1472-test failures on one run) — resolved via `touch` + rebuild, not a real regression. Final: full suite 1033 tests, 0 failures. | src/controller/repl.c, .wolf/buglog.json, .wolf/anatomy.md | remediated + verified, buglog bug-1474 "fix" field appended with [UPDATE] note (forensic history preserved) | ~large |
|
||||
| 04:44 | bug-1474 live-soak RE-VERIFICATION (run-ha-soak.sh --config-upgrade-wal-trim, post-fix, deploy /home/claude/tmp/xce-ha-waltrim3): CORE ASSERTION PASSED — floor segment seg-...05.wal correctly PRESERVED, below-floor seg-01/03/04 correctly TRIMMED, active seg-...15.wal PRESENT (confirmed both via script log and independent on-disk `find`). New WARN("repl_add_standby: reusing guard-holding slot...") fired 2x on ctrl-2 (primary) — confirms the still-open ha.c duplicate-reconnect defect (bug-1474 fix-angle-(a), NOT fixed) is still spawning duplicate/late-joiner reconnects, exercising the fix's Pass-2 exhausted-fallback path for real; despite that, the specific segment under test stayed protected this run. Round's OVERALL verdict still FAIL, but at a LATER, unrelated step ("cluster did not reconverge to 3-of-3 after restarting ctrl-1") — root-caused to TWO separate, pre-existing, out-of-scope defects, NEITHER the original bug-1474 symptom: (1) NEW crash — ctrl-3 ASAN heap-use-after-free in config_repl.c's load_primary_config/config_activate_committed/handle_repl_config_push path (logged as bug-1476); (2) ctrl-1's post-restart catch-up got stuck at lsn=21 needing just lsn=22, ctrl-2 retrying identically every ~3s with zero progress — exact match to already-logged, not-yet-fixed bug-1475 (prevIdx off-by-one in ha_dispatch_repl_action_for_standby/ha_stream_wal_to_standby). First soak attempt (subagent-launched via Bash run_in_background from inside an Agent-tool call) died silently mid-election when the subagent's own turn ended — background process did NOT survive subagent completion despite setsid nohup being used internally by run-ha-soak.sh's start_node (the DRIVER script itself, launched via the Bash tool's run_in_background flag rather than a raw shell `setsid nohup ... & disown`, did not survive). Relaunched directly from the main agent thread with explicit `setsid nohup ... & disown` + polling loop, which survived and completed cleanly. | .wolf/buglog.json (bug-1476 new entry) | bug-1474's own fix confirmed GOOD at live-soak level; 2 separate follow-on blockers identified for task 8.9 (bug-1475, bug-1476) | ~large |
|
||||
|
||||
## Session: 2026-07-26 22:31
|
||||
|
||||
| Time | Action | File(s) | Outcome | ~Tokens |
|
||||
|------|--------|---------|---------|--------|
|
||||
| 22:36 | Created reports/jul19-jul26-2026.md | — | ~3715 |
|
||||
| 22:36 | Session end: 1 writes across 1 files (jul19-jul26-2026.md) | 4 reads | ~237768 tok |
|
||||
| 22:54 | Session end: 1 writes across 1 files (jul19-jul26-2026.md) | 11 reads | ~414061 tok |
|
||||
| 23:03 | Edited src/controller/ha.c | added 2 condition(s) | ~900 |
|
||||
| 23:03 | Edited src/controller/config_repl.c | modified 1475() | ~613 |
|
||||
| 23:04 | Edited src/controller/config_repl.c | modified if() | ~38 |
|
||||
| 23:12 | Edited src/controller/testsuite/test_config_repl.c | modified 1475() | ~578 |
|
||||
| 23:13 | Edited src/controller/testsuite/test_config_repl.c | added error handling | ~281 |
|
||||
| 03:41 | Session end: 6 writes across 4 files (jul19-jul26-2026.md, ha.c, config_repl.c, test_config_repl.c) | 13 reads | ~448433 tok |
|
||||
| 03:42 | Edited src/controller/testsuite/test_config_repl.c | added 1 condition(s) | ~2472 |
|
||||
| 03:43 | Edited src/controller/testsuite/test_config_repl.c | 2→3 lines | ~56 |
|
||||
|
||||
## Session: 2026-07-27 01:17
|
||||
|
||||
| Time | Action | File(s) | Outcome | ~Tokens |
|
||||
|------|--------|---------|---------|--------|
|
||||
| 01:50 | Edited .kiro/specs/scheduler-framework/requirements.md | inline fix | ~222 |
|
||||
| 01:51 | Edited .kiro/specs/scheduler-framework/requirements.md | inline fix | ~295 |
|
||||
| 01:52 | Edited .kiro/specs/scheduler-framework/requirements.md | modified pointer() | ~2660 |
|
||||
| 01:52 | Edited .kiro/specs/scheduler-framework/requirements.md | modified enum() | ~1084 |
|
||||
| 01:53 | Edited .kiro/specs/scheduler-framework/requirements.md | expanded (+89 lines) | ~1478 |
|
||||
| 01:54 | Edited .kiro/specs/scheduler-framework/spec.json | inline fix | ~11 |
|
||||
| 01:54 | Session end: 6 writes across 2 files (requirements.md, spec.json) | 8 reads | ~40857 tok |
|
||||
| 02:07 | Session end: 6 writes across 2 files (requirements.md, spec.json) | 9 reads | ~40857 tok |
|
||||
| 02:09 | Edited .kiro/specs/scheduler-framework/requirements.md | added 2 condition(s) | ~862 |
|
||||
| 02:10 | Edited .kiro/specs/scheduler-framework/requirements.md | 14→16 lines | ~450 |
|
||||
| 02:11 | Edited .kiro/specs/scheduler-framework/requirements.md | modified v2() | ~679 |
|
||||
| 02:11 | Edited .kiro/specs/scheduler-framework/requirements.md | 6→4 lines | ~277 |
|
||||
| 02:12 | Edited .kiro/specs/scheduler-framework/requirements.md | modified struct() | ~325 |
|
||||
| 02:12 | Edited .kiro/specs/scheduler-framework/requirements.md | modified next_job() | ~767 |
|
||||
| 02:16 | Session end: 12 writes across 2 files (requirements.md, spec.json) | 10 reads | ~46842 tok |
|
||||
| 02:18 | Edited .kiro/specs/scheduler-framework/requirements.md | expanded (+32 lines) | ~904 |
|
||||
| 02:18 | Edited .kiro/specs/scheduler-framework/requirements.md | 7→7 lines | ~277 |
|
||||
| 02:19 | Edited .kiro/specs/scheduler-framework/requirements.md | modified pending() | ~957 |
|
||||
| 02:19 | Session end: 15 writes across 2 files (requirements.md, spec.json) | 10 reads | ~49731 tok |
|
||||
| 02:28 | Edited .kiro/specs/scheduler-framework/requirements.md | 12→7 lines | ~405 |
|
||||
| 02:28 | Edited .kiro/specs/scheduler-framework/requirements.md | 2→2 lines | ~137 |
|
||||
| 02:28 | Edited .kiro/specs/scheduler-framework/requirements.md | "project → queue → job" → "xce_job_picker_iface_t" | ~458 |
|
||||
| 02:29 | Edited .kiro/specs/scheduler-framework/requirements.md | "ha-consistency-invariants" → "builtin_fcfs" | ~314 |
|
||||
| 02:29 | Edited .kiro/specs/scheduler-framework/requirements.md | 12→12 lines | ~282 |
|
||||
| 02:30 | Edited .kiro/specs/scheduler-framework/requirements.md | modified Preemption() | ~285 |
|
||||
| 02:31 | Session end: 21 writes across 2 files (requirements.md, spec.json) | 10 reads | ~52078 tok |
|
||||
| 02:35 | Created .kiro/specs/scheduler-framework/brief.md | — | ~6523 |
|
||||
| 02:36 | Session end: 22 writes across 3 files (requirements.md, spec.json, brief.md) | 10 reads | ~59067 tok |
|
||||
| 02:56 | Edited .kiro/specs/scheduler-framework/requirements.md | added 1 condition(s) | ~1009 |
|
||||
| 02:56 | Session end: 23 writes across 3 files (requirements.md, spec.json, brief.md) | 10 reads | ~60293 tok |
|
||||
| 03:09 | Edited .kiro/specs/scheduler-framework/requirements.md | 3→3 lines | ~96 |
|
||||
| 03:09 | Edited .kiro/specs/scheduler-framework/requirements.md | 3→1 lines | ~13 |
|
||||
| 03:10 | Edited .kiro/specs/scheduler-framework/requirements.md | removed 69 lines | ~87 |
|
||||
| 03:11 | Edited .kiro/specs/scheduler-framework/requirements.md | added 1 condition(s) | ~1185 |
|
||||
| 03:11 | Session end: 27 writes across 3 files (requirements.md, spec.json, brief.md) | 10 reads | ~61772 tok |
|
||||
| 03:22 | Edited .kiro/specs/scheduler-framework/requirements.md | added 2 condition(s) | ~1754 |
|
||||
| 03:23 | Session end: 28 writes across 3 files (requirements.md, spec.json, brief.md) | 10 reads | ~64586 tok |
|
||||
| 03:29 | Edited .kiro/specs/scheduler-framework/requirements.md | slots() → slot() | ~62 |
|
||||
| 03:30 | Session end: 29 writes across 3 files (requirements.md, spec.json, brief.md) | 10 reads | ~65362 tok |
|
||||
| 03:33 | Edited .kiro/specs/scheduler-framework/requirements.md | added 1 condition(s) | ~572 |
|
||||
| 03:33 | Edited .kiro/specs/scheduler-framework/requirements.md | modified pointer() | ~366 |
|
||||
| 03:34 | Edited .kiro/specs/scheduler-framework/requirements.md | 3→5 lines | ~74 |
|
||||
| 03:35 | Edited .kiro/specs/scheduler-framework/requirements.md | modified pointer() | ~1197 |
|
||||
| 03:35 | Edited .kiro/specs/scheduler-framework/requirements.md | 14 → 15 | ~27 |
|
||||
| 03:36 | Edited .kiro/specs/scheduler-framework/requirements.md | 15 → 16 | ~15 |
|
||||
| 03:36 | Session end: 35 writes across 3 files (requirements.md, spec.json, brief.md) | 10 reads | ~68162 tok |
|
||||
| 03:40 | Edited .kiro/specs/scheduler-framework/requirements.md | modified directly() | ~472 |
|
||||
| 03:41 | Edited .kiro/specs/scheduler-framework/requirements.md | 7→8 lines | ~135 |
|
||||
| 03:41 | Session end: 37 writes across 3 files (requirements.md, spec.json, brief.md) | 10 reads | ~69033 tok |
|
||||
| 03:43 | Edited .kiro/specs/scheduler-framework/requirements.md | 3→8 lines | ~374 |
|
||||
| 03:43 | Session end: 38 writes across 3 files (requirements.md, spec.json, brief.md) | 10 reads | ~69453 tok |
|
||||
| 03:50 | Edited .kiro/specs/scheduler-framework/requirements.md | 4→4 lines | ~227 |
|
||||
| 03:51 | Edited .kiro/specs/scheduler-framework/requirements.md | modified via() | ~450 |
|
||||
| 03:51 | Session end: 40 writes across 3 files (requirements.md, spec.json, brief.md) | 11 reads | ~70526 tok |
|
||||
| 03:55 | Edited .kiro/specs/scheduler-framework/requirements.md | modified directly() | ~842 |
|
||||
| 03:56 | Edited .kiro/specs/scheduler-framework/requirements.md | 2→6 lines | ~121 |
|
||||
| 03:56 | Session end: 42 writes across 3 files (requirements.md, spec.json, brief.md) | 11 reads | ~72053 tok |
|
||||
| 04:01 | Session end: 42 writes across 3 files (requirements.md, spec.json, brief.md) | 13 reads | ~72053 tok |
|
||||
| 04:04 | Edited .kiro/specs/scheduler-framework/requirements.md | expanded (+16 lines) | ~707 |
|
||||
| 04:05 | Session end: 43 writes across 3 files (requirements.md, spec.json, brief.md) | 13 reads | ~72887 tok |
|
||||
| 19:36 | Edited src/controller/testsuite/test_config_repl.c | 10→14 lines | ~213 |
|
||||
| 19:41 | Session end: 44 writes across 4 files (requirements.md, spec.json, brief.md, test_config_repl.c) | 15 reads | ~262461 tok |
|
||||
| 19:52 | Edited src/controller/config_repl.c | modified if() | ~180 |
|
||||
| 19:53 | Edited src/controller/ha.c | modified persist_durable_floor() | ~231 |
|
||||
| 19:53 | Edited src/controller/ha.c | modified if() | ~167 |
|
||||
| 19:53 | Edited src/controller/ha.c | expanded (+9 lines) | ~223 |
|
||||
| 19:53 | Edited src/controller/ha.c | added error handling | ~166 |
|
||||
| 19:54 | Edited src/lib/raft/raft.c | modified on_get_entry_term() | ~243 |
|
||||
| 19:54 | Edited src/lib/raft/raft.c | expanded (+6 lines) | ~209 |
|
||||
| 19:54 | Edited src/lib/raft/raft.c | 3→4 lines | ~56 |
|
||||
| 19:54 | Edited src/lib/raft/raft.c | 4→6 lines | ~87 |
|
||||
| 19:55 | Edited src/lib/raft/raft.c | 1→2 lines | ~22 |
|
||||
| 19:59 | Edited src/controller/testsuite/test_config_repl.c | 4→5 lines | ~49 |
|
||||
| 19:59 | Edited src/controller/testsuite/test_config_repl.c | expanded (+8 lines) | ~167 |
|
||||
| 19:59 | Edited src/controller/testsuite/test_config_repl.c | 4→5 lines | ~48 |
|
||||
| 20:02 | Session end: 57 writes across 7 files (requirements.md, spec.json, brief.md, test_config_repl.c, config_repl.c) | 18 reads | ~391509 tok |
|
||||
| 20:05 | Edited src/controller/persist.c | added error handling | ~1320 |
|
||||
| 20:07 | Edited src/controller/config_repl.c | removed 13 lines | ~27 |
|
||||
| 20:07 | Edited src/controller/ha.c | removed 18 lines | ~46 |
|
||||
| 20:07 | Edited src/controller/ha.c | modified if() | ~67 |
|
||||
| 20:07 | Edited src/controller/ha.c | reduced (-9 lines) | ~63 |
|
||||
| 20:08 | Edited src/controller/ha.c | reduced (-9 lines) | ~36 |
|
||||
| 20:08 | Edited src/lib/raft/raft.c | reduced (-9 lines) | ~92 |
|
||||
| 20:08 | Edited src/lib/raft/raft.c | reduced (-6 lines) | ~98 |
|
||||
| 20:08 | Edited src/lib/raft/raft.c | 4→3 lines | ~36 |
|
||||
| 20:08 | Edited src/lib/raft/raft.c | 6→4 lines | ~49 |
|
||||
| 20:09 | Edited src/lib/raft/raft.c | 2→1 lines | ~5 |
|
||||
| 20:09 | Edited src/controller/testsuite/test_config_repl.c | 5→4 lines | ~35 |
|
||||
| 20:09 | Edited src/controller/testsuite/test_config_repl.c | reduced (-8 lines) | ~38 |
|
||||
| 20:09 | Edited src/controller/testsuite/test_config_repl.c | 2→1 lines | ~7 |
|
||||
| 20:14 | Edited src/controller/testsuite/test_config_repl.c | modified side() | ~880 |
|
||||
| 20:14 | Edited src/controller/testsuite/test_config_repl.c | modified RED() | ~45 |
|
||||
| 20:15 | Edited src/controller/ha.c | added error handling | ~416 |
|
||||
| 20:16 | Edited src/controller/config_repl.c | expanded (+18 lines) | ~410 |
|
||||
| 03:22 | bug-1475: root-caused via real fprintf instrumentation (streamer/JIT-resync/raft logOk) that the two already-landed fixes left applied_lsn stuck at 0 | src/controller/persist.c (TERM_INDEX_UNSET sentinel fix), src/controller/ha.c + config_repl.c (cross-ref comments only), src/controller/testsuite/test_config_repl.c (doc comments), .wolf/buglog.json | FIXED: full suite 1034/1034 (was 1033/1034); term-index growth silently reclassified a skipped WAL_CHECKPOINT lsn's term from absent(0) to WAL_LEGACY_TERM(1) once a later lsn was recorded, causing sender/receiver to disagree on prev_log_term | ~140k |
|
||||
| 20:28 | Session end: 75 writes across 8 files (requirements.md, spec.json, brief.md, test_config_repl.c, config_repl.c) | 19 reads | ~419701 tok |
|
||||
| 20:48 | Session end: 75 writes across 8 files (requirements.md, spec.json, brief.md, test_config_repl.c, config_repl.c) | 21 reads | ~423430 tok |
|
||||
| 20:52 | Session end: 75 writes across 8 files (requirements.md, spec.json, brief.md, test_config_repl.c, config_repl.c) | 22 reads | ~459739 tok |
|
||||
| 20:55 | Session end: 75 writes across 8 files (requirements.md, spec.json, brief.md, test_config_repl.c, config_repl.c) | 22 reads | ~459739 tok |
|
||||
| 20:57 | Created .kiro/specs/scheduler-core/spec.json | — | ~202 |
|
||||
| 20:57 | Created .kiro/specs/scheduler-allocation/spec.json | — | ~210 |
|
||||
| 20:58 | Created .kiro/specs/scheduler-preemption/spec.json | — | ~213 |
|
||||
| 21:00 | Created .kiro/specs/scheduler-framework/spec.json | — | ~211 |
|
||||
| 21:01 | Session end: 79 writes across 8 files (requirements.md, spec.json, brief.md, test_config_repl.c, config_repl.c) | 30 reads | ~467627 tok |
|
||||
| 21:03 | Edited src/controller/persist.c | added error handling | ~835 |
|
||||
| 21:04 | Created .kiro/specs/scheduler-allocation/brief.md | — | ~3600 |
|
||||
| 21:05 | Edited src/controller/testsuite/test_config_repl.c | added error handling | ~2390 |
|
||||
| 21:06 | Edited src/controller/testsuite/test_config_repl.c | 3→7 lines | ~91 |
|
||||
| 21:06 | Edited src/controller/testsuite/test_config_repl.c | 2→3 lines | ~58 |
|
||||
| 21:07 | Created .kiro/specs/scheduler-preemption/brief.md | — | ~3104 |
|
||||
| 21:07 | Created .kiro/specs/scheduler-preemption/requirements.md | — | ~2437 |
|
||||
| 21:10 | Session end: 86 writes across 8 files (requirements.md, spec.json, brief.md, test_config_repl.c, config_repl.c) | 30 reads | ~484551 tok |
|
||||
| 21:12 | Edited src/controller/testsuite/test_config_repl.c | expanded (+27 lines) | ~618 |
|
||||
| 21:14 | Edited src/controller/testsuite/test_config_repl.c | 3→6 lines | ~103 |
|
||||
| 21:15 | Created .kiro/specs/scheduler-core/brief.md | — | ~2121 |
|
||||
| 21:16 | Session end: 89 writes across 8 files (requirements.md, spec.json, brief.md, test_config_repl.c, config_repl.c) | 30 reads | ~488105 tok |
|
||||
| 21:16 | Edited src/controller/testsuite/test_config_repl.c | expanded (+12 lines) | ~628 |
|
||||
| 21:17 | Created .kiro/specs/scheduler-allocation/requirements.md | — | ~6636 |
|
||||
| 21:17 | Edited src/controller/persist.c | 6→3 lines | ~35 |
|
||||
| 21:18 | Created .kiro/specs/scheduler-core/requirements.md | — | ~9658 |
|
||||
| 04:18 | Created brief.md and requirements.md for scheduler-allocation spec at .kiro/specs/scheduler-allocation/ | brief.md, requirements.md | Both files written, scoped to allocation-only concerns (plan_iter, host_iter, parjob, resreq cache, seqjob rewrap) | ~18000tok |
|
||||
| 21:21 | Session end: 93 writes across 8 files (requirements.md, spec.json, brief.md, test_config_repl.c, config_repl.c) | 31 reads | ~506322 tok |
|
||||
| 21:22 | Edited src/controller/persist.c | 3→6 lines | ~120 |
|
||||
| 21:27 | Session end: 94 writes across 8 files (requirements.md, spec.json, brief.md, test_config_repl.c, config_repl.c) | 31 reads | ~506427 tok |
|
||||
| 21:35 | Edited src/controller/persist.c | 10→9 lines | ~81 |
|
||||
| 21:36 | Edited src/controller/persist.c | 9→10 lines | ~74 |
|
||||
| 21:46 | Edited src/include/persist.h | expanded (+6 lines) | ~231 |
|
||||
| 21:46 | Edited src/controller/ha.c | 6→8 lines | ~160 |
|
||||
| 21:51 | Session end: 98 writes across 9 files (requirements.md, spec.json, brief.md, test_config_repl.c, config_repl.c) | 32 reads | ~507012 tok |
|
||||
| 22:06 | Edited src/include/myconf.h | 7→8 lines | ~35 |
|
||||
| 22:06 | Edited src/include/myconf.h | expanded (+10 lines) | ~236 |
|
||||
| 22:06 | Edited src/controller/myconf.c | modified myconf_fini() | ~92 |
|
||||
| 22:06 | Edited src/controller/main.c | added 1 condition(s) | ~132 |
|
||||
| 22:10 | Edited src/controller/config_repl.c | modified XCE_TESTING() | ~229 |
|
||||
| 22:10 | Edited src/controller/config_repl.c | added 1 condition(s) | ~169 |
|
||||
| 22:11 | Edited src/controller/config_repl.h | expanded (+10 lines) | ~221 |
|
||||
| 22:11 | Session end: 105 writes across 13 files (requirements.md, spec.json, brief.md, test_config_repl.c, config_repl.c) | 40 reads | ~554405 tok |
|
||||
| 22:11 | Edited src/controller/testsuite/test_config_repl.c | modified t43_attach_myconf() | ~235 |
|
||||
| 22:14 | Edited src/controller/testsuite/test_config_repl.c | added 3 condition(s) | ~2535 |
|
||||
| 22:14 | Edited src/controller/testsuite/test_config_repl.c | 3→5 lines | ~72 |
|
||||
| 22:18 | Edited src/controller/testsuite/test_config_repl.c | expanded (+9 lines) | ~173 |
|
||||
| 22:20 | Edited src/controller/config_repl.c | added 1 condition(s) | ~2124 |
|
||||
|
||||
## Session: 2026-07-28 23:29
|
||||
|
||||
| Time | Action | File(s) | Outcome | ~Tokens |
|
||||
|------|--------|---------|---------|--------|
|
||||
| 00:28 | Edited .kiro/specs/scheduler-core/requirements.md | modified cbuf() | ~376 |
|
||||
| 00:29 | Edited .kiro/specs/scheduler-core/requirements.md | added 2 condition(s) | ~646 |
|
||||
| 00:29 | Edited .kiro/specs/scheduler-core/requirements.md | "sched_job_picker()" → "jbuf" | ~146 |
|
||||
| 00:30 | Edited .kiro/specs/scheduler-core/requirements.md | inline fix | ~104 |
|
||||
| 00:30 | Edited .kiro/specs/scheduler-core/requirements.md | inline fix | ~94 |
|
||||
| 00:30 | Edited .kiro/specs/scheduler-core/requirements.md | added 1 condition(s) | ~886 |
|
||||
| 00:31 | Edited .kiro/specs/scheduler-core/requirements.md | 4→5 lines | ~332 |
|
||||
| 00:32 | Edited .kiro/specs/scheduler-core/requirements.md | modified next_job() | ~999 |
|
||||
| 00:32 | Edited .kiro/specs/scheduler-core/requirements.md | modified struct() | ~247 |
|
||||
| 00:33 | Edited .kiro/specs/scheduler-core/requirements.md | inline fix | ~141 |
|
||||
| 00:33 | Session end: 10 writes across 1 files (requirements.md) | 13 reads | ~174206 tok |
|
||||
| 00:33 | Session end: 10 writes across 1 files (requirements.md) | 13 reads | ~174206 tok |
|
||||
| 00:40 | Edited src/controller/config_repl.c | modified XCE_TESTING() | ~1087 |
|
||||
|
||||
## Session: 2026-07-28 00:54
|
||||
|
||||
| Time | Action | File(s) | Outcome | ~Tokens |
|
||||
|------|--------|---------|---------|--------|
|
||||
| 01:05 | Edited src/controller/config_repl.c | modified XCE_TESTING() | ~1156 |
|
||||
| 01:14 | Session end: 1 writes across 1 files (config_repl.c) | 4 reads | ~133541 tok |
|
||||
| 01:16 | Session end: 1 writes across 1 files (config_repl.c) | 4 reads | ~133541 tok |
|
||||
| 01:18 | Session end: 1 writes across 1 files (config_repl.c) | 4 reads | ~133541 tok |
|
||||
| 01:24 | Session end: 1 writes across 1 files (config_repl.c) | 4 reads | ~133541 tok |
|
||||
| 01:34 | Session end: 1 writes across 1 files (config_repl.c) | 5 reads | ~219255 tok |
|
||||
| 01:45 | Session end: 1 writes across 1 files (config_repl.c) | 7 reads | ~357623 tok |
|
||||
| 01:47 | root-caused bug-1477 (ctrl-3 permanent catch-up wedge, skip_election never retries cluster_init_phase2 so commit engine/apply pump never wires) | .wolf/buglog.json, ha.c, myapp.c, cluster.c, persist.c | logged bug-1477, out-of-scope for config-upgrade-snapshot, routed upstream to HA bootstrap lineage | ~180000 tok |
|
||||
| 01:47 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | added error handling | ~630 |
|
||||
| 01:48 | Session end: 2 writes across 2 files (config_repl.c, tasks.md) | 7 reads | ~358297 tok |
|
||||
| 01:51 | Session end: 2 writes across 2 files (config_repl.c, tasks.md) | 7 reads | ~358297 tok |
|
||||
| 02:07 | Session end: 2 writes across 2 files (config_repl.c, tasks.md) | 9 reads | ~371122 tok |
|
||||
| 02:12 | Edited src/controller/ha.c | added error handling | ~827 |
|
||||
| 02:12 | Edited src/controller/ha.c | modified target() | ~158 |
|
||||
| 02:12 | Edited src/controller/ha.h | modified for() | ~313 |
|
||||
| 02:13 | Edited src/controller/testsuite/test_ha_role.c | added error handling | ~902 |
|
||||
| 02:13 | Edited src/controller/testsuite/test_ha_role.c | 3→8 lines | ~138 |
|
||||
|
||||
## Session: 2026-07-28 02:14
|
||||
|
||||
| Time | Action | File(s) | Outcome | ~Tokens |
|
||||
|------|--------|---------|---------|--------|
|
||||
| 02:20 | bug-1478 root-caused+fixed: ha_dispatch_repl_action_for_standby target_lsn leaked a WAL_CHECKPOINT-consumed LSN as a phantom convergence target, permanently wedging config-upgrade-wal-trim (task 8.9) catch-up; extracted ha_compute_repl_target_lsn, 4 new tests, RED->GREEN confirmed, 1040/1040 clean 2x | src/controller/ha.c, ha.h, testsuite/test_ha_role.c, .wolf/buglog.json, .wolf/cerebrum.md | fixed, unit-verified; live soak re-run still recommended | ~9500 |
|
||||
|
||||
## Session: 2026-07-28 02:21
|
||||
|
||||
| Time | Action | File(s) | Outcome | ~Tokens |
|
||||
|------|--------|---------|---------|--------|
|
||||
| 02:27 | Edited src/controller/config_repl.c | modified used() | ~283 |
|
||||
| 02:28 | Session end: 1 writes across 1 files (config_repl.c) | 3 reads | ~45187 tok |
|
||||
| 02:30 | Session end: 1 writes across 1 files (config_repl.c) | 3 reads | ~45187 tok |
|
||||
| 02:30 | Session end: 1 writes across 1 files (config_repl.c) | 3 reads | ~45187 tok |
|
||||
| 02:31 | Session end: 1 writes across 1 files (config_repl.c) | 3 reads | ~45187 tok |
|
||||
| 02:33 | Session end: 1 writes across 1 files (config_repl.c) | 3 reads | ~45187 tok |
|
||||
| 02:35 | Session end: 1 writes across 1 files (config_repl.c) | 4 reads | ~63902 tok |
|
||||
| 02:39 | Edited .kiro/specs/scheduler-core/requirements.md | added 2 condition(s) | ~1395 |
|
||||
| 02:39 | Edited .kiro/specs/scheduler-core/requirements.md | expanded (+6 lines) | ~743 |
|
||||
| 02:39 | Edited .kiro/specs/scheduler-core/requirements.md | modified struct() | ~52 |
|
||||
| 02:40 | Edited .kiro/specs/scheduler-core/requirements.md | 9→10 lines | ~156 |
|
||||
| 02:40 | Session end: 5 writes across 2 files (config_repl.c, requirements.md) | 4 reads | ~66417 tok |
|
||||
| 02:43 | Edited .kiro/specs/scheduler-core/requirements.md | inline fix | ~354 |
|
||||
| 02:43 | Edited .kiro/specs/scheduler-core/requirements.md | 14→15 lines | ~620 |
|
||||
| 02:44 | Session end: 7 writes across 2 files (config_repl.c, requirements.md) | 4 reads | ~69147 tok |
|
||||
| 02:44 | Session end: 7 writes across 2 files (config_repl.c, requirements.md) | 4 reads | ~69147 tok |
|
||||
| 02:47 | Edited src/controller/testsuite/test_convergence_verification.c | modified coverage() | ~250 |
|
||||
| 02:47 | Session end: 8 writes across 3 files (config_repl.c, requirements.md, test_convergence_verification.c) | 4 reads | ~69415 tok |
|
||||
| 02:47 | Edited src/controller/testsuite/test_convergence_verification.c | max() → ha_compute_repl_target_lsn() | ~36 |
|
||||
| 02:48 | Edited src/controller/testsuite/test_convergence_verification.c | 3→5 lines | ~82 |
|
||||
| 02:48 | Edited src/controller/testsuite/test_convergence_verification.c | 3→5 lines | ~82 |
|
||||
| 02:49 | Edited .kiro/specs/scheduler-core/requirements.md | modified line() | ~768 |
|
||||
| 02:49 | Session end: 12 writes across 3 files (config_repl.c, requirements.md, test_convergence_verification.c) | 5 reads | ~148861 tok |
|
||||
| 02:49 | Session end: 12 writes across 3 files (config_repl.c, requirements.md, test_convergence_verification.c) | 5 reads | ~148861 tok |
|
||||
| 02:51 | Session end: 12 writes across 3 files (config_repl.c, requirements.md, test_convergence_verification.c) | 5 reads | ~148861 tok |
|
||||
| 02:51 | Edited .kiro/specs/scheduler-core/requirements.md | modified loop() | ~535 |
|
||||
| 02:52 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | removed 6 lines | ~120 |
|
||||
| 02:53 | Edited .kiro/specs/config-upgrade-snapshot/tasks.md | modified used() | ~615 |
|
||||
|
|
|
|||
|
|
@ -2612,8 +2612,14 @@ static int repl_ep_still_registered(replicator_t *r, repl_ep_t ep)
|
|||
return found;
|
||||
}
|
||||
|
||||
/* re-derive the replication target exactly as ha.c:2771-2776 / D6:
|
||||
* target = max(committed_lsn, last_written), last_written = current_lsn - 1. */
|
||||
/* re-derive the replication target via the SAME pure function
|
||||
* repl_ack_decode_and_dispatch (ha.c) uses, so this continuation path and
|
||||
* the heartbeat decision path can never disagree on what a standby must
|
||||
* catch up to (bug-1478: this function used to duplicate the raw
|
||||
* max(committed_lsn, last_written) formula inline, independently of the
|
||||
* ha.c call site the fix landed in — a local-only, never-replicated
|
||||
* WAL_CHECKPOINT trailing an already-advancing committed_lsn permanently
|
||||
* wedged catch-up here as well, via the exact same mechanism). */
|
||||
static uint64_t catchup_derive_target(cluster_t *c)
|
||||
{
|
||||
uint64_t committed = atomic_load_explicit(&c->persist.committed_lsn,
|
||||
|
|
@ -2623,7 +2629,7 @@ static uint64_t catchup_derive_target(cluster_t *c)
|
|||
? c->persist.current_lsn - 1
|
||||
: 0;
|
||||
pthread_mutex_unlock(&c->persist.lock);
|
||||
return last_written > committed ? last_written : committed;
|
||||
return ha_compute_repl_target_lsn(committed, last_written);
|
||||
}
|
||||
|
||||
/* Invoke the catch-up streaming path for [standby_lsn+1, target]. Mirrors the
|
||||
|
|
|
|||
|
|
@ -3673,6 +3673,52 @@ void ha_process_standby_state(ha_state_t *ha, int peer_idx,
|
|||
}
|
||||
}
|
||||
|
||||
/*
|
||||
* ha_compute_repl_target_lsn: the replication/convergence target that
|
||||
* ha_dispatch_repl_action_for_standby's heartbeat-driven decision compares
|
||||
* every standby's reported applied_lsn against.
|
||||
*
|
||||
* The correct target is simply the real, raft/commit-engine-authoritative
|
||||
* committed_lsn — the only position a standby can safely (and eventually)
|
||||
* be told it must catch up to. The ONE documented exception is cluster
|
||||
* BOOTSTRAP: when committed_lsn is genuinely stuck at 0 (no standby has ever
|
||||
* acked anything yet, so the commit engine's majority computation has never
|
||||
* had anything to advance against), last_written (the raw locally-appended
|
||||
* WAL tip) is used instead, so a first-ever connecting standby is told to
|
||||
* catch up to real data instead of being handed "nothing to stream".
|
||||
*
|
||||
* bug-1478: this fallback must apply ONLY in that committed_lsn==0 bootstrap
|
||||
* case — NOT unconditionally whenever last_written happens to exceed
|
||||
* committed_lsn. last_written (persist.current_lsn - 1) counts EVERY
|
||||
* locally-appended WAL entry, including a local-only, NEVER-replicated
|
||||
* WAL_CHECKPOINT (persist_write_checkpoint_marker bumps current_lsn
|
||||
* directly, bypassing raft_log_append/the commit engine entirely — the
|
||||
* same architectural gap flagged in bug-1466). Once such a checkpoint
|
||||
* trails an already-advancing (non-zero) committed_lsn, last_written
|
||||
* permanently sits one LSN past it — and since a checkpoint's own consumed
|
||||
* LSN slot is, by design, never streamed or applied by any standby
|
||||
* (ha_stream_wal_to_standby / persist_wal_write_replicated /
|
||||
* persist_wal_async all explicitly refuse WAL_CHECKPOINT), no standby's
|
||||
* applied_lsn can EVER legitimately reach that value. Taking the
|
||||
* max(committed_lsn, last_written) unconditionally turned the bootstrap
|
||||
* convenience into a permanent, unsatisfiable catch-up target for every
|
||||
* standby the instant a marker-triggered checkpoint (config-upgrade-
|
||||
* snapshot task 5) or a periodic WAL-threshold checkpoint interleaves ahead
|
||||
* of the next real committed advance — ha_evaluate_repl_action's
|
||||
* standby_lsn < committed_lsn check then never reaches HA_REPL_CONVERGED,
|
||||
* and ha_stream_wal_to_standby is re-invoked every heartbeat forever,
|
||||
* finding nothing new to send each time (bug-1478).
|
||||
*
|
||||
* Pure, side-effect-free — safe to unit test directly.
|
||||
*/
|
||||
uint64_t ha_compute_repl_target_lsn(uint64_t committed_lsn,
|
||||
uint64_t last_written)
|
||||
{
|
||||
if (committed_lsn == 0 && last_written > 0)
|
||||
return last_written;
|
||||
return committed_lsn;
|
||||
}
|
||||
|
||||
/* ───────────────────────────────────────────────────────────────────────
|
||||
* raft-drive-async Task 3.2 — SnapshotSend slow-lane offload (bug-1198).
|
||||
*
|
||||
|
|
@ -6376,12 +6422,14 @@ static int repl_ack_decode_and_dispatch(endpoint_t *ep, vote_ep_ctx_t *vctx,
|
|||
oldest_wal_lsn = ha_compute_oldest_wal_lsn(c->persist.wal_path);
|
||||
}
|
||||
#endif
|
||||
/* Replication target: use max(committed_lsn, last_written) so
|
||||
* standbys can catch up even when committed_lsn is stuck at 0
|
||||
* (no replicator-registered standbys to advance it). */
|
||||
uint64_t target_lsn = committed_lsn;
|
||||
if (last_written > target_lsn)
|
||||
target_lsn = last_written;
|
||||
/* Replication target (bug-1478): committed_lsn, with the
|
||||
* last_written bootstrap fallback ONLY when committed_lsn is
|
||||
* genuinely stuck at 0 — see ha_compute_repl_target_lsn's own
|
||||
* doc comment for why an unconditional max() permanently wedges
|
||||
* catch-up once a local-only checkpoint trails a real,
|
||||
* already-advancing committed_lsn. */
|
||||
uint64_t target_lsn =
|
||||
ha_compute_repl_target_lsn(committed_lsn, last_written);
|
||||
|
||||
ha_dispatch_repl_action_for_standby(ha, c, ep, peer_idx,
|
||||
committed_tag, target_lsn,
|
||||
|
|
|
|||
|
|
@ -1127,6 +1127,24 @@ void ha_process_standby_state(ha_state_t *ha, int peer_idx,
|
|||
uint64_t reported_tag, uint64_t reported_lsn,
|
||||
uint64_t committed_tag, uint64_t committed_lsn);
|
||||
|
||||
/*
|
||||
* ha_compute_repl_target_lsn: the replication/convergence target used by
|
||||
* ha_dispatch_repl_action_for_standby's heartbeat-driven decision — normally
|
||||
* committed_lsn itself, falling back to last_written (the raw locally-
|
||||
* appended WAL tip) ONLY when committed_lsn is genuinely stuck at 0
|
||||
* (cluster bootstrap, before any standby has ever acked anything).
|
||||
*
|
||||
* bug-1478: the fallback must NOT apply once committed_lsn is already > 0 —
|
||||
* see ha.c's doc comment on this function for the full mechanism (a
|
||||
* local-only WAL_CHECKPOINT trailing an already-advancing committed_lsn
|
||||
* otherwise leaks an unstreamable/unreachable target into every subsequent
|
||||
* heartbeat decision, permanently wedging catch-up).
|
||||
*
|
||||
* Pure, side-effect-free.
|
||||
*/
|
||||
uint64_t ha_compute_repl_target_lsn(uint64_t committed_lsn,
|
||||
uint64_t last_written);
|
||||
|
||||
/* ── Task 3.5c: Standby snapshot receive handler ──────────────────────────── */
|
||||
|
||||
/*
|
||||
|
|
|
|||
|
|
@ -276,17 +276,22 @@ static int cv_standby_in_catch_up(repl_ep_t ep)
|
|||
|
||||
/* ── Tests ──────────────────────────────────────────────────────────────── */
|
||||
|
||||
/* 5.1: a walAck with acked_lsn < target = max(committed_lsn, last_written)
|
||||
* schedules EXACTLY ONE continuation that streams [acked_lsn+1, target] without
|
||||
* any intervening heartbeat/stateReport dispatch.
|
||||
/* 5.1: a walAck with acked_lsn < target = ha_compute_repl_target_lsn(
|
||||
* committed_lsn, last_written) schedules EXACTLY ONE continuation that
|
||||
* streams [acked_lsn+1, target] without any intervening heartbeat/
|
||||
* stateReport dispatch.
|
||||
* RED: the walAck continuation scaffold schedules nothing → no task, the stream
|
||||
* hook is never called. */
|
||||
static void test_walack_behind_target_schedules_one_continuation(void)
|
||||
{
|
||||
/* appended_lsn=9 → last_written=9; committed_lsn=4 → target=max(4,9)=9. */
|
||||
/* appended_lsn=9 → last_written=9; committed_lsn=9 → target=9. Kept equal
|
||||
* to last_written (rather than a lower value like the historical 4) so
|
||||
* this test exercises catch-up continuation scheduling only — the
|
||||
* committed_lsn-vs-last_written edge case itself (bug-1478) has its own
|
||||
* dedicated pure-function coverage in test_ha_role.c. */
|
||||
cv_setup("behind", /*n_peers=*/3, /*n_standbys=*/1,
|
||||
/*appended_lsn=*/9, /*n_vcpus=*/8);
|
||||
atomic_store(&g_cv.persist.committed_lsn, 4);
|
||||
atomic_store(&g_cv.persist.committed_lsn, 9);
|
||||
|
||||
const int peer_idx = 0;
|
||||
const uint64_t acked_lsn = 3; /* behind target 9 */
|
||||
|
|
@ -325,8 +330,8 @@ static void test_walack_behind_target_schedules_one_continuation(void)
|
|||
"[acked_lsn+1, target]) (Req 5.1)");
|
||||
TEST_ASSERT_EQUAL_UINT64_MESSAGE(target,
|
||||
atomic_load(&g_stream_last_target_lsn),
|
||||
"the continuation must stream up to target = max(committed_lsn, "
|
||||
"last_written) (Req 5.1)");
|
||||
"the continuation must stream up to target = "
|
||||
"ha_compute_repl_target_lsn(committed_lsn, last_written) (Req 5.1)");
|
||||
|
||||
cv_teardown();
|
||||
printf("test_walack_behind_target_schedules_one_continuation: checked\n");
|
||||
|
|
@ -338,9 +343,11 @@ static void test_walack_behind_target_schedules_one_continuation(void)
|
|||
* "exactly one continuation across two ACKs" assertion fails (0 != 1). */
|
||||
static void test_walack_single_flight_no_duplicate_continuation(void)
|
||||
{
|
||||
/* committed_lsn == appended_lsn (target=9 either way) — see the "behind"
|
||||
* test's own note on why these don't use a lower committed_lsn. */
|
||||
cv_setup("single", /*n_peers=*/3, /*n_standbys=*/1,
|
||||
/*appended_lsn=*/9, /*n_vcpus=*/8);
|
||||
atomic_store(&g_cv.persist.committed_lsn, 4);
|
||||
atomic_store(&g_cv.persist.committed_lsn, 9);
|
||||
|
||||
const int peer_idx = 0;
|
||||
|
||||
|
|
@ -382,9 +389,11 @@ static void test_walack_single_flight_no_duplicate_continuation(void)
|
|||
* RED: the scaffold never schedules/clears, so neither continuation appears. */
|
||||
static void test_walack_reschedules_after_continuation_completes(void)
|
||||
{
|
||||
/* committed_lsn == appended_lsn (target=9 either way) — see the "behind"
|
||||
* test's own note on why these don't use a lower committed_lsn. */
|
||||
cv_setup("respawn", /*n_peers=*/3, /*n_standbys=*/1,
|
||||
/*appended_lsn=*/9, /*n_vcpus=*/8);
|
||||
atomic_store(&g_cv.persist.committed_lsn, 4);
|
||||
atomic_store(&g_cv.persist.committed_lsn, 9);
|
||||
|
||||
const int peer_idx = 0;
|
||||
|
||||
|
|
|
|||
|
|
@ -660,6 +660,64 @@ static void test_evaluate_repl_action_stream_wal_fresh_system(void)
|
|||
printf("test_evaluate_repl_action_stream_wal_fresh_system: PASS\n");
|
||||
}
|
||||
|
||||
/* ── bug-1478: ha_compute_repl_target_lsn ─────────────────────────────────
|
||||
*
|
||||
* repl_ack_decode_and_dispatch (ha.c) computes the replication/convergence
|
||||
* target passed into ha_dispatch_repl_action_for_standby as
|
||||
* max(committed_lsn, last_written), where last_written is the raw locally-
|
||||
* appended WAL tip (persist.current_lsn - 1). A live 3-node HA soak
|
||||
* (config-upgrade-snapshot task 8.9, --config-upgrade-wal-trim) caught this
|
||||
* unconditional max() permanently wedging catch-up: once a marker-triggered
|
||||
* checkpoint (a local-only, NEVER-replicated WAL_CHECKPOINT entry —
|
||||
* persist_write_checkpoint_marker bumps current_lsn directly, bypassing
|
||||
* raft_log_append/the commit engine, bug-1466) trails an already-advancing
|
||||
* committed_lsn, last_written sits permanently one LSN past it — a position
|
||||
* no standby's applied_lsn can EVER legitimately reach (the checkpoint's own
|
||||
* slot is, by design, never streamed or applied). ha_evaluate_repl_action's
|
||||
* standby_lsn < committed_lsn check then never reaches HA_REPL_CONVERGED,
|
||||
* and the primary re-streams the same empty range every heartbeat, forever.
|
||||
* Ground-truthed via `xce_test waldump` on the live soak's preserved WAL
|
||||
* (ctrl-2: lsn 21 = the WAL_CONFIG_MARKER, lsn 22 = WAL_CHECKPOINT, 0 bytes)
|
||||
* and via ctrl-2's own commit_advance_locked log (the REAL, majority-gated
|
||||
* committed_lsn never advanced past 21 — only the target_lsn fallback
|
||||
* leaked 22 into the heartbeat decision).
|
||||
*/
|
||||
static void test_compute_repl_target_lsn_bootstrap_fallback(void)
|
||||
{
|
||||
/* committed_lsn genuinely stuck at 0 (no standby has ever acked) — the
|
||||
* documented bootstrap exception: fall back to last_written so a
|
||||
* first-ever connecting standby can catch up to real data. */
|
||||
TEST_ASSERT_EQUAL_UINT64(42, ha_compute_repl_target_lsn(0, 42));
|
||||
printf("test_compute_repl_target_lsn_bootstrap_fallback: PASS\n");
|
||||
}
|
||||
|
||||
static void test_compute_repl_target_lsn_zero_zero(void)
|
||||
{
|
||||
/* Genuinely fresh cluster, nothing appended yet. */
|
||||
TEST_ASSERT_EQUAL_UINT64(0, ha_compute_repl_target_lsn(0, 0));
|
||||
printf("test_compute_repl_target_lsn_zero_zero: PASS\n");
|
||||
}
|
||||
|
||||
static void test_compute_repl_target_lsn_ignores_trailing_checkpoint_lsn(void)
|
||||
{
|
||||
/* bug-1478's exact reproduction shape: committed_lsn=21 is already
|
||||
* advancing (non-zero, real, majority-gated), but last_written=22
|
||||
* reflects a local-only WAL_CHECKPOINT that trails it. The bootstrap
|
||||
* fallback must NOT fire here — the target must stay at the real
|
||||
* committed_lsn (21), never the unstreamable checkpoint slot (22). */
|
||||
TEST_ASSERT_EQUAL_UINT64(21, ha_compute_repl_target_lsn(21, 22));
|
||||
printf("test_compute_repl_target_lsn_ignores_trailing_checkpoint_lsn: PASS\n");
|
||||
}
|
||||
|
||||
static void test_compute_repl_target_lsn_committed_already_ahead(void)
|
||||
{
|
||||
/* Normal steady-state case: committed_lsn is already the higher/current
|
||||
* value; last_written (a possibly-stale, lock-free read) trails it.
|
||||
* The target must simply be committed_lsn. */
|
||||
TEST_ASSERT_EQUAL_UINT64(100, ha_compute_repl_target_lsn(100, 50));
|
||||
printf("test_compute_repl_target_lsn_committed_already_ahead: PASS\n");
|
||||
}
|
||||
|
||||
/* The transitional-leader timeout default test was removed: the config knob
|
||||
* and its default macro are deleted by ha-single-leader-role (the wall-clock
|
||||
* force-promotion watchdog it configured is gone — Req 1.5). */
|
||||
|
|
@ -892,6 +950,11 @@ void test_ha_role(void)
|
|||
RUN_TEST(test_evaluate_repl_action_stream_wal);
|
||||
/* Task 6.1: (tag=T, lsn=0) with oldest_wal_lsn=0 → STREAM_WAL */
|
||||
RUN_TEST(test_evaluate_repl_action_stream_wal_fresh_system);
|
||||
/* bug-1478: ha_compute_repl_target_lsn */
|
||||
RUN_TEST(test_compute_repl_target_lsn_bootstrap_fallback);
|
||||
RUN_TEST(test_compute_repl_target_lsn_zero_zero);
|
||||
RUN_TEST(test_compute_repl_target_lsn_ignores_trailing_checkpoint_lsn);
|
||||
RUN_TEST(test_compute_repl_target_lsn_committed_already_ahead);
|
||||
/* Task 3.3: ha_dispatch_repl_action_for_standby PUSH_CONFIG wiring */
|
||||
RUN_TEST(test_dispatch_repl_action_push_config_when_tag_behind);
|
||||
RUN_TEST(test_dispatch_repl_action_no_push_when_converged);
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue