wdgzyjx/myxce
1
0
Fork 0
forked from lei.chen/myxce
Code Pull requests Activity

chore(wolf): record rule — never push to Rongsong's branch without permission

Browse source 
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
lei.chen 2026-08-01 03:41:47 +00:00
parent d6ae04f47f
commit b20f2abb27
2 changed files with 2 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
.wolf/cerebrum.md
View file

@ -10,6 +10,7 @@
- Run soak tests with specific round counts and deploy directories for precise regression tracking - Run soak tests with specific round counts and deploy directories for precise regression tracking
- Prefer parallel agent runs for independent analysis work; sequential for dependent tasks - Prefer parallel agent runs for independent analysis work; sequential for dependent tasks
- Keep test logs for analysis, don't skip intermediate output even if a test hangs - Keep test logs for analysis, don't skip intermediate output even if a test hangs
- NEVER push code to Rongsong's branch (`origin/working`, rshen/xce.git) unless the user explicitly allows it. We only sync Rongsong's code INTO our own branch (local working / myxce fork). When "push" is mentioned without a target repo, default to myxce (user's fork) and confirm first; `git push origin working` requires explicit permission (2026-08-01).
## Key Learnings ## Key Learnings
- (2026-07-28, bug-1479 ROOT-CAUSED not fixed) A permanent quorum-loss family member unrelated to admission control: when repl.c's `repl_broadcast` (live single-entry WAL tail-follow) gets an `ae_reject_logmismatch` NACK back from a standby (a genuinely NORMAL Raft outcome after rapid leader-election churn leaves a conflicting uncommitted tail entry at the same LSN with a different term — confirmed byte-precise via `xce_test waldump`: ctrl-2's own lsn=13 was its own deposed term=3 noop, while the canonical/surviving log has a DIFFERENT term=4 noop at the same lsn), NOTHING in ha.c's `repl_ack_decode_and_dispatch` NACK branch (`!success`) does anything but check for a higher observed term and then literally `WARN("...success=0 — ignoring")` — no `repl_on_ack` call (the ONLY site that decrements `pending_ack_count`), no nextIndex backoff-and-retry, no catch-up trigger. Consequence: `pending_ack_count` climbs 1-per-broadcast-attempt with zero decrements until `repl_broadcast`'s own `MAX_PENDING_ACKS=16` defensive eviction fires and marks the standby `in_catch_up=1` — which correctly excludes it from `repl_active_standby_count`/quorum, but NOTHING then drives an actual catch-up stream/snapshot-install for it (the only other `repl_complete_catchup` call site, config_repl.c's ACK-driven `catchup_lane_fn`, can never engage a standby that never sends a genuine ACK in the first place). Net effect: a standby with ONE conflicting log entry from election churn (routine, expected) becomes PERMANENTLY excluded from quorum with no self-healing path — in a 3-node cluster this is one unlucky election sequence away from total write-quorum loss. **General lesson**: when a symptom looks like "resource exhaustion" (admission cap, connection pool, whatever), always verify the actual counter's INCREMENT and DECREMENT sites are symmetric across EVERY reply outcome (success AND failure/reject), not just the happy path — a counter that only decrements on success but increments unconditionally will silently drift to its ceiling under ANY sustained failure condition, which looks identical to a genuine leak from the outside (repeated timeouts / repeated rejects) but has a completely different fix (handle the reject/failure path, not "find the missing release call"). Confirmed the ORIGINAL admission-slot-leak hypothesis for this bug was WRONG by checking `commit_resolve()`/`wait_release_locked()` — those DO release correctly on every timeout, 1:1 (17 timeouts, 17 releases) — before chasing the real mechanism. - (2026-07-28, bug-1479 ROOT-CAUSED not fixed) A permanent quorum-loss family member unrelated to admission control: when repl.c's `repl_broadcast` (live single-entry WAL tail-follow) gets an `ae_reject_logmismatch` NACK back from a standby (a genuinely NORMAL Raft outcome after rapid leader-election churn leaves a conflicting uncommitted tail entry at the same LSN with a different term — confirmed byte-precise via `xce_test waldump`: ctrl-2's own lsn=13 was its own deposed term=3 noop, while the canonical/surviving log has a DIFFERENT term=4 noop at the same lsn), NOTHING in ha.c's `repl_ack_decode_and_dispatch` NACK branch (`!success`) does anything but check for a higher observed term and then literally `WARN("...success=0 — ignoring")` — no `repl_on_ack` call (the ONLY site that decrements `pending_ack_count`), no nextIndex backoff-and-retry, no catch-up trigger. Consequence: `pending_ack_count` climbs 1-per-broadcast-attempt with zero decrements until `repl_broadcast`'s own `MAX_PENDING_ACKS=16` defensive eviction fires and marks the standby `in_catch_up=1` — which correctly excludes it from `repl_active_standby_count`/quorum, but NOTHING then drives an actual catch-up stream/snapshot-install for it (the only other `repl_complete_catchup` call site, config_repl.c's ACK-driven `catchup_lane_fn`, can never engage a standby that never sends a genuine ACK in the first place). Net effect: a standby with ONE conflicting log entry from election churn (routine, expected) becomes PERMANENTLY excluded from quorum with no self-healing path — in a 3-node cluster this is one unlucky election sequence away from total write-quorum loss. **General lesson**: when a symptom looks like "resource exhaustion" (admission cap, connection pool, whatever), always verify the actual counter's INCREMENT and DECREMENT sites are symmetric across EVERY reply outcome (success AND failure/reject), not just the happy path — a counter that only decrements on success but increments unconditionally will silently drift to its ceiling under ANY sustained failure condition, which looks identical to a genuine leak from the outside (repeated timeouts / repeated rejects) but has a completely different fix (handle the reject/failure path, not "find the missing release call"). Confirmed the ORIGINAL admission-slot-leak hypothesis for this bug was WRONG by checking `commit_resolve()`/`wait_release_locked()` — those DO release correctly on every timeout, 1:1 (17 timeouts, 17 releases) — before chasing the real mechanism.

1
.wolf/memory.md
View file

@ -7262,3 +7262,4 @@
| 02:31 | Session end: 35 writes across 11 files (_live_e2e_common.py, test_e2e_live_push.py, test_e2e_live_ha_push.py, manual_ha_deploy.sh, test_repro_bug1485.py) | 30 reads | ~366672 tok | | 02:31 | Session end: 35 writes across 11 files (_live_e2e_common.py, test_e2e_live_push.py, test_e2e_live_ha_push.py, manual_ha_deploy.sh, test_repro_bug1485.py) | 30 reads | ~366672 tok |
| 02:32 | Session end: 35 writes across 11 files (_live_e2e_common.py, test_e2e_live_push.py, test_e2e_live_ha_push.py, manual_ha_deploy.sh, test_repro_bug1485.py) | 30 reads | ~366672 tok | | 02:32 | Session end: 35 writes across 11 files (_live_e2e_common.py, test_e2e_live_push.py, test_e2e_live_ha_push.py, manual_ha_deploy.sh, test_repro_bug1485.py) | 30 reads | ~366672 tok |
| 03:04 | Merged origin/working (64 commits: config-upgrade-snapshot feature + HA/repl bug fixes), deduped capnpc override — kept xce.nix fetchFromGitHub pin (bug-918 canonical fix), dropped flake.nix ghfast.top overlay | src/flake.nix, src/nixSupport/xce.nix | merged clean, 0 conflicts; nix files parse OK | ~1200 | | 03:04 | Merged origin/working (64 commits: config-upgrade-snapshot feature + HA/repl bug fixes), deduped capnpc override — kept xce.nix fetchFromGitHub pin (bug-918 canonical fix), dropped flake.nix ghfast.top overlay | src/flake.nix, src/nixSupport/xce.nix | merged clean, 0 conflicts; nix files parse OK | ~1200 |
| 03:50 | User rule: never push to Rongsong's branch (origin/working) without explicit permission; we only sync his code into our own branch. Written to Claude memory (xce-push-policy-rongsong) + cerebrum User Preferences | .wolf/cerebrum.md, /home/lchen/.claude/projects/-home-lchen-code/memory/ | persisted both stores | ~400 |