finalize configuration replication
This commit is contained in:
parent
e1517bbf28
commit
fa66d1ba9c
30 changed files with 8867 additions and 1560 deletions
|
|
@ -1,6 +1,40 @@
|
|||
{
|
||||
"hooks": {
|
||||
"PostToolUse": [
|
||||
{
|
||||
"matcher": "Edit|Write|Bash",
|
||||
"hooks": []
|
||||
},
|
||||
{
|
||||
"matcher": "Read",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"$CLAUDE_PROJECT_DIR/.wolf/hooks/post-read.js\"",
|
||||
"timeout": 5
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": "Write|Edit|MultiEdit",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"$CLAUDE_PROJECT_DIR/.wolf/hooks/post-write.js\"",
|
||||
"timeout": 10
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"SessionStart": [
|
||||
{
|
||||
"matcher": "",
|
||||
"hooks": []
|
||||
},
|
||||
{
|
||||
"matcher": "",
|
||||
"hooks": []
|
||||
},
|
||||
{
|
||||
"matcher": "",
|
||||
"hooks": [
|
||||
|
|
@ -34,28 +68,6 @@
|
|||
]
|
||||
}
|
||||
],
|
||||
"PostToolUse": [
|
||||
{
|
||||
"matcher": "Read",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"$CLAUDE_PROJECT_DIR/.wolf/hooks/post-read.js\"",
|
||||
"timeout": 5
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": "Write|Edit|MultiEdit",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node \"$CLAUDE_PROJECT_DIR/.wolf/hooks/post-write.js\"",
|
||||
"timeout": 10
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"Stop": [
|
||||
{
|
||||
"matcher": "",
|
||||
|
|
|
|||
|
|
@ -58,11 +58,11 @@ bash src/tools/deploy-ha.sh "$BINARY" "$DATADIR" "$TOP" "$PEERS"
|
|||
```
|
||||
|
||||
This:
|
||||
- Creates `$TOP/ctrl-{1..N}/conf`, `ctrl-{1..N}/data`, `ctrl-{1..N}/certs`, `logs/`
|
||||
- Copies `xce.crt` and `xce.key` into each controller's `certs/` dir
|
||||
- Places `cluster.bin.<tag>` + `conf/current` in `ctrl-1/conf/` only
|
||||
- Creates `$TOP/certs/`, `$TOP/controller/ctrl-{1..N}/conf`, `$TOP/controller/ctrl-{1..N}/data`, `$TOP/logs/`
|
||||
- Copies `xce.crt` and `xce.key` into the shared `$TOP/certs/` dir
|
||||
- Places `cluster.bin.<tag>` + `conf/current` in `controller/ctrl-1/conf/` only
|
||||
(standbys receive config from primary via `OPCode_replConfigPush`)
|
||||
- Generates `$TOP/scripts/start.sh` and `cleanup.sh`
|
||||
- Generates `$TOP/scripts/start.sh` (with `--workdir $TOP/controller --certdir $TOP/certs`) and `cleanup.sh`
|
||||
|
||||
## Step 4 — Start all nodes
|
||||
|
||||
|
|
@ -88,7 +88,7 @@ Wait ~15 seconds for election and config replication, then check:
|
|||
grep -i "CTRL_PRIMARY\|push_config\|Config committed" "$TOP/logs/ctrl-1.log" | grep -v DEBUG3
|
||||
|
||||
# 2. Confirm ctrl-2 and ctrl-3 received config
|
||||
find "$TOP/ctrl-2/conf" "$TOP/ctrl-3/conf" -type f | sort
|
||||
find "$TOP/controller/ctrl-2/conf" "$TOP/controller/ctrl-3/conf" -type f | sort
|
||||
|
||||
# 3. Confirm standbys set primary_ctrl_fd
|
||||
grep "primary control channel established" "$TOP/logs/ctrl-2.log" "$TOP/logs/ctrl-3.log"
|
||||
|
|
@ -97,7 +97,7 @@ grep "primary control channel established" "$TOP/logs/ctrl-2.log" "$TOP/logs/ctr
|
|||
**Healthy cluster signs:**
|
||||
- `ctrl-1.log`: `→ CTRL_PRIMARY_COMMITTING` then `→ CTRL_PRIMARY`
|
||||
- `ctrl-1.log`: `Config committed on majority (tag=..., pushed=2/2 standbys)`
|
||||
- `ctrl-2/conf/cluster.bin.*` and `ctrl-3/conf/cluster.bin.*` exist
|
||||
- `controller/ctrl-2/conf/cluster.bin.*` and `controller/ctrl-3/conf/cluster.bin.*` exist
|
||||
- `ctrl-2.log` / `ctrl-3.log`: `handle_repl_config_push: config saved and loaded`
|
||||
|
||||
## Step 6 — Raft trace (DEBUG3 logs with -vvv)
|
||||
|
|
@ -120,8 +120,8 @@ Stop all instances, clean up state, and restart without redeploying:
|
|||
kill -9 $(pgrep xce_controller) 2>/dev/null || true
|
||||
|
||||
# 2. Run the generated cleanup script
|
||||
# Keeps: ctrl-1/conf/* (primary config + current pointer), all certs
|
||||
# Wipes: ctrl-{1..N}/data/*, ctrl-{2..N}/conf/*, logs/*.log
|
||||
# Keeps: controller/ctrl-1/conf/* (primary config + current pointer), $TOP/certs/
|
||||
# Wipes: controller/ctrl-{1..N}/data/*, controller/ctrl-{2..N}/conf/*, logs/*.log
|
||||
bash "$TOP/scripts/cleanup.sh"
|
||||
|
||||
# 3. Restart all nodes
|
||||
|
|
@ -133,13 +133,12 @@ VERBOSE=-vvv "$TOP/scripts/start.sh" ctrl-3 &
|
|||
```
|
||||
|
||||
**What cleanup.sh preserves:**
|
||||
- `ctrl-1/certs/` — TLS cert and key
|
||||
- `ctrl-1/conf/cluster.bin.*` + `conf/current` — primary config (standbys re-receive via push)
|
||||
- `ctrl-{2,3}/certs/` — TLS cert and key
|
||||
- `$TOP/certs/` — shared TLS cert and key
|
||||
- `controller/ctrl-1/conf/cluster.bin.*` + `conf/current` — primary config (standbys re-receive via push)
|
||||
|
||||
**What cleanup.sh removes:**
|
||||
- All `data/` dirs (WAL, snapshots, `ha-term.bin`) — forces fresh election next start
|
||||
- `ctrl-{2,3}/conf/` — standbys rebuild config from primary's `OPCode_replConfigPush`
|
||||
- All `controller/ctrl-{1..N}/data/` dirs (WAL, snapshots, `ha-term.bin`) — forces fresh election next start
|
||||
- All `controller/ctrl-{2..N}/conf/` — standbys rebuild config from primary's `OPCode_replConfigPush`
|
||||
- All `logs/*.log` — clean log slate
|
||||
|
||||
If you need a full redeploy (different peers or new binary), run steps 3–5 instead.
|
||||
|
|
@ -149,7 +148,7 @@ If you need a full redeploy (different peers or new binary), run steps 3–5 ins
|
|||
| Symptom | Cause | Fix |
|
||||
|---|---|---|
|
||||
| `cluster_open_listener: no port` | Peers missing from `--peers` | Ensure `name=host:port` format in generated start.sh |
|
||||
| `ctrl-2/3/conf` empty | `push_config_to_standbys` bug or `OPCode_replConfigPush` not handled | Verify binary is latest build |
|
||||
| `controller/ctrl-2/3/conf` empty | `push_config_to_standbys` bug or `OPCode_replConfigPush` not handled | Verify binary is latest build |
|
||||
| `repl_ack_decode_and_dispatch: unexpected opcode` | Heartbeat NOP on promoted ep | Fixed in current build; update binary |
|
||||
| `election_timeout` loops without primary | All nodes have `config_tag=0` (cold-start) or network issue | Check peers string matches actual listen ports |
|
||||
| `ha-term.bin` causes "already voted" on restart | Old state from previous run | Run clean restart above |
|
||||
|
|
|
|||
|
|
@ -1 +0,0 @@
|
|||
{"detected":false,"exe":null,"args":null,"extensions":[]}
|
||||
Loading…
Add table
Add a link
Reference in a new issue