feat: add --peers to controller for HA peer list and priority election

- myconf_t gains peers_cfg (xce_cluster_config_t) populated from --peers
- controller/main.c: parse --peers host:port,... via argp; ca_cert_path
  auto-derived from workdir/certs/xce.crt (no --ca-cert flag needed)
- cluster_init: derive my_priority from position in peers list (index 0 =
  primary); fall back to 0 when --peers not given (single-node mode)
This commit is contained in:
Rongsong Shen 2026-05-08 04:40:27 +00:00
parent d61f91b99c
commit 59597e105e
10 changed files with 8046 additions and 325 deletions

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,39 @@
# XCE Persistence & HA — Implementation Overview
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make the XCE controller survive restarts and primary failures with no job/worker/queue state loss.
**Architecture:** WAL + periodic checkpoint (dual-zarray COW) for single-controller persistence; LSF-style priority election + fan-out TCP replication for HA. libxce smart discovery handles client auto-reconnect after failover.
**Tech Stack:** C (controller/lib), Cap'n Proto (serialization), Nix flakes + meson + ninja (build), Go (API server discovery bridge)
---
## Phase Map
Each phase is an independently shippable increment. Create the next phase plan only after the current phase passes its gate tests.
| Phase | Plan file | Goal | Gate |
|-------|-----------|------|------|
| **P0** | `2026-05-07-p0-single-controller-persistence.md` | Controller survives restart | `P0-G` smoke tests green |
| **P1** | _create after P0 ships_ | Standby takes over when primary fails | Election + promotion tests green |
| **P2** | _create after P1 ships_ | Primary replicates to standbys in real time | Replication round-trip + catch-up tests green |
| **P2.5** | _create alongside P2_ | Workers/schedulers auto-reconnect after failover | Discovery + reconnect tests green |
| **P3** | _create after P2 ships_ | End-to-end failover + hardening + metrics | Full failover test + benchmark passing |
| **P4** | _create after P3 ships_ | Docs, runbook, stale-ref cleanup | Runbook review complete |
## Out of Scope (all phases)
- Admin CLI for manual failover (`xce hostcontrol --failover`)
- WAL compaction beyond checkpoint trimming
- Multi-DC replication
- Runtime config changes without restart
- Snapshot compression
- `config_version` hash in heartbeat (deferred to P3 tooling)
- Degraded-mode restricted-state timeout (`repl_degraded_timeout_s`)
- `MAX_HA_CANDIDATES > 4` / `MAX_STANDBYS > 3`
## Design Document
`src/docs/persistence-and-ha-design.md` — all implementation detail lives here. Read the relevant Part before starting each task.