61e1391cd0
fix bug-1060 bug-1067
2026-06-24 17:05:43 +08:00
3ad646ad0e
spec(ha-commit-durability): REOPEN for bug-795 total-outage cold-restart-all committed-job loss
...
Phase 6 added (6.1 reproduce-on-real-fs gates 6.2 root-cause / 6.3 fix / 6.4 revalidate).
Surfaced by the xce-test-e2e-primitives cold-restart-all soak step; Phase 5 only
validated rolling single-node failover. CLAUDE.md durability claim qualified pending 6.x.
2026-06-20 00:33:18 +00:00
f139e1c08e
docs(ha-commit-durability): flip bug-661 KNOWN GAP to GUARANTEED + re-verify protocol checklist (task 5.3)
2026-06-18 11:15:13 +00:00
59d9df246c
fix(ha-commit-durability)!: controller never self-terminates — loud non-promotable fence replaces recovery FATAL/abort
...
Per design invariant (a controller exits only on crash or admin kill), replace the
self-abort recovery terminal (tasks 3.3/4.3/4.4) with a loud, permanent, non-promotable
fence that stays alive and keeps retrying catch-up (self-heals on peer return; no manual
admin recovery). Records the never-self-terminate invariant as project-wide guidance in
src/controller/CLAUDE.md + .kiro/steering/ha-async-io.md.
2026-06-18 11:10:19 +00:00
dbc631664d
docs(ha-single-leader-role): sync steering + contributor docs to three-state model + own-term gate (task 6.1)
2026-06-13 14:58:55 +00:00
06135954ff
chore: baseline snapshot — prior-session work + ha-consistency-invariants spec family
...
Pre-existing uncommitted work: ha-repl-hardening payload data plane (7.x),
heartbeat liveness fix (Req 1.3), connection-reuse ASAN fixes, doc cleanup.
New: ha-consistency-invariants umbrella (requirements/design/research) and
its four implementation sub-specs (specs generated via batch, cross-spec
review applied), steering roadmap + product.md Raft+CP decision.
2026-06-11 13:21:50 +00:00
fa66d1ba9c
finalize configuration replication
2026-06-09 16:40:13 +08:00
e31673dd88
chore: checkpoint all prior work for clean state
...
Captures all uncommitted work from ha-config-bootstrap,
ha-heartbeat-liveness, and associated spec/tooling updates:
- src: ha.c, ha.h, cluster.c, config_repl.c, myapp.c, raft.c —
heartbeat liveness, RDHUP integration, committing phase fixes
- tests: test_ha_role.c, test_config_repl.c, test_ha_rdhup_integration.c,
test_p1_integration.c, test_p3_hardening.c, scheduler test
- specs: election-iomgr-refactor, ha-heartbeat-liveness,
ha-peer-connection-reuse, raft-library (new); ha-config-bootstrap
and election-iomgr-refactor task/research updates
- tooling: .wolf/ state, ha/data cluster fixtures, raft/raft.tla,
src/build symlink, xce-build skill update
2026-06-04 13:04:16 +00:00
78b7942bbe
docs(ha): use generated cleanup.sh in clean restart; clarify what it preserves
...
cleanup.sh is generated by deploy-ha.sh and already does the right thing:
keeps ctrl-1/conf/* and all certs, wipes ctrl-{2..N}/conf, all data/, and logs.
Update skill and CLAUDE.md to use it as the primary clean-restart command
instead of manual rm loops.
2026-06-01 06:03:34 +00:00
484b110009
docs(ha): add xce-deploy-ha skill and update controller CLAUDE.md
...
- .claude/skills/xce-deploy-ha/SKILL.md: new skill for deploying a
local 3-node XCE HA controller cluster. Covers build prerequisites,
deploy via src/tools/deploy-ha.sh, node startup with -vvv raft trace,
health verification checks, clean restart procedure, and a
troubleshooting table for common failure modes.
- src/controller/CLAUDE.md: add HA Cluster Deployment section with
quick-deploy commands, health check one-liners, and reference to the
xce-deploy-ha skill. Add src/tools/deploy-ha.sh to key files.
2026-06-01 06:01:13 +00:00
7b39650229
refactor(ha): passive standby model — primary connects outbound to all peers
...
Remove fetch_config_from_primary, ha_find_and_connect_to_primary, and the
tier-1/2/3 discovery path. Standbys never initiate outbound connections to
the primary; the primary owns all inter-controller channels.
Key changes:
- ha.c: add primary heartbeat loop (ha_start_primary_heartbeat) that sends
OPCode_nop to connected standbys every heartbeat_interval_ms; loop
self-cancels on demotion via role check before re-arm.
- ha.c: add outbound reconnect in primary_heartbeat_fn — for peers absent
from standby_endpoints, open fresh outbound TLS connection (on_standby_connected
sends ctrlhello; standby_ep_ctx_t mirrors vote_ep_ctx_t layout).
- ha.c: remove ha_find_and_connect_to_primary + helpers (ha_raw_tcp_connect,
ha_tcp_connect_hook); remove election_timeout_fn (superseded by
raft_election_timeout_fn); remove fetch_config_from_primary call from
ha_elect_evaluate_and_signal STANDBY branch.
- cluster.c: OPCode_requestVote denial no longer reuses inbound connection;
late joiners are bootstrapped via primary heartbeat outbound connect.
- config_repl.c: remove connect_ssl_raw, do_one_config_fetch,
fetch_config_from_primary (~250 LOC).
- myapp.c: remove my_tag > 0 fetch branch; all standbys wait passively.
- ha.h: remove ha_find_and_connect_to_primary declaration + macros +
ha_tcp_connect_hook seam; add ha_start_primary_heartbeat declaration.
- meson.build: remove test_ha_connect.c (tests deleted function).
- controller_test.c: remove test_ha_connect registration.
- CLAUDE.md, src/controller/CLAUDE.md: document passive standby model,
inbound-vs-outbound connection ownership, async I/O constraint,
heartbeat model.
- request.h: add xce_request declaration (removes scattered local externs).
2026-06-01 04:17:05 +00:00
c8e0f65a39
enhance xce-build skill
2026-05-30 08:13:59 +08:00
c9dc8c4676
Merge Go frontend (apiserver + xcecli) into meson build
2026-04-14 11:23:18 +08:00
