Commit graph

9 commits

Author SHA1 Message Date
9f539583d3
docs: sync skills and steering docs with bug-1562..1566 findings
- .claude/skills/xce-build/SKILL.md: --incremental flag,
  xceadmin wheel building, corrected the standalone-install
  guidance (the earlier --system-site-packages venv approach was
  wrong -- nix devShell packages are $PYTHONPATH entries, not
  merged into site-packages).
- .claude/skills/xce-deploy-ha/SKILL.md: $BASH_BIN resolution,
  worker/scheduler startup, xceadmin usage against a deployment,
  troubleshooting rows for the newly-fixed bugs.
- .kiro/steering/ha-async-io.md: bug-1562's standby liveness
  watchdog re-arm invariant.
- .kiro/steering/persistence.md: bug-1563 (fsync backoff) and
  bug-1564 (persist_flush durable-advance callback).
- .kiro/steering/build-deployment.md: xceadmin wheel docs,
  bug-1565's --incremental flag, deploy dir tree updates.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-17 10:21:25 +00:00
d70bf6a402 update documents 2026-08-10 17:51:51 +08:00
d8d1901901 docs(steering): sync steering + xce-audit-invariants skill after Jul 13-18 work
- ha-async-io.md: document the same-batch endpoint lifetime hazard
  (bug-1447) and its ref-whole-batch-before-dispatch fix pattern
- xce-audit-invariants SKILL.md: add object-lifetime-within-a-dispatch-batch
  as a 6th audit dimension, so a future pass checks other batch loops
  (vtaskmgr drain, peer-list iteration) for the same shape
- product.md: replace the stale "HA consistency family is the current
  focus" framing with the actual 2026-07-18 status (umbrella complete,
  9-spec validate-impl batch closed, job-array-single-wal-record shipped)
- roadmap.md: correct 5 entries that were stale relative to actual spec
  status (ha-consistency-invariants/ha-wal-replication/libxce-failover
  pulled out of "In Flight", ha-term-safety and raft-drive-async marked
  implemented+GO, raft-sim-conformance and job-array-single-wal-record
  given closure annotations) -- struck through, not deleted, per this
  file's own additive-update convention
- testing.md: document that meson's aggregate Fail:0 can mask Unity
  sub-group failures -- always check the standalone binary's own summary
- src/controller/CLAUDE.md: fix a stale line-citation for the votedFor
  atomicity invariant (had drifted onto unrelated post-election code)
2026-07-19 08:14:11 +00:00
40f510fb14 fix(controller): bug-1433 async-thread fdatasync block + bug-1434 committed_lsn CAS race
bug-1433: raft_cb_become_leader runs synchronously off peer_ctrl_dispatch_to_pool,
which hardcodes VTASK_LIVENESS for every inbound peer-control packet including
vote replies. The bug-1431 round-2 fix (flusher_flush_once force-flush in
persist_promotion_advance_committed) was reachable from that never-blocks task
class -- same bug family as bug-1198. Removed the force-flush; round-3
(durable-capped only) plus the already-shipped bug-1432 immediate-first-heartbeat
fix close the visibility-latency gap non-blockingly. TRACKED RESIDUAL:
persist_apply_pump_run_sync is still reachable on the same VTASK_LIVENESS path
and does bounded pread() -- lower risk (page-cache-hot) but strictly still
blocking I/O; not safe to remove outright (reopens bug-1043-family job loss),
needs its own follow-up (reclassify peer_ctrl_classify's winning-vote-reply case
or dispatch promotion tail onto a fresh VTASK_LIVE_BLOCK vtask).

bug-1434: persist_promotion_advance_committed and ha_standby_advance_committed_and_apply
raced on persist_t.committed_lsn via an unlocked load-then-conditional-store during
the promotion window (before ha->role flips to CTRL_PRIMARY) -- a losing racer could
regress committed_lsn. Added persist_committed_lsn_advance_atomic (CAS retry loop,
lock-free) and routed both call sites through it.

Validated: 19/19 unit test suites, 6-round chaotic soak (0 loss/divergence) + 6-round
sequential+leadership-coverage soak, both PASS. Also folds in raft-apply-decoupling
and raft-ha-core-safety validate-impl phase bumps to "implemented" and clears
config-upgrade-snapshot's now-satisfied entity-bearing-snapshots block.
2026-07-11 12:16:28 +00:00
5f307e297b docs: promote raft-apply-decoupling spec family from /kiro-discovery
Adds the Raft Apply-Ordering & Hook Decoupling roadmap phase: canonical
pre-hooks -> raft commit -> apply -> notify ordering, with the raft protocol
step lifted out of the notification hook fan-out (root of the bug-795/828/
851/1060 divergence family). Decision: apply-after-commit via a single
shared apply_committed_entry() seam reused by online apply and restart
replay.

- raft-drive-async: full spec docs (brief/requirements/research/design)
  committed alongside its already-tracked tasks.md; async-drive prerequisite
  for the apply-decoupling work, sequenced to start first (threatens
  failover liveness today).
- raft-apply-decoupling: new spec (brief + facts-in-raft.md) covering the
  apply seam and the facts (WAL_MODE_RETAIN) apply-timing question (open
  decision D1).
- job-array-single-wal-record: promoted from backlog to a tracked spec,
  sequenced after raft-apply-decoupling.
- ha-consistency-invariants: add criterion 2.9 (Facts-Apply invariant),
  documenting the current on-receipt fact-reaction deviation this family
  will reconcile.
- raft-commit-safety / raft-log-replication: reconcile spec.json phase to
  "implemented" (both were mislabeled tasks-generated despite full 18/18
  and 24/24 task completion + raft_sim GO).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SDWVNQLKxqTVHh42cH7rV3
2026-07-03 02:41:11 +00:00
4338a97d0e docs(steering): sync HA protocol with ha-term-safety — per-entry term, (term,lsn) reconcile, commit-source ownership, bug-828 truncate rule 2026-06-23 04:48:06 +00:00
59d9df246c fix(ha-commit-durability)!: controller never self-terminates — loud non-promotable fence replaces recovery FATAL/abort
Per design invariant (a controller exits only on crash or admin kill), replace the
self-abort recovery terminal (tasks 3.3/4.3/4.4) with a loud, permanent, non-promotable
fence that stays alive and keeps retrying catch-up (self-heals on peer return; no manual
admin recovery). Records the never-self-terminate invariant as project-wide guidance in
src/controller/CLAUDE.md + .kiro/steering/ha-async-io.md.
2026-06-18 11:10:19 +00:00
dbc631664d docs(ha-single-leader-role): sync steering + contributor docs to three-state model + own-term gate (task 6.1) 2026-06-13 14:58:55 +00:00
06135954ff chore: baseline snapshot — prior-session work + ha-consistency-invariants spec family
Pre-existing uncommitted work: ha-repl-hardening payload data plane (7.x),
heartbeat liveness fix (Req 1.3), connection-reuse ASAN fixes, doc cleanup.
New: ha-consistency-invariants umbrella (requirements/design/research) and
its four implementation sub-specs (specs generated via batch, cross-spec
review applied), steering roadmap + product.md Raft+CP decision.
2026-06-11 13:21:50 +00:00