docs(specs): gap-analysis validation pass across all 9 HA-family specs

Appended "Gap Analysis — Validation Pass (2026-06-09)" to each spec's
research.md with requirement-to-asset maps, status tags, and effort/risk.

Key outcomes:
- ha-repl-hardening: REOPENED with task series 7.1-7.7 — replication
  payload data plane is missing (snapshot chunks and WAL catch-up ship
  header-only; standbys converge on LSN without holding entity state)
- worker-join-refactor, resval-id-decoupling: marked implemented
  (all criteria re-verified, full test suite green)
- buglog: 335/336 (scheduler dequeue, open), 337/338/339 (fixed in
  previous commit); cerebrum Decision Log corrected (term-only vote
  deny was recorded as intentional but was a regression)
This commit is contained in:
Rongsong Shen 2026-06-10 01:26:08 +00:00
parent 9821ec4f3c
commit cdd446fc17
21 changed files with 3955 additions and 288 deletions

View file

@ -1,30 +1,24 @@
# anatomy.md
> Auto-maintained by OpenWolf. Last scanned: 2026-06-09T08:36:14.668Z
> Files: 590 tracked | Anatomy hits: 0 | Misses: 0
## ../../../../../claude/.claude/plans/
- `functional-tumbling-squirrel.md` — Plan: Address ha-repl-hardening Design Review Issues (~924 tok)
- `hazy-exploring-swan.md` — Plan: HA Replication Hardening (~1100 tok)
- `prancy-herding-nest.md` — Plan: HA Election + Config Bootstrap Enhancement (~13241 tok)
## ../../../../../claude/.claude/projects/-home-rshen-projects-xce-xce/memory/
- `MEMORY.md` — Memory Index (~41 tok)
- `project_ha_bootstrap_progress.md` (~363 tok)
> Auto-maintained by OpenWolf. Last scanned: 2026-06-10T01:00:00.579Z
> Files: 505 tracked | Anatomy hits: 0 | Misses: 0
## ./
- `.clang-format` (~226 tok)
- `.gitignore` — Git ignore rules (~88 tok)
- `CLAUDE.md` — OpenWolf (~2900 tok)
- `.opencode.json` (~55 tok)
- `CLAUDE.md` — OpenWolf (~2137 tok)
- `package-lock.json` — npm lock file (~1039 tok)
- `package.json` — Node.js package manifest (~1 tok)
- `README.md` — Project documentation (~612 tok)
## .claude/
- `settings.json` (~441 tok)
- `settings.local.json` (~30 tok)
- `settings.json` (~497 tok)
- `settings.json.bak` (~329 tok)
- `settings.local.json` (~137 tok)
- `ucai-formatter-cache.local.json` (~17 tok)
## .claude/rules/
@ -183,11 +177,18 @@
## .claude/skills/xce-deploy-ha/
- `SKILL.md` — XCE HA Cluster Deployment (~1286 tok)
- `SKILL.md` — XCE HA Cluster Deployment (~1334 tok)
## .kiro/settings/
## .code-review-graph/
- `mcp.json` (~8 tok)
- `.gitignore` — Git ignore rules (~38 tok)
## .codegraph/
- `.gitignore` — Git ignore rules (~47 tok)
- `codegraph.db-shm` (~8738 tok)
- `daemon.log` (~1092 tok)
- `daemon.pid` (~44 tok)
## .kiro/settings/rules/
@ -229,11 +230,15 @@
- `structure.md` — Project Structure (~218 tok)
- `tech.md` — Technology Stack (~198 tok)
## .kiro/specs/.claude/
- `ucai-formatter-cache.local.json` (~17 tok)
## .kiro/specs/election-iomgr-refactor/
- `design.md` — Design Document: election-iomgr-refactor (~6671 tok)
- `requirements.md` — Requirements Document (~2695 tok)
- `research.md` — Research & Design Decisions (~5189 tok)
- `research.md` — Research & Design Decisions (~8796 tok)
- `spec.json` (~129 tok)
- `tasks.md` — Implementation Plan (~3693 tok)
@ -242,9 +247,9 @@
- `brief.md` — Brief: ha-election-raft-config (~16310 tok)
- `design.md` — Design: HA Config Bootstrap & Raft-Style Quorum Election (~11439 tok)
- `election-trace.md` — Election Trace: 3-Node Cold Start (2026-06-04 00:15:58) (~2907 tok)
- `manual-tests.md` — Manual Integration Tests: ha-config-bootstrap (~3677 tok)
- `manual-tests.md` — Manual Integration Tests: ha-config-bootstrap (~3748 tok)
- `requirements.md` — Requirements: HA Config Bootstrap & Raft-Style Quorum Election (~3890 tok)
- `research.md` — Cross-Spec Gap Audit (2026-06-03) (~8392 tok)
- `research.md` — Cross-Spec Gap Audit (2026-06-03) (~14349 tok)
- `spec.json` (~198 tok)
- `tasks.md` — Implementation Plan: HA Config Bootstrap & Raft-Style Quorum Election (~5619 tok)
@ -253,7 +258,7 @@
- `design.md` — Design Document: ha-heartbeat-liveness (~5409 tok)
- `manual-tests.md` — Manual Integration Tests: ha-heartbeat-liveness — Task 4.5 (~1726 tok)
- `requirements.md` — Requirements Document (~2054 tok)
- `research.md` — Gap Analysis: ha-heartbeat-liveness (~2290 tok)
- `research.md` — Gap Analysis: ha-heartbeat-liveness (~5437 tok)
- `spec.json` (~129 tok)
- `tasks.md` — Implementation Plan (~2227 tok)
@ -261,7 +266,7 @@
- `design.md` — Design Document: ha-peer-connection-reuse (~5369 tok)
- `requirements.md` — Requirements Document (~1605 tok)
- `research.md` — Research & Design Decisions: ha-peer-connection-reuse (~3801 tok)
- `research.md` — Research & Design Decisions: ha-peer-connection-reuse (~6598 tok)
- `spec.json` (~130 tok)
- `tasks.md` — Implementation Plan: ha-peer-connection-reuse (~2022 tok)
@ -270,15 +275,15 @@
- `design.md` — Design: HA Replication Hardening (~5514 tok)
- `raft-tla-audit.md` — Raft TLA+ Specification Audit (~1591 tok)
- `requirements.md` — Requirements Document (~2817 tok)
- `research.md` — Gap Analysis: HA Replication Hardening (~2756 tok)
- `spec.json` (~128 tok)
- `tasks.md` — Implementation Plan: HA Replication Hardening (~3816 tok)
- `research.md` — Gap Analysis: HA Replication Hardening (~6234 tok)
- `spec.json` (~165 tok)
- `tasks.md` — Implementation Plan: HA Replication Hardening (~4464 tok)
## .kiro/specs/raft-library/
- `design.md` — Design Document: raft-library (~7756 tok)
- `requirements.md` — Requirements Document (~2115 tok)
- `research.md` — Research & Design Decisions: raft-library (~949 tok)
- `research.md` — Research & Design Decisions: raft-library (~3768 tok)
- `spec.json` (~127 tok)
- `tasks.md` — Implementation Plan (~4312 tok)
@ -286,15 +291,15 @@
- `design.md` — Design: resval-id-decoupling (~3914 tok)
- `requirements.md` — Requirements: resval-id-decoupling (~2360 tok)
- `research.md` — Gap Analysis: resval-id-decoupling (~3505 tok)
- `spec.json` — Declares is (~229 tok)
- `research.md` — Gap Analysis: resval-id-decoupling (~5568 tok)
- `spec.json` — Declares is (~226 tok)
- `tasks.md` — Implementation Tasks: resval-id-decoupling (~2424 tok)
## .kiro/specs/scheduler-controller-msg-gap/
- `design.md` — Design: scheduler-controller-msg-gap (~4756 tok)
- `requirements.md` — Requirements: scheduler-controller-msg-gap (~2712 tok)
- `research.md` — Research Log: scheduler-controller-msg-gap (~1854 tok)
- `research.md` — Research Log: scheduler-controller-msg-gap (~5630 tok)
- `spec.json` (~184 tok)
- `tasks.md` — Tasks: scheduler-controller-msg-gap (~2324 tok)
@ -302,8 +307,8 @@
- `design.md` — Design Document: worker-join-refactor (~4709 tok)
- `requirements.md` — Requirements Document (~1401 tok)
- `research.md` — Research Log: worker-join-refactor (~3357 tok)
- `spec.json` (~129 tok)
- `research.md` — Research Log: worker-join-refactor (~5421 tok)
- `spec.json` (~128 tok)
- `tasks.md` — Implementation Plan (~1658 tok)
## .kiro/steering/
@ -326,6 +331,15 @@
- `2026-05-13-config-replication.md` — Config Replication Implementation Plan (~21611 tok)
- `2026-05-20-config-repl-wire-protocol.md` — Config Replication — Wire Protocol Completion + Bug Fixes (~5134 tok)
## ha/data/
- `xce.crt` — ---BEGIN CERTIFICATE----- (~381 tok)
- `xce.key` — ---BEGIN PRIVATE KEY----- (~455 tok)
## raft/
- `raft.tla` — ------------------------------- MODULE raft --------------------------------- (~5350 tok)
## src/
- `flake.nix` (~571 tok)
@ -447,50 +461,47 @@
- `requestMachines.sh` (~20 tok)
- `requestReturnMachines.sh` (~38 tok)
## src/build/lib/
- `controller.capnp.c` — include "controller.capnp.h" (~134780 tok)
## src/controller/
- `.ha.c.~undo-tree~` — Declares xce_cluster_config_t (~40575 tok)
- `apiserver.c` — include <errno.h> (~5303 tok)
- `CLAUDE.md` — XCE Controller (~2625 tok)
- `cluster.c` — include "config.h" (~12219 tok)
- `config_repl.c` — define _GNU_SOURCE (~14728 tok)
- `config_repl.h` — pragma once (~2481 tok)
- `CLAUDE.md` — XCE Controller (~2630 tok)
- `cluster.c` — include "config.h" (~13092 tok)
- `config_repl.c` — define _GNU_SOURCE (~15780 tok)
- `config_repl.h` — pragma once (~2658 tok)
- `ctrl_metrics.c` — include "ctrl_metrics.h" (~767 tok)
- `dataex.c` — include <errno.h> (~2466 tok)
- `ha.c` — define _GNU_SOURCE (~39651 tok)
- `ha.h` — ifndef HA_H (~9764 tok)
- `ha.c` — define _GNU_SOURCE (~42722 tok)
- `ha.h` — ifndef HA_H (~10718 tok)
- `job.c` — include <errno.h> (~8259 tok)
- `join.c` — include "config.h" (~2347 tok)
- `main.c` — include <argp.h> (~1996 tok)
- `join.c` — include "config.h" (~2514 tok)
- `main.c` — include <argp.h> (~2262 tok)
- `meson.build` (~1080 tok)
- `misc.c` — include <errno.h> (~1359 tok)
- `myapp.c` — define _GNU_SOURCE (~3209 tok)
- `myconf.c` — define _GNU_SOURCE (~1100 tok)
- `myapp.c` — define _GNU_SOURCE (~3831 tok)
- `myconf.c` — define _GNU_SOURCE (~1179 tok)
- `opget.c` — include "config.h" (~1725 tok)
- `opupd.c` — include "config.h" (~714 tok)
- `persist.c` — define _GNU_SOURCE (~13088 tok)
- `persist.c` — define _GNU_SOURCE (~14023 tok)
- `queue.c` — include <errno.h> (~1260 tok)
- `repl.c` — include <assert.h> (~3964 tok)
- `repl.c` — include <assert.h> (~4248 tok)
- `reply.c` — include "config.h" (~590 tok)
- `resources.c` — include <errno.h> (~50 tok)
- `sched.c` — include <errno.h> (~5230 tok)
- `startup_validation.c` — define _GNU_SOURCE (~1423 tok)
- `startup_validation.c` — define _GNU_SOURCE (~1524 tok)
- `task.c` — include <errno.h> (~1376 tok)
- `worker.c` — include <errno.h> (~5201 tok)
## src/controller/include/
- `cluster.h` — if !defined(_CLUSTER_H_) (~1669 tok)
- `cluster.h` — if !defined(_CLUSTER_H_) (~1788 tok)
- `worker.h` — if !defined(_WORKER_H_) (~190 tok)
## src/controller/testsuite/
- `controller_test.c` — include <argp.h> (~1305 tok)
- `controller_test.c` — include <argp.h> (~1398 tok)
- `test_cluster_persist.c` — define _GNU_SOURCE (~1062 tok)
- `test_config_repl.c` — define _GNU_SOURCE (~22128 tok)
- `test_config_repl.c` — define _GNU_SOURCE (~23709 tok)
- `test_ctrl_metrics.c` — define _GNU_SOURCE (~940 tok)
- `test_e2e_capnp.c` — include <stdint.h> (~1690 tok)
- `test_e2e_checkpoint.c` — define _GNU_SOURCE (~1793 tok)
@ -498,44 +509,41 @@
- `test_e2e_hooks.c` — define _GNU_SOURCE (~2346 tok)
- `test_e2e_wal.c` — define _GNU_SOURCE (~2546 tok)
- `test_gc_retention.c` — define _GNU_SOURCE (~2447 tok)
- `test_ha_bootstrap_support.c` — ifdef XCE_TESTING (~1008 tok)
- `test_ha_bootstrap.c` — does: defer_config_fetch (~5828 tok)
- `test_ha_bootstrap_support.c` — ifdef XCE_TESTING (~1080 tok)
- `test_ha_bootstrap.c` — does: defer_config_fetch (~6244 tok)
- `test_ha_cluster.c` — include <stdatomic.h> (~1351 tok)
- `test_ha_connect.c` — define _GNU_SOURCE (~1882 tok)
- `test_ha_connection_reuse.c` — define _GNU_SOURCE (~6304 tok)
- `test_ha_ctrlhello.c` (~1322 tok)
- `test_ha_durability.c` — Declares char (~4075 tok)
- `test_ha_election.c` — Declares char (~11144 tok)
- `test_ha_evaluate_committing.c` — Declares checks (~7656 tok)
- `test_ha_failover.c` — define _GNU_SOURCE (~3781 tok)
- `test_ha_liveness_sigstop.sh` — test_ha_liveness_sigstop.sh — Task 4.5 integration test: missed-heartbeat (~2606 tok)
- `test_ha_liveness.c` — Declares directly (~6006 tok)
- `test_ha_rdhup_integration.c` — Declares char (~4347 tok)
- `test_ha_connection_reuse.c` — define _GNU_SOURCE (~6754 tok)
- `test_ha_ctrlhello.c` (~1416 tok)
- `test_ha_durability.c` — Declares char (~4366 tok)
- `test_ha_election.c` — Declares char (~11940 tok)
- `test_ha_evaluate_committing.c` — Declares checks (~8203 tok)
- `test_ha_failover.c` — define _GNU_SOURCE (~4051 tok)
- `test_ha_liveness.c` — Declares directly (~6435 tok)
- `test_ha_rdhup_integration.c` — Declares char (~4658 tok)
- `test_ha_redirect.c` — include <stdatomic.h> (~905 tok)
- `test_ha_repl_ack.c` (~3093 tok)
- `test_ha_repl_push.c` — Declares is (~2818 tok)
- `test_ha_role.c` — include <signal.h> (~9651 tok)
- `test_ha_start_election_async.c` — Declares returns (~1957 tok)
- `test_ha_startup.c` — Declares char (~8594 tok)
- `test_ha_vote_gate.c` — define _GNU_SOURCE (~2679 tok)
- `test_ha_vote_send.c` — define _GNU_SOURCE (~5635 tok)
- `test_ha_vote_send.c` — define _GNU_SOURCE (~3212 tok)
- `test_ha_repl_ack.c` (~3314 tok)
- `test_ha_repl_push.c` — Declares is (~3020 tok)
- `test_ha_role.c` — include <signal.h> (~10340 tok)
- `test_ha_start_election_async.c` — Declares returns (~2097 tok)
- `test_ha_startup.c` — Declares char (~9207 tok)
- `test_ha_vote_gate.c` — define _GNU_SOURCE (~2934 tok)
- `test_ha_vote_send.c` — define _GNU_SOURCE (~6038 tok)
- `test_hooks.c` — include <stdio.h> (~365 tok)
- `test_p1_integration.c` — include <stdatomic.h> (~1496 tok)
- `test_p1_integration.c` — include <stdatomic.h> (~1270 tok)
- `test_p2_integration.c` — include <errno.h> (~2096 tok)
- `test_p3_hardening.c` — define _GNU_SOURCE (~5279 tok)
- `test_p3_hardening.c` — define _GNU_SOURCE (~5003 tok)
- `test_p3_manual_failover.c` — include <stdatomic.h> (~984 tok)
- `test_persist_checkpoint.c` — define _GNU_SOURCE (~1815 tok)
- `test_persist_hooks.c` — define _GNU_SOURCE (~4222 tok)
- `test_persist_recovery.c` — define _GNU_SOURCE (~2290 tok)
- `test_persist_repl.c` — include <errno.h> (~1396 tok)
- `test_persist_schema.c` — include <stdint.h> (~1190 tok)
- `test_persist_truncate.c` — define _GNU_SOURCE (~1973 tok)
- `test_persist_truncate.c` — define _GNU_SOURCE (~2114 tok)
- `test_persist_types.c` — include <stddef.h> (~697 tok)
- `test_persist_wal.c` — define _GNU_SOURCE (~1708 tok)
- `test_repl_apply.c` — include <stdio.h> (~1862 tok)
- `test_repl_catchup.c` — include <errno.h> (~1183 tok)
- `test_repl_majority.c` — include <errno.h> (~2108 tok)
- `test_repl_majority.c` — include <errno.h> (~2258 tok)
- `test_repl_unit.c` — include <assert.h> (~1662 tok)
- `test_worker_join.c` — include <rtti.h> (~4851 tok)
- `test.c` — include <argp.h> (~18744 tok)
@ -580,7 +588,7 @@
- `control_host_test.go` — go:build !cgo (~211 tok)
- `convert_pure.go` — rawStartInfo (93 fields) (~3096 tok)
- `convert_test.go` — TestRawJobToModel_BasicFields, TestRawJobToModel_NilSubmit, TestRawJobToModel_Timestamps, TestRawWorkerToModel_BasicFields + 14 more (~6770 tok)
- `convert.go` — go:build cgo (~3126 tok)
- `convert.go` — go:build cgo (~3349 tok)
- `handler_test.go` — TestOnHostConfigUpdated_UnregisteredHostIgnored, TestOnHostConfigUpdated_DefaultPseudoHostIgnored, TestOnHostConfigUpdated_PartialMerge, TestOnClus... (~2994 tok)
- `handler.go` — DefaultHandler (8 fields); methods: OnJobCreated, OnJobUpdated, OnJobDeleted, OnJobFinished (~1054 tok)
- `heartbeat_test.go` — TestSetHeartbeatInterval_ValidValue, TestSetHeartbeatInterval_ZeroIgnored, TestSetHeartbeatInterval_NegativeIgnored, TestRunHeartbeat_CallsFunction... (~593 tok)
@ -856,138 +864,3 @@
- `config.go` — APIServerInfo (64 fields); methods: Len, Swap, Less, GetConfig (~1607 tok)
- `logger.go` — LoggerConfig (48 fields) (~1877 tok)
- `logger.yaml` (~205 tok)
- `schema.go` — ValidateYamlFormat, ValidateYAML (~753 tok)
## src/frontend/xcecli/config/schema/
- `logger_schema.json` (~930 tok)
## src/frontend/xcecli/filters/
- `host_filter.go` — HostsFilter (7 fields); methods: HandleHostFilter (~166 tok)
- `job_filter.go` — JobsFilter (61 fields); methods: HandleJobFilter (~1511 tok)
- `queue_filter.go` — QueuesFilter (7 fields); methods: HandleQueueFilter (~119 tok)
- `resource_filter.go` — ResourceDistFilter (13 fields); methods: HandleResourceDistFilter, HandleResourceDefFilter (~271 tok)
- `role_filter.go` — RolesFilter (9 fields); methods: HandleRoleFilter (~149 tok)
- `user_filter.go` — UserRolesFilter (8 fields); methods: HandleUserRoleFilter (~149 tok)
## src/frontend/xcecli/models/
- `cluster.go` — ClusterResponse (15 fields) (~201 tok)
- `exitstatus_test.go` — TestQueryJob_ExitStatus_Null, TestQueryJob_ExitStatus_String, TestQueryJob_ExitStatus_Zero (~292 tok)
- `hosts.go` — HostsResponse (56 fields); methods: UnmarshalJSON (~1288 tok)
- `jobs.go` — JobsResponse (149 fields); methods: UnmarshalJSON, String, UnmarshalJSON (~5415 tok)
- `logon.go` — LogonData define the data structure in the logon response (~168 tok)
- `ordermap.go` — UnmarshalOrderMapRow extracts the Data sub-object from the server's OrderMap wire format (~166 tok)
- `permissions.go` — PermsResponse (6 fields) (~98 tok)
- `queues.go` — QueuesResponse (33 fields); methods: UnmarshalJSON (~1072 tok)
- `resource.go` — ResourceValue (66 fields); methods: UnmarshalJSON, String, UnmarshalJSON, String (~1674 tok)
- `response_test.go` — TestHostsResponse_FullUnmarshal, TestQueuesResponse_FullUnmarshal, TestJobsResponse_FullUnmarshal, TestRolesResponse_FullUnmarshal + 3 more (~1644 tok)
- `roles.go` — CreateRoleRequest (25 fields); methods: UnmarshalJSON (~361 tok)
- `users.go` — AddRoleForUserRequest (26 fields); methods: UnmarshalJSON (~343 tok)
## src/frontend/xcecli/router/
- `router.go` — Execute (~919 tok)
## src/frontend/xcecli/test/
- `t.sh` (~118 tok)
## src/include/
- `attrs.h` — if !defined(_ATTRS_H_) (~26 tok)
- `bitset.h` — if !defined(_BITSET_H_) (~71 tok)
- `byteorder.h` — pragma once (~247 tok)
- `client.h` — if !defined(_CLIENT_H_) (~1023 tok)
- `clusterconfig.h` — if !defined(_CLUSTERCONFIG_H_) (~474 tok)
- `config.h` — if !defined(_CONFIG_H_) (~993 tok)
- `configobj.h` — if !defined(_CONFIGOBJ_H_) (~46 tok)
- `ctrl_metrics.h` — pragma once (~539 tok)
- `dataq.h` — if !defined(_DATAQ_H_) (~190 tok)
- `delay.h` — public delay API; adds delay_task_ms() and nextrun_ts field to delay_task_t (~320 tok)
- `group.h` — if !defined(_GROUP_H_) (~227 tok)
- `helper.h` — if !defined(_HELPER_H_) (~207 tok)
- `hook.h` — if !defined(__HOOK_H__) (~195 tok)
- `htab.h` — if !defined(_HTAB_H_) (~268 tok)
- `idgen.h` — if !defined(_IDGEN_H_) (~117 tok)
- `jobj.h` — if !defined(_JOBJ_H_) (~103 tok)
- `kv.h` — if !defined(_KV_H_) (~407 tok)
- `kvmgr.h` — if !defined(__KVMGR_H__) (~306 tok)
- `misc.h` — if !defined(_MISC_H_) (~88 tok)
- `myconf.h` — if !defined(_MYCONF_H_) (~458 tok)
- `mylock.h` — if !defined(_MYLOCK_H_) (~281 tok)
- `netio.h` — if !defined(_NETIO_H_) (~1293 tok)
- `objs.h` — if !defined(_OBJS_H_) (~36 tok)
- `ordq.h` — if !defined(_ORDQ_H_) (~266 tok)
- `persist.h` — pragma once (~1914 tok)
- `preason.h` — if !defined(_PREASON_H_) (~108 tok)
- `qobj.h` — if !defined(_QOBJ_H_) (~41 tok)
- `repl.h` — pragma once (~979 tok)
- `request.h` — if !defined(_REQUEST_H_) (~2108 tok)
- `resobj.h` — if !defined(_RESOBH_H_) (~46 tok)
- `resreq.h` — if !defined(_RESREQ_H_) (~3223 tok)
- `typemap.h` — if !defined(_TYPEMAP_H_) (~3593 tok)
- `utils.h` — if !defined(_UTILS_H_) (~1574 tok)
- `vtaskmgr.h` — if !defined(_VTASKMGR_H_) (~442 tok)
- `wal_types.h` — pragma once (~679 tok)
- `wobj.h` — if !defined(_WOBBJ_H_) (~42 tok)
- `xcealloc.h` — if !defined(_XCEALLOC_H_) (~146 tok)
- `xceapi.h` — if !defined(_XCEAPI_H_) (~436 tok)
- `xcectx.h` — if !defined(_XCECTX_H_) (~388 tok)
- `xcelimit.h` — if !defined(_XCELIMIT_H_) (~281 tok)
- `xcelog.h` — ifndef _XCELOG_H_ (~819 tok)
- `xlist.h` — if !defined(__XLIST_H__) (~193 tok)
- `xstring.h` — if !defined(__XSTRING_H__) (~54 tok)
- `xvec.h` — if !defined(_XVEC_H_) (~293 tok)
- `zarray.h` — if !defined(_ZARRAY_H_) (~384 tok)
## src/include/xce/
- `cluster_def.h` — ifndef XCE_CLUSTER_DEF_H (~1178 tok)
- `cluster.h` — A single controller candidate. (~1211 tok)
- `errnox.h` — if !defined(__XCE_ERRNOX_H__) (~262 tok)
- `types.h` — < NULL. (~5454 tok)
## src/lib/
- `delay.c` — include <errno.h> (~1798 tok)
- `helper.c` — include <errno.h> (~2956 tok)
- `meson.build` (~1513 tok)
- `netio.c` — include <arpa/inet.h> (~5716 tok)
- `request.c` — include <errno.h> (~12892 tok)
## src/lib/raft/
- `raft_sim.c` — ifdef XCE_TESTING (~1627 tok)
- `raft.c` — include "raft.h" (~2452 tok)
- `raft.h` — ifndef RAFT_H (~3049 tok)
## src/lib/testsuite/
- `test_delay.c` — delay unit tests; adds test_delay_task_ms_fires_within_tolerance and test_delay_task_ms_invalid_args (~1600 tok)
- `test_libxce_retry.c` — define _GNU_SOURCE (~2741 tok)
- `test_netio_on_connected.c` — define _GNU_SOURCE (~1431 tok)
- `test_raft.c` — include <errno.h> (~25610 tok)
- `test.c` — include <argp.h> (~971 tok)
## src/scheduler/testsuite/
- `test.c` — define UNIT_TEST 1 (~4290 tok)
## src/schema/
- `controller.capnp` — Declares OPCode (~8223 tok)
## src/tools/
- `deploy-ha.sh` (~505 tok)
- `test-heartbeat-tolerance.sh` — test-heartbeat-tolerance.sh — Task 4.5: missed-heartbeat tolerance via SIGSTOP/SIGCONT (~2377 tok)
## src/xce.build/
- `build.ninja` — This is the build file for project "xce" (~75089 tok)
## src/xce.build/lib/
- `controller.capnp.c` — include "controller.capnp.h" (~134780 tok)

View file

@ -5292,6 +5292,246 @@
"related_bugs": [],
"occurrences": 2,
"last_seen": "2026-06-09T08:35:24.717Z"
},
{
"id": "bug-331",
"timestamp": "2026-06-09T09:28:15.795Z",
"error_message": "Significant refactor of ",
"file": "src/controller/myapp.c",
"root_cause": "5 lines replaced/restructured",
"fix": "Rewrote 27→36 lines (5 removed)",
"tags": [
"auto-detected",
"refactor",
"c"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-09T09:28:15.795Z"
},
{
"id": "bug-332",
"timestamp": "2026-06-09T09:41:02.753Z",
"error_message": "Significant refactor of ",
"file": "src/lib/raft/raft.c",
"root_cause": "4 lines replaced/restructured",
"fix": "Rewrote 9→15 lines (4 removed)",
"tags": [
"auto-detected",
"refactor",
"c"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-09T09:41:02.753Z"
},
{
"id": "bug-333",
"timestamp": "2026-06-09T10:23:01.661Z",
"error_message": "Significant refactor of ",
"file": "src/controller/ha.c",
"root_cause": "37 lines replaced/restructured",
"fix": "Rewrote 41→4 lines (37 removed)",
"tags": [
"auto-detected",
"refactor",
"c"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-09T10:23:01.661Z"
},
{
"id": "bug-334",
"timestamp": "2026-06-09T12:56:11.095Z",
"error_message": "Wrong condition in logic",
"file": "src/controller/ha.c",
"root_cause": "Condition was: if (my_role == CTRL_STANDBY && ha_heartbeat_is_fresh(h)",
"fix": "Changed to: if (my_role == CTRL_STANDBY && ha->primary_ctrl_fd >= )",
"tags": [
"auto-detected",
"logic-fix",
"c"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-09T12:56:11.095Z"
},
{
"id": "bug-335",
"timestamp": "2026-06-09T00:00:00Z",
"error_message": "UPDATE_RESVALS dequeue writes worker resval at index 0 unconditionally instead of resolving resource idx by name",
"file": "src/scheduler/src/scheduler.c",
"root_cause": "zarray_put(&w->resvals, 0, rv->val) hardcodes index 0; worker resvals are indexed by resource idx (see schedobj.c:291)",
"fix": "NOT YET FIXED — found during scheduler-controller-msg-gap validation gap analysis (research.md G-1). Fix: key_find rv->name in sched->resmgr, put at r->idx",
"tags": [
"scheduler",
"resvals",
"latent-bug",
"gap-analysis",
"c"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-09T00:00:00Z"
},
{
"id": "bug-336",
"timestamp": "2026-06-09T00:00:00Z",
"error_message": "XCE_SCHED_OP_UPDATE_RESOURCES cannot update existing resources; makeResource returns NULL/EEXIST and return value is silently ignored",
"file": "src/scheduler/src/scheduler.c",
"root_cause": "makeResource_ in src/lib/resobj.c rejects duplicates with EEXIST; dequeue case ignores NULL return, so 'add or update' (EARS-UB-1.1) is only 'add'",
"fix": "NOT YET FIXED — found during scheduler-controller-msg-gap validation gap analysis (research.md G-2). Fix: on EEXIST mutate existing resource fields and log failures",
"tags": [
"scheduler",
"resources",
"latent-bug",
"gap-analysis",
"silent-error",
"c"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-09T00:00:00Z"
},
{
"id": "bug-337",
"timestamp": "2026-06-09T00:02:00Z",
"error_message": "Vote-deny gate inert: ha_heartbeat_is_fresh reads ha->last_heartbeat which has no production writer after the liveness refactor",
"file": "src/controller/ha.c",
"root_cause": "ha-heartbeat-liveness refactor moved activity tracking to ha->last_activity but ha_heartbeat_is_fresh (ha.c:1762) still reads the orphaned last_heartbeat timespec; only tests write it, so freshness is always stale after ~heartbeat_timeout_ms of uptime and standbys grant votes despite a live primary (gates at ha.c:214 and ha.c:1841)",
"fix": "FIXED 2026-06-09: ha_heartbeat_is_fresh now reads ha->last_activity (returns 0 when last_activity==0); dead last_heartbeat field removed from ha_state_t; test_ha_vote_gate.c helper rewritten as set_last_activity_age and fresh/stale tests now also set primary_ctrl_fd. All tests green.",
"tags": [
"ha",
"heartbeat",
"liveness",
"vote-gate",
"dead-field",
"gap-analysis"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-09T17:30:00Z"
},
{
"id": "bug-338",
"timestamp": "2026-06-09T00:00:00Z",
"error_message": "controller_unit_test: test_vote_granted_to_same_candidate_same_term FAIL (Expected 1 Was 0); test_standby_fresh_heartbeat_denies_vote_request FAIL (Expected 0 Was 1, Req 6.2)",
"file": "src/controller/ha.c",
"root_cause": "Uncommitted handle_request_vote changes: (1) duplicate-vote deny made candidate-agnostic (denies same-candidate idempotent re-grant, diverging from raft.c which keeps re-grant); (2) STANDBY vote deny now requires primary_ctrl_fd >= 0, tests use ha_init default fd=-1",
"fix": "FIXED 2026-06-09: handle_request_vote Req 2.6 check restored to same-candidate idempotent re-grant (deny only when voted_for_name differs from candidate_id), matching raft.c; test_standby_fresh_heartbeat_denies_vote_request updated to set primary_ctrl_fd=999. Also fixed myapp.c leader_known check to treat a vote for myconf->name as a self-vote (production records real name, not the 'self' test placeholder). All tests green.",
"tags": [
"ha",
"election",
"vote-gate",
"raft",
"unit-test",
"uncommitted-drift",
"gap-analysis"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-09T17:30:00Z"
},
{
"id": "bug-339",
"timestamp": "2026-06-10T00:23:18.666Z",
"error_message": "Missing guard clause",
"file": "src/controller/ha.c",
"root_cause": "No early return/throw for edge case: last == 0",
"fix": "Added guard clause: if (last == 0)",
"tags": [
"auto-detected",
"guard-clause",
"c"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-10T00:23:18.666Z"
},
{
"id": "bug-340",
"timestamp": "2026-06-10T00:23:21.918Z",
"error_message": "Wrong condition in logic",
"file": "src/controller/ha.c",
"root_cause": "Condition was: if (ha->voted_for_term == term && ha->voted_for_name[0)",
"fix": "Changed to: if (ha->voted_for_term == term && ha->voted_for_name[0)",
"tags": [
"auto-detected",
"logic-fix",
"c"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-10T00:23:21.918Z"
},
{
"id": "bug-341",
"timestamp": "2026-06-10T00:23:52.563Z",
"error_message": "Wrong reference: last_heartbeat should be voted_for_term",
"file": "src/controller/ha.h",
"root_cause": "Used \"last_heartbeat\" instead of \"voted_for_term\"",
"fix": "Changed last_heartbeat → voted_for_term",
"tags": [
"auto-detected",
"wrong-reference",
"h"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-10T00:23:52.563Z"
},
{
"id": "bug-342",
"timestamp": "2026-06-10T00:24:41.873Z",
"error_message": "Significant refactor of ",
"file": "src/controller/myapp.c",
"root_cause": "4 lines replaced/restructured",
"fix": "Rewrote 13→17 lines (4 removed)",
"tags": [
"auto-detected",
"refactor",
"c"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-10T00:24:41.873Z"
},
{
"id": "bug-339",
"timestamp": "2026-06-09T17:30:00Z",
"error_message": "raft_init discards persisted vote term: voted_for_term reset to 0 even when initial_voted_for is restored, so duplicate-vote check cannot match after restart (possible double vote in same term)",
"file": "src/lib/raft/raft.c",
"root_cause": "raft_init set s->voted_for_term=0 unconditionally; persistence writes (term, voted_for) as a pair, so a restored vote belongs to initial_term",
"fix": "FIXED 2026-06-09: voted_for_term = (initial_voted_for != -1) ? initial_term : 0. Added test_proto_H6_restored_vote_blocks_second_vote and strengthened H1 assertion in src/lib/testsuite/test_raft.c. Found as candidate bug C2 in raft-library gap analysis.",
"tags": [
"raft",
"election-safety",
"restart",
"persistence",
"gap-analysis"
],
"related_bugs": [
"bug-338"
],
"occurrences": 1,
"last_seen": "2026-06-09T17:30:00Z"
},
{
"id": "bug-344",
"timestamp": "2026-06-10T00:32:34.054Z",
"error_message": "Incorrect value in code",
"file": "../../../../../claude/.claude/projects/-home-rshen-projects-xce-xce/memory/project_ha_bootstrap_progress.md",
"root_cause": "Had `ha->last_heartbeat`",
"fix": "Changed to `ha->last_activity`",
"tags": [
"auto-detected",
"wrong-value",
"md"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-06-10T00:32:34.054Z"
}
]
}

View file

@ -29,15 +29,24 @@
- vote_ep_ctx_t lifetime (Task 2.4 review fix): do NOT free vctx in vote_ep_err_handler. ep->data (vctx) is also used by vote_reply_msg_handler. Cleanup deferred to ha_election_cleanup() (task 5.1). Freeing in err_handler causes UAF.
- timeout_ctx field in ha_elect_ctx_t (Task 2.4 review fix): added void *timeout_ctx field pointing to election_timeout_ctx_t. Set after delay_task_ms succeeds. Allows task 4.1 (quorum-reached path) to cancel the watchdog by setting timeout_ctx->cancelled=1 atomically.
- Heartbeat freshness source of truth (2026-06-09): `ha_heartbeat_is_fresh` derives from `ha->last_activity` (whole CLOCK_MONOTONIC seconds, written by ha_on_heartbeat_received, lock-free time_t read). The old `last_heartbeat` timespec field was REMOVED from ha_state_t — do not reintroduce it. The standby vote-deny gate is `role==CTRL_STANDBY && primary_ctrl_fd >= 0 && ha_heartbeat_is_fresh(ha)`; RDHUP resets primary_ctrl_fd to -1 so a standby whose primary died can grant votes immediately even if last_activity is still fresh.
- raft_init restore contract (2026-06-09): persistence writes (term, voted_for) as a pair, so `raft_init` must set `voted_for_term = initial_term` when `initial_voted_for != -1`. Leaving it 0 lets a restarted node double-vote in the same term (Election Safety violation). Covered by test_proto_H6.
## Do-Not-Repeat
<!-- Mistakes made and corrected. Each entry prevents the same mistake recurring. -->
<!-- Format: [YYYY-MM-DD] Description of what went wrong and what to do instead. -->
- [2026-06-09] Do NOT make the duplicate-vote deny candidate-agnostic ("term-only deny"). Raft §5.2 grants iff votedFor ∈ {Nil, candidateId} — same-candidate re-grant must stay idempotent (network retransmission path). An uncommitted ha.c change did this and broke test_vote_granted_to_same_candidate_same_term; ha.c must match raft.c's Step 4 semantics (deny only a DIFFERENT candidate in the same term). See bug-338.
- [2026-06-09] The "self" voted_for_name placeholder only exists in test mode; production raft_cb_persist records the node's REAL name for a self-vote. Any check for "did I vote for myself" must compare against BOTH "self" and c->myconf->name (see myapp.c leader_known).
- [2026-06-04] capnpc-c 1.0.1 has a bug with `:group` inside a `union` when `$C.codecgen` is active: the encode/decode generator emits `d->FIELDNAME` (flat) instead of `d->data.GROUPNAME.FIELDNAME`, AND places the case as dead code after the prior `break`. Do NOT use inline `:group` in a union that has `$C.codecgen`. Instead, define a named outer struct (`struct Foo $C.mapname("foo_t")`) and reference it as a pointer union member. The named struct generates correct `encode_Foo_ptr`/`decode_Foo_ptr` calls. The `read_RequestReply`/`write_RequestReply` wire layer works correctly with groups but the codec layer does not.
## Decision Log
<!-- Significant technical decisions with rationale. Why X was chosen over Y. -->
- Election liveness fix (2026-06-09): `skip_election` path in myapp.c had two bugs: (1) `voted_for_name != ""` triggered even for self-votes and stale terms — fixed by adding `strcmp(name,"self")!=0 && voted_for_term==ha->term`; (2) `ha_arm_heartbeat_timeout_vtask` was never called in skip_election path — fixed by adding arm call after role=CTRL_STANDBY. raft.c Step 4 simplified to term-only deny (same-candidate idempotent re-grant preserved). ha.c `handle_request_vote_req` also simplified to term-only deny.
- CORRECTION (2026-06-09 validation pass): the ha.c "term-only deny" part was WRONG — it denied same-candidate re-grants, diverging from raft.c and failing tests. Reverted to deny-different-candidate-only (see Do-Not-Repeat). The myapp.c "self" check was also incomplete — production self-votes are persisted under the node's real name, so the check now also compares myconf->name.
- Task 4.1 (ha_elect_evaluate_and_signal COMMITTING path): Added cluster_t *c field to ha_elect_ctx_t so evaluate can call cluster_init_phase2 without signature change. election_timeout_ctx_t and committing_timeout_ctx_t must be defined BEFORE ha_elect_evaluate_and_signal. cluster_init_phase2 wrapped in #ifndef XCE_TESTING to avoid linker error (cluster.c not in unit-test binary). ha_phase2_hook seam used in XCE_TESTING builds. On PRIMARY: cancel election timeout watchdog, call phase2 if first_boot, set CTRL_PRIMARY_COMMITTING, reset repl_ack_count=0, arm committing_timeout_fn. On STANDBY: call phase2 if first_boot, set CTRL_STANDBY.

View file

@ -1,5 +1,5 @@
{
"last_heartbeat": "2026-06-08T10:03:14.767Z",
"last_heartbeat": "2026-06-10T01:06:27.659Z",
"engine_status": "running",
"execution_log": [
{
@ -167,6 +167,39 @@
"status": "success",
"timestamp": "2026-06-08T09:00:00.899Z",
"duration_ms": 4
},
{
"task_id": "project-suggestions",
"status": "failed",
"timestamp": "2026-06-10T00:19:56.961Z",
"duration_ms": 9810,
"error": "claude -p failed: Exit code 1: SessionEnd hook [export PATH=\"$($SHELL -lc 'echo $PATH' 2>/dev/null):$PATH\"; _R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=$(ls -dt $HOME/.claude/plugins/cache/thedotmack/claude-mem/[0-9]*/ 2>/dev/null | head -1); _R=\"${_R%/}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" hook claude-code session-complete] failed: Hook cancelled"
},
{
"task_id": "token-audit",
"status": "success",
"timestamp": "2026-06-10T00:20:16.187Z",
"duration_ms": 53
},
{
"task_id": "project-suggestions",
"status": "failed",
"timestamp": "2026-06-10T00:20:16.188Z",
"duration_ms": 19227,
"error": "claude -p failed: Exit code 1: SessionEnd hook [export PATH=\"$($SHELL -lc 'echo $PATH' 2>/dev/null):$PATH\"; _R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=$(ls -dt $HOME/.claude/plugins/cache/thedotmack/claude-mem/[0-9]*/ 2>/dev/null | head -1); _R=\"${_R%/}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" hook claude-code session-complete] failed: Hook cancelled"
},
{
"task_id": "cerebrum-reflection",
"status": "failed",
"timestamp": "2026-06-10T00:20:16.188Z",
"duration_ms": 9557,
"error": "claude -p failed: Exit code 1: SessionEnd hook [export PATH=\"$($SHELL -lc 'echo $PATH' 2>/dev/null):$PATH\"; _R=\"${CLAUDE_PLUGIN_ROOT}\"; [ -z \"$_R\" ] && _R=$(ls -dt $HOME/.claude/plugins/cache/thedotmack/claude-mem/[0-9]*/ 2>/dev/null | head -1); _R=\"${_R%/}\"; [ -z \"$_R\" ] && _R=\"$HOME/.claude/plugins/marketplaces/thedotmack/plugin\"; node \"$_R/scripts/bun-runner.js\" \"$_R/scripts/worker-service.cjs\" hook claude-code session-complete] failed: Hook cancelled"
},
{
"task_id": "anatomy-rescan",
"status": "success",
"timestamp": "2026-06-10T01:00:00.580Z",
"duration_ms": 50
}
],
"dead_letter_queue": [],

View file

@ -91,3 +91,23 @@
[2026-06-08T08:03:14.768Z] [INFO] Dashboard server listening on port 18791
[2026-06-08T09:00:00.895Z] [INFO] Executing task: Consolidate old memory
[2026-06-08T09:00:00.900Z] [INFO] Task Consolidate old memory completed in 4ms
[2026-06-10T00:06:27.655Z] [INFO] Scheduled task: Full anatomy rescan (0 */6 * * *)
[2026-06-10T00:06:27.657Z] [INFO] Scheduled task: Consolidate old memory (0 2 * * *)
[2026-06-10T00:06:27.657Z] [INFO] Scheduled task: Token audit report (0 0 * * 1)
[2026-06-10T00:06:27.657Z] [INFO] Scheduled task: Cerebrum reflection (0 3 * * 0)
[2026-06-10T00:06:27.657Z] [INFO] Scheduled task: AI suggestions (0 4 * * 1)
[2026-06-10T00:06:27.659Z] [INFO] File watcher started on .wolf/
[2026-06-10T00:06:27.659Z] [INFO] OpenWolf daemon started
[2026-06-10T00:06:27.660Z] [INFO] Dashboard server listening on port 18791
[2026-06-10T00:07:13.306Z] [INFO] WebSocket client connected
[2026-06-10T00:12:13.316Z] [INFO] WebSocket client connected
[2026-06-10T00:19:47.150Z] [INFO] Executing task: AI suggestions
[2026-06-10T00:19:56.961Z] [ERROR] Task AI suggestions failed (attempt 1): claude -p failed: Exit code 1: SessionEnd hook [export PATH="$($SHELL -lc 'echo $PATH' 2>/dev/null):$PATH"; _R="${CLAUDE_PLUGIN_ROOT}"; [ -z "$_R" ] && _R=$(ls -dt $HOME/.claude/plugins/cache/thedotmack/claude-mem/[0-9]*/ 2>/dev/null | head -1); _R="${_R%/}"; [ -z "$_R" ] && _R="$HOME/.claude/plugins/marketplaces/thedotmack/plugin"; node "$_R/scripts/bun-runner.js" "$_R/scripts/worker-service.cjs" hook claude-code session-complete] failed: Hook cancelled
[2026-06-10T00:19:56.961Z] [INFO] Executing task: AI suggestions
[2026-06-10T00:20:06.631Z] [INFO] Executing task: Cerebrum reflection
[2026-06-10T00:20:16.134Z] [INFO] Executing task: Token audit report
[2026-06-10T00:20:16.188Z] [INFO] Task Token audit report completed in 53ms
[2026-06-10T00:20:16.188Z] [ERROR] Task AI suggestions failed (attempt 1): claude -p failed: Exit code 1: SessionEnd hook [export PATH="$($SHELL -lc 'echo $PATH' 2>/dev/null):$PATH"; _R="${CLAUDE_PLUGIN_ROOT}"; [ -z "$_R" ] && _R=$(ls -dt $HOME/.claude/plugins/cache/thedotmack/claude-mem/[0-9]*/ 2>/dev/null | head -1); _R="${_R%/}"; [ -z "$_R" ] && _R="$HOME/.claude/plugins/marketplaces/thedotmack/plugin"; node "$_R/scripts/bun-runner.js" "$_R/scripts/worker-service.cjs" hook claude-code session-complete] failed: Hook cancelled
[2026-06-10T00:20:16.188Z] [ERROR] Task Cerebrum reflection failed (attempt 1): claude -p failed: Exit code 1: SessionEnd hook [export PATH="$($SHELL -lc 'echo $PATH' 2>/dev/null):$PATH"; _R="${CLAUDE_PLUGIN_ROOT}"; [ -z "$_R" ] && _R=$(ls -dt $HOME/.claude/plugins/cache/thedotmack/claude-mem/[0-9]*/ 2>/dev/null | head -1); _R="${_R%/}"; [ -z "$_R" ] && _R="$HOME/.claude/plugins/marketplaces/thedotmack/plugin"; node "$_R/scripts/bun-runner.js" "$_R/scripts/worker-service.cjs" hook claude-code session-complete] failed: Hook cancelled
[2026-06-10T01:00:00.530Z] [INFO] Executing task: Full anatomy rescan
[2026-06-10T01:00:00.580Z] [INFO] Task Full anatomy rescan completed in 50ms

View file

@ -1,80 +1,110 @@
{
"session_id": "session-2026-06-09-2117",
"started": "2026-06-09T04:17:13.321Z",
"session_id": "session-2026-06-10-1720",
"started": "2026-06-10T00:20:07.286Z",
"files_read": {
"/home/rshen/projects/xce/worktrees/persistent_ha/src/schema/controller.capnp": {
"count": 1,
"tokens": 8223,
"first_read": "2026-06-09T06:41:14.575Z"
"/home/rshen/projects/xce/worktrees/persistent_ha/src/lib/testsuite/test_raft.c": {
"count": 4,
"tokens": 25610,
"first_read": "2026-06-10T00:20:59.235Z"
},
"/home/rshen/projects/xce/worktrees/persistent_ha/src/controller/cluster.c": {
"count": 2,
"tokens": 12219,
"first_read": "2026-06-09T06:43:58.698Z"
},
"/home/rshen/projects/xce/worktrees/persistent_ha/src/include/clusterconfig.h": {
"/home/rshen/projects/xce/worktrees/persistent_ha/src/controller/ha.h": {
"count": 1,
"tokens": 474,
"first_read": "2026-06-09T06:46:31.070Z"
},
"/home/rshen/projects/xce/worktrees/persistent_ha/src/controller/config_repl.c": {
"count": 1,
"tokens": 14728,
"first_read": "2026-06-09T06:47:59.696Z"
},
"/home/rshen/projects/xce/worktrees/persistent_ha/src/controller/myconf.c": {
"count": 1,
"tokens": 1100,
"first_read": "2026-06-09T07:31:53.498Z"
},
"/home/rshen/projects/xce/worktrees/persistent_ha/src/controller/testsuite/test.c": {
"count": 1,
"tokens": 18744,
"first_read": "2026-06-09T07:31:53.527Z"
},
"/home/rshen/projects/xce/worktrees/persistent_ha/src/controller/startup_validation.c": {
"count": 1,
"tokens": 1423,
"first_read": "2026-06-09T08:04:51.033Z"
},
"/home/rshen/projects/xce/worktrees/persistent_ha/src/docs/cluster-config-split-analysis.md": {
"count": 1,
"tokens": 2796,
"first_read": "2026-06-09T08:32:42.474Z"
"tokens": 10007,
"first_read": "2026-06-10T00:23:31.412Z"
}
},
"files_written": [
{
"file": "/home/rshen/projects/xce/worktrees/persistent_ha/src/docs/cluster-config-split-analysis.md",
"file": "/home/rshen/projects/xce/worktrees/persistent_ha/src/lib/testsuite/test_raft.c",
"action": "edit",
"tokens": 308,
"at": "2026-06-09T08:33:08.519Z"
"tokens": 114,
"at": "2026-06-10T00:22:18.260Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/persistent_ha/src/docs/cluster-config-split-analysis.md",
"file": "/home/rshen/projects/xce/worktrees/persistent_ha/src/lib/testsuite/test_raft.c",
"action": "edit",
"tokens": 577,
"at": "2026-06-09T08:33:34.327Z"
"tokens": 523,
"at": "2026-06-10T00:22:44.317Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/persistent_ha/src/docs/cluster-config-split-analysis.md",
"file": "/home/rshen/projects/xce/worktrees/persistent_ha/src/lib/testsuite/test_raft.c",
"action": "edit",
"tokens": 618,
"at": "2026-06-09T08:35:24.714Z"
"tokens": 36,
"at": "2026-06-10T00:22:57.391Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/persistent_ha/src/docs/cluster-config-split-analysis.md",
"file": "/home/rshen/projects/xce/worktrees/persistent_ha/src/lib/raft/raft.c",
"action": "edit",
"tokens": 165,
"at": "2026-06-09T08:36:14.677Z"
"tokens": 144,
"at": "2026-06-10T00:23:00.680Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/persistent_ha/src/controller/ha.c",
"action": "edit",
"tokens": 260,
"at": "2026-06-10T00:23:18.664Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/persistent_ha/src/controller/ha.c",
"action": "edit",
"tokens": 121,
"at": "2026-06-10T00:23:21.916Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/persistent_ha/src/controller/ha.h",
"action": "edit",
"tokens": 85,
"at": "2026-06-10T00:23:49.279Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/persistent_ha/src/controller/ha.h",
"action": "edit",
"tokens": 28,
"at": "2026-06-10T00:23:52.561Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/persistent_ha/src/controller/testsuite/test_ha_vote_gate.c",
"action": "edit",
"tokens": 122,
"at": "2026-06-10T00:24:14.357Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/persistent_ha/src/controller/testsuite/test_ha_vote_gate.c",
"action": "edit",
"tokens": 109,
"at": "2026-06-10T00:24:18.132Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/persistent_ha/src/controller/testsuite/test_ha_vote_gate.c",
"action": "edit",
"tokens": 63,
"at": "2026-06-10T00:24:21.425Z"
},
{
"file": "/home/rshen/projects/xce/worktrees/persistent_ha/src/controller/myapp.c",
"action": "edit",
"tokens": 258,
"at": "2026-06-10T00:24:41.870Z"
},
{
"file": "/home/claude/.claude/projects/-home-rshen-projects-xce-xce/memory/project_ha_bootstrap_progress.md",
"action": "edit",
"tokens": 147,
"at": "2026-06-10T00:32:34.052Z"
}
],
"edit_counts": {
"src/docs/cluster-config-split-analysis.md": 4
"src/lib/testsuite/test_raft.c": 3,
"src/lib/raft/raft.c": 1,
"src/controller/ha.c": 2,
"src/controller/ha.h": 2,
"src/controller/testsuite/test_ha_vote_gate.c": 3,
"src/controller/myapp.c": 1,
"../../../../../claude/.claude/projects/-home-rshen-projects-xce-xce/memory/project_ha_bootstrap_progress.md": 1
},
"anatomy_hits": 8,
"anatomy_hits": 2,
"anatomy_misses": 0,
"repeated_reads_warned": 1,
"repeated_reads_warned": 3,
"cerebrum_warnings": 0,
"stop_count": 3
"stop_count": 1
}

View file

@ -2,7 +2,9 @@
> Chronological action log. Hooks and AI append to this file automatically.
> Old sessions are consolidated by the daemon weekly.
| 09:45 | Fixed election liveness: (1) raft.c step-4 deny is now term-only (same-candidate idempotent re-grant preserved); (2) ha.c vote gate also term-only; (3) myapp.c skip-election guard adds voted_for_name!="self" + voted_for_term==term; (4) watchdog armed after skip_election | raft.c, ha.c, myapp.c | FIXED | ~200 |
| 09:12 | Task 6.6 residual fix: removed double-free in ha_disarm_heartbeat_timeout_vtask (line 3424 free(ctx) removed); deferred free now handled solely by liveness_poll_fn; build+tests GREEN | ha.c | FIXED | ~80 |
| 09:45 | Fix test_proto_B5_idempotent_same_candidate: modified raft_on_vote_request Step 4 to allow idempotent re-grant when same candidate re-requests vote in same term; different candidate still denied; all tests GREEN | lib/raft/raft.c | FIXED | ~60 |
| 21:20 | Updated cluster-config-split-analysis.md: added decisions to drop port fallback and remove candidates from ClusterConfig; detailed per-file impact table; revised implementation phases (Phase 1 = immediate removal, Phases 2-5 = split/migration) | src/docs/cluster-config-split-analysis.md | DONE | ~120 |
| 07:46 | Task 6.6 (ha-repl-hardening): integration test for snapshot transfer on lagging standby; found and fixed 2 ASAN bugs: double-free in ha_connect_to_peer (bug-006) and heap-UAF in liveness_poll_fn on ctx re-arm (bug-007); confirmed snapshot trigger path (oldest_wal_lsn=1 > applied_lsn=0 → requesting snapshot transfer, checkpoint_lsn=0, size=16) fires correctly after standby kill+restart; all 3 nodes stable, no ASAN errors; standby-side apply blocked by known capnp payload deferral limitation | ha.c | GREEN (trigger confirmed, apply deferred by design) | ~800 |
| 06:30 | Task 6.4 (ha-repl-hardening): audited snapshot chunk transfer tests; tests 2+3 already covered (reconnect discards snap.part, lsn<oldest_wal triggers snapshot); added test_snapshot_recv_128kb_two_chunks (131072-byte receive path, 2×65536 chunks, verifies total_size/rename/applied_lsn=checkpoint_lsn); registered in dispatch; 1/1 controller test OK ASAN | test_config_repl.c | GREEN | ~400 |
@ -695,3 +697,88 @@
| 01:35 | Edited src/docs/cluster-config-split-analysis.md | added 1 condition(s) | ~577 |
| 01:36 | Edited src/docs/cluster-config-split-analysis.md | 6→7 lines | ~154 |
| 01:36 | Session end: 4 writes across 1 files (cluster-config-split-analysis.md) | 8 reads | ~61375 tok |
## Session: 2026-06-09 01:59
| Time | Action | File(s) | Outcome | ~Tokens |
|------|--------|---------|---------|--------|
## Session: 2026-06-09 02:09
| Time | Action | File(s) | Outcome | ~Tokens |
|------|--------|---------|---------|--------|
| 02:27 | Edited src/lib/raft/raft.c | modified if() | ~129 |
| 02:27 | Edited src/controller/ha.c | strcmp() → deny() | ~60 |
| 02:28 | Edited src/controller/myapp.c | modified if() | ~497 |
| 02:28 | Edited src/controller/myapp.c | modified if() | ~281 |
| 02:41 | Edited src/lib/raft/raft.c | voted() → grant() | ~210 |
| 02:46 | Session end: 5 writes across 3 files (raft.c, ha.c, myapp.c) | 5 reads | ~72268 tok |
| 03:17 | Edited src/controller/ha.c | added 3 condition(s) | ~497 |
| 03:22 | Edited src/controller/ha.h | expanded (+15 lines) | ~199 |
| 03:22 | Edited src/controller/ha.c | added 2 condition(s) | ~530 |
| 03:23 | Edited src/controller/ha.c | removed 41 lines | ~43 |
| 03:23 | Edited src/controller/ha.c | removed 33 lines | ~2 |
| 03:23 | Edited src/controller/myapp.c | 5→10 lines | ~121 |
| 03:26 | Session end: 11 writes across 4 files (raft.c, ha.c, myapp.c, ha.h) | 8 reads | ~84261 tok |
| 04:02 | Session end: 11 writes across 4 files (raft.c, ha.c, myapp.c, ha.h) | 9 reads | ~84766 tok |
| 05:56 | Edited src/controller/ha.c | 2→3 lines | ~47 |
| 06:00 | Session end: 12 writes across 4 files (raft.c, ha.c, myapp.c, ha.h) | 10 reads | ~84423 tok |
| 06:02 | Session end: 12 writes across 4 files (raft.c, ha.c, myapp.c, ha.h) | 10 reads | ~84423 tok |
| 06:06 | Edited src/lib/raft/raft.c | modified if() | ~178 |
| 06:10 | Session end: 13 writes across 4 files (raft.c, ha.c, myapp.c, ha.h) | 11 reads | ~84614 tok |
## Session: 2026-06-09 16:49
| Time | Action | File(s) | Outcome | ~Tokens |
|------|--------|---------|---------|--------|
## Session: 2026-06-09 16:51
| Time | Action | File(s) | Outcome | ~Tokens |
|------|--------|---------|---------|--------|
## Session: 2026-06-09 16:52
| Time | Action | File(s) | Outcome | ~Tokens |
|------|--------|---------|---------|--------|
| 14:xx | Validation-pass gap analysis appended for worker-join-refactor | .kiro/specs/worker-join-refactor/research.md | implemented, only 5.4 test-run unverified | ~9k |
| 16:59 | Edited .kiro/specs/resval-id-decoupling/research.md | modified and() | ~2232 |
| 00:02 | Validation-pass gap analysis appended for scheduler-controller-msg-gap (found G-1 resvals idx-0 bug, G-2 resource EEXIST, leaks, test debt) | .kiro/specs/scheduler-controller-msg-gap/research.md | partial, not 100% | ~12k |
| 00:02 | Gap analysis validation pass for ha-heartbeat-liveness; found dead last_heartbeat field feeding vote gate | .kiro/specs/ha-heartbeat-liveness/research.md, .wolf/buglog.json | appended analysis + bug entry | ~25k |
| 00:03 | Gap analysis validation pass for ha-config-bootstrap spec (50 ACs verified against on-disk code) | .kiro/specs/ha-config-bootstrap/research.md | appended analysis: ~40/50 implemented, 7 partial, 3 pivoted | ~45k |
| 16:55 | Gap analysis validation pass for ha-peer-connection-reuse (18/19 reqs Implemented, 5.5 Partial: ASAN leaks); ran controller_unit_test (769 PASS/0 FAIL) | .kiro/specs/ha-peer-connection-reuse/research.md | appended analysis | ~30k |
| 00:13 | gap-analysis validation pass: election-iomgr-refactor — found 2 failing unit tests from uncommitted ha.c vote changes | .kiro/specs/election-iomgr-refactor/research.md | appended analysis + buglog entry | ~30k |
| 00:13 | Ran /kiro-validate-gap across all 9 specs via 9 parallel agents; appended Gap Analysis Validation Pass to each research.md | .kiro/specs/*/research.md | found 1 critical (repl data plane), 4 bugs logged (335-338) | ~1130k |
| 17:14 | Created ../../../../../claude/.claude/projects/-home-rshen-projects-xce-xce/memory/project_ha_bootstrap_progress.md | — | ~576 |
| 17:14 | Edited ../../../../../claude/.claude/projects/-home-rshen-projects-xce-xce/memory/MEMORY.md | inline fix | ~54 |
| 17:15 | Session end: 3 writes across 3 files (research.md, project_ha_bootstrap_progress.md, MEMORY.md) | 50 reads | ~194368 tok |
## Session: 2026-06-10 17:19
| Time | Action | File(s) | Outcome | ~Tokens |
|------|--------|---------|---------|--------|
## Session: 2026-06-10 17:19
| Time | Action | File(s) | Outcome | ~Tokens |
|------|--------|---------|---------|--------|
## Session: 2026-06-10 17:20
| Time | Action | File(s) | Outcome | ~Tokens |
|------|--------|---------|---------|--------|
| 17:22 | Edited src/lib/testsuite/test_raft.c | 7→10 lines | ~106 |
| 17:22 | Edited src/lib/testsuite/test_raft.c | modified test_proto_H6_restored_vote_blocks_second_vote() | ~488 |
| 17:22 | Edited src/lib/testsuite/test_raft.c | 1→2 lines | ~34 |
| 17:23 | Edited src/lib/raft/raft.c | 4→8 lines | ~135 |
| 17:23 | Edited src/controller/ha.c | added 1 condition(s) | ~243 |
| 17:23 | Edited src/controller/ha.c | 5→8 lines | ~113 |
| 17:23 | Edited src/controller/ha.h | 4→4 lines | ~79 |
| 17:23 | Edited src/controller/ha.h | inline fix | ~26 |
| 17:24 | Edited src/controller/testsuite/test_ha_vote_gate.c | set_last_heartbeat_age() → set_last_activity_age() | ~114 |
| 17:24 | Edited src/controller/testsuite/test_ha_vote_gate.c | 2→7 lines | ~102 |
| 17:24 | Edited src/controller/testsuite/test_ha_vote_gate.c | set_last_heartbeat_age() → set_last_activity_age() | ~59 |
| 17:24 | Edited src/controller/myapp.c | 13→17 lines | ~241 |
| 17:45 | Fixed bug-337 (freshness→last_activity), bug-338 (same-candidate re-grant + myapp self check), bug-339 (raft_init voted_for_term restore); full build+tests green; reopened ha-repl-hardening tasks 7.1-7.7; marked worker-join-refactor + resval-id-decoupling implemented | ha.c ha.h myapp.c raft.c test_raft.c test_ha_vote_gate.c tasks.md spec.json | all tests pass, 0 FAIL | ~80k |
| 17:32 | Edited ../../../../../claude/.claude/projects/-home-rshen-projects-xce-xce/memory/project_ha_bootstrap_progress.md | 3→3 lines | ~138 |
| 17:32 | Session end: 13 writes across 7 files (test_raft.c, raft.c, ha.c, ha.h, test_ha_vote_gate.c) | 2 reads | ~37627 tok |

File diff suppressed because it is too large Load diff