Commit graph

27 commits

Author SHA1 Message Date
65ddafeb66 feat(controller-status): add responderTimeMs @13 to CtrlStatus schema + typemap (task 6.1)
Append-only UInt64 epoch-millis field for the responder snapshot timestamp
(Req 11.1, wire-format setup). Hand-authored xce_ctrlstatus_t typemap gains a
matching responder_time_ms member; codec round-trips it. Controller suite 1/1 OK.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 08:56:51 +00:00
a6e03d87ab feat(controller-status): handle_ctrlstatus read-only state snapshot + dispatch + configChecksum as Text hex (task 2.1) 2026-06-20 04:53:27 +00:00
97a501e96e feat(controller-status): add ctrlstatus opcode + CtrlStatus/CtrlPeer schema + typemap typedefs (task 1.1) 2026-06-20 04:32:42 +00:00
0d660264f5 feat(ha-prevote): add preVote request/reply wire opcodes (task 1.1)
Append preVoteRequest @38 / preVoteReply @39 to the OPCode enum, reusing the
existing RequestVoteRequest/RequestVoteReply payload structs and union arms
(no new structs, no isPreVote flag). Keeps the real-vote wire path unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 03:59:47 +00:00
566c91cd8b feat(ha-wal-replication): standby WAL ACK reply (tasks 3.1-3.3)
Standby sends WalAckReply via xce_reply after each successfully
applied WAL entry. ha_apply_wal_catchup_entry signature extended
with endpoint_t* and cluster_t* parameters. walackreplymsg union
arm added to RequestReply capnp schema and xce_reply_t. Error
entries skip ACK. Dispatch gate test confirms OPCode_walCatchupEntry
allowed in CTRL_STANDBY/ELECTING/INIT.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-06-10 09:14:25 +00:00
0121bc0683 feat(ha-repl-hardening): add snapshot chunk fields to capnp schema (task 1.3)
Extend SnapshotTransfer struct from 2 to 6 fields:
- snapshotLsn (existing), chunkIndex, chunkOffset, totalSize,
  isLast, payload (existing)
No resumption fields per restart-on-reconnect policy.
2026-06-04 13:55:45 +00:00
51907ebfbc feat(ha-repl-hardening): add batch entries list to WalCatchupEntry schema (task 1.2)
Add self-referential entries @3 :List(WalCatchupEntry) field with
$C.maplistcount("n_entries"). Single-entry path unaffected — n_entries
defaults to 0, entries to NULL when not using batch mode.
2026-06-04 13:46:04 +00:00
9a977b3a43 feat(ha-repl-hardening): add StateReport to RequestReply capnp schema (task 1.1)
Add StateReport struct {appliedLsn, configTag} as stateReportMsg @15
variant in RequestReply.data union. Standby populates this in heartbeat
replies so the primary can track per-standby convergence.

capnpc-c 1.0.1 has a bug with inline :group inside codecgen unions;
workaround uses named outer struct (wire-equivalent). typemap.h
extended with state_report_t and union member for C codec access.
2026-06-04 13:34:11 +00:00
6753b364c8 fix(ha-bootstrap): wire OP_requestVote RPC through standard Request envelope
The vote_request_thread was sending raw capnp(RequestVoteRequest) bytes
but the server-side iomgr decodes every incoming packet as a Request
struct. This caused the term field (uint64) to be misread as the opcode
field (uint16), so vote RPCs were silently misrouted (term=1 → OPCode_reply,
term=2 → OPCode_hello) and no node ever got a vote from its peers.

Fix:
- controller.capnp: add requestvotemsg @27 to Request.body and
  requestvotereplymsg @14 to RequestReply.data
- typemap.h: move request_vote_req_t/reply_t before xce_request_t;
  add requestvotemsg/requestvotereplymsg to body/data unions
- ha.c: rewrite vote_request_thread send/recv to use xce_encode()
  wrapper and decode outer Request→inner RequestReply on receive
- cluster.c: add case OPCode_requestVote handler calling
  handle_request_vote() and replying via xce_reply()

All 188 unit tests pass.
2026-05-29 17:01:59 +00:00
45f40c51c1 fix(ha-bootstrap): restore vcpus@7 field as deprecated UInt32 for config blob backward compatibility 2026-05-29 13:55:23 +00:00
94bf5d94b8 feat(ha-config-bootstrap): implement OP_ctrlHello state sync handler (task 9.3) 2026-05-29 05:22:17 +00:00
3ab2e6e4bf feat(ha-config-bootstrap): implement --vcpus CLI, wire validation, remove vcpus from config (task 2.2)
- Add --vcpus N argp option in main.c; store in myconf.vcpus
- Replace inline argp guard with validate_vcpus() call (returns -EINVAL for <=0)
- Add validate_startup_config() in startup_validation.c: returns 0; writes
  deprecation warning into caller-supplied warn_buf when vcpus key present
- Wire validate_startup_config() into myapp_run() after conffile resolved;
  emit WARN() if warn_buf non-empty (Req 4.4)
- Remove vcpus field from cluster_config_t (clusterconfig.h)
- Preserve capnp ordinal: vcpus @7 -> vcpusDeprecated @7: Void in controller.capnp
- vtaskmgr_init uses only myconf->vcpus with hardcoded default 8
- Add stub C types in typemap.h for capnp schema extensions from task 1.1
- 96 tests pass
2026-05-28 04:17:06 +00:00
6adb7e806f feat(ha-config-bootstrap): add 6 new opcodes and HA message structs to controller.capnp 2026-05-28 03:02:40 +00:00
78adf3f843 fix: normalize capnp union arm naming to lowercase; fix UpgradeConfigRequest field order
- Rename replConfigRequestMsg/replConfigPushRequestMsg/upgradeConfigRequestMsg
  to replconfigrequestmsg/replconfigpushrequestmsg/upgradeconfigrequestmsg in
  Request.body union to match existing lowercase convention
- Rename replConfigReplyMsg/replConfigPushReplyMsg/upgradeConfigReplyMsg
  to replconfigreplymsg/replconfigpushreplymsg/upgradeconfigreplymsg in
  RequestReply.data union
- Swap UpgradeConfigRequest fields so sha256@0/bytes@1 matches sibling
  ReplConfigPushRequest pattern
- Move HeartbeatMsg out of the config replication section into its own
  '# ── HA heartbeat ──' section above the replication structs
2026-05-13 14:10:33 +00:00
683c092b77 feat: add replConfig/replConfigPush/upgradeConfig opcodes and message structs to capnp schema 2026-05-13 14:06:51 +00:00
d47b3375e7 controller will pass resource <name, value> pair than <idx, value> pair 2026-05-13 07:18:03 +00:00
4903c83f28 feat: add PersistConfig, SnapshotHeader, ReplEnvelope Cap'n Proto schemas (P0-9)
Extends src/schema/controller.capnp with persistence structures needed
for WAL replication and snapshots:

- PersistConfig (8 fields, @0-@7): capnp-serialisable copy of the
  persistence configuration; mapped to capnp_persist_cfg_t to avoid
  collision with the plain-C persist_config_t in persist.h
- SnapshotHeader (9 fields, @0-@8): snapshot manifest with LSN, term,
  timestamp, record counts and checksums; mapped to capnp_snap_hdr_t
- ReplEnvelope (6 fields, @0-@5): replication wire envelope (NOT the
  on-disk format); named to avoid collision with wal_entry_hdr_t and
  the WALEntry_ internal symbol c-capnproto generates; mapped to
  capnp_repl_envelope_t

Also extends existing structs:
- Job.transientSince @11 (UInt64=0, mapname transient_since): set when
  job enters JOB_STAT_TRANSIENT; also adds uint64_t transient_since to
  xce_job_t in types.h
- Worker.deadSince @6 (UInt64=0, mapname dead_since): mirrors existing
  time_t dead_since field in xce_worker_t

OPCode enum extended with 7 replication opcodes @22-@28:
  replEntry, replAck, redirect, heartbeat, steppingDown,
  replSnapshot, replCatchUpDone

New C typedefs (capnp_persist_cfg_t, capnp_snap_hdr_t,
capnp_repl_envelope_t) added to typemap.h for the encode/decode shims.

TDD: test_persist_schema.c added (6 tests, all GREEN). Controller test
suite remains 1/1 PASS.
2026-05-07 10:19:33 +00:00
177cf45af5 docs: mark Phase 6 complete in phase6-scheduler-integration-plan 2026-04-30 07:17:32 +00:00
dcb1b4fe25 feat: P0-2a C-side plumbing for host control
Add the full call chain for host control operations from the Go API
server through the C library to the controller:

- schema/controller.capnp: HostControl struct + hostctrlmsg @21
  in Request.body union
- include/typemap.h: xce_hostctrl_t* hostctrlmsg in body union
  (placed after @20 clrpreasonmsg, matching capnp ordinal)
- include/client.h, xceapi.h, request.h: declarations for
  xcecli_hostctrl, xce_hostctrl, xce_hostctrl_req
- lib/client.c: xcecli_hostctrl (validate + delegate to xce_hostctrl)
- lib/xceapi.c: xce_hostctrl (validate ctx state + send + wait reply)
- lib/request.c: xce_hostctrl_req (validate hostname/empty/nil,
  accept only HOST_OPEN/HOST_CLOSE, reject HOST_TAG/UNTAG)
- lib/helper.c: Request_body_hostctrlmsg cases in
  xce_request_body() and xce_request_msg_name()
- controller/cluster.c: OPCode_hostcontrol dispatch
- controller/include/cluster.h: do_hostcontrol_op declaration
- controller/worker.c: do_hostcontrol_op + fix pre-existing
  goto-clean bug in worker_ctrl that leaked write lock and
  reference on HOST_OPEN/HOST_CLOSE no-op paths

TDD: 8 xce_hostctrl_req validation tests (null ep, null ctrl,
null/empty hostname, HOST_DO_NONE no-op, HOST_TAG/UNTAG reject,
negative action reject, HOST_OPEN/CLOSE accept via xce_request).

116/116 xcelib tests pass. CGO_ENABLED=0 Go backend tests pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-17 07:46:41 +00:00
3b728e5135 improve support of api server in controller 2026-01-23 16:05:23 +08:00
70970434ca initial work of api server 2026-01-20 18:20:24 +08:00
ecf1916d78 support limit 2026-01-12 15:27:54 +08:00
780e9fb313 scheduling using real slots number 2025-12-30 18:13:15 +08:00
4db6b039c5 save works 2025-12-18 14:55:58 +08:00
cb0c7fe9e7 merge job rusage and pending reason support 2025-12-16 14:36:28 +08:00
e497dd1b69 job submit and support run in worker 2025-11-28 17:24:31 +08:00
Rongsong Shen
346768a144 use nix build & meson & zlog 2025-11-03 18:19:59 +08:00