Commit graph

9 commits

Author SHA1 Message Date
ca1d9ae42d fix(ha-commit-engine): raise commit-barrier default 50->500ms + add --sync-timeout-ms
Live symptom: jobsub intermittently failed with 'Connection timed out' even
though the cluster had quorum. Root cause: the synchronous commit barrier waits
only DEFAULT_SYNC_TIMEOUT_MS=50ms for durable majority replication, and that was
not configurable. 50ms is too tight for durable sync majority replication —
especially since standbys now fdatasync each replicated entry before ACK — so a
multi-entry jobsub's later WAL entry intermittently misses the window:
  commit: WAIT deadline expired without commit (lsn=3 committed_lsn=2 timeout_ms=50)
-> persist hook returns -ETIMEDOUT -> handle_jobsubmit rejects -> client ETIMEDOUT.

Fix:
- Raise DEFAULT_SYNC_TIMEOUT_MS 50 -> 500 (config.h) — realistic for durable sync
  majority replication, still bounded for failure detection.
- Add a --sync-timeout-ms controller flag threaded arguments -> myconf ->
  persist pcfg (override applied before persist_init); 0 = use default, positive
  overrides — so operators can tune per environment.

TDD: literal-pinned persist_config_defaults().sync_timeout_ms == 500. All
timeout-sensitive tests already pass an explicit timeout_ms (none rely on the
default), so none are slowed; full controller suite green under ASAN. No
commit-engine/barrier/replication/wire change beyond the default value + plumbing.
bug-561.
2026-06-14 20:19:05 +00:00
3ab2e6e4bf feat(ha-config-bootstrap): implement --vcpus CLI, wire validation, remove vcpus from config (task 2.2)
- Add --vcpus N argp option in main.c; store in myconf.vcpus
- Replace inline argp guard with validate_vcpus() call (returns -EINVAL for <=0)
- Add validate_startup_config() in startup_validation.c: returns 0; writes
  deprecation warning into caller-supplied warn_buf when vcpus key present
- Wire validate_startup_config() into myapp_run() after conffile resolved;
  emit WARN() if warn_buf non-empty (Req 4.4)
- Remove vcpus field from cluster_config_t (clusterconfig.h)
- Preserve capnp ordinal: vcpus @7 -> vcpusDeprecated @7: Void in controller.capnp
- vtaskmgr_init uses only myconf->vcpus with hardcoded default 8
- Add stub C types in typemap.h for capnp schema extensions from task 1.1
- 96 tests pass
2026-05-28 04:17:06 +00:00
1c4a103a66 fix: resolve probe_ctx_t type tag, add DEFAULT_MAX_WAL_RECORDS macro, fix atomic type consistency, move ha_seed_jitter to ha.c, add API doc comments 2026-05-13 14:27:07 +00:00
2c375cf476 feat: add config replication fields to myconf_t, cluster_t, ha_state_t, persist_t; create byteorder.h 2026-05-13 14:15:30 +00:00
59597e105e feat: add --peers to controller for HA peer list and priority election
- myconf_t gains peers_cfg (xce_cluster_config_t) populated from --peers
- controller/main.c: parse --peers host:port,... via argp; ca_cert_path
  auto-derived from workdir/certs/xce.crt (no --ca-cert flag needed)
- cluster_init: derive my_priority from position in peers list (index 0 =
  primary); fall back to 0 when --peers not given (single-node mode)
2026-05-08 04:40:27 +00:00
b52fbef36d feat: wire persist_init/recover/start/stop into cluster lifecycle (P0-F)
- Add `persist_t persist` field to cluster_t (cluster_def.h includes persist.h)
- Add `workdir` and `name` fields to myconf_t; main.c passes them through
- cluster_init: call persist_set_workdir, persist_init, persist_recover after apiserver_init
- cluster_start: call persist_start (launches fsync thread + registers WAL hooks)
- cluster_fini: call persist_stop + persist_fini before delaymgr/vtaskmgr teardown
- persist_init: store cluster back-pointer (replaces (void)c stub)
- persist_start: register persist_job/queue/worker_hook_fun via xce_add_hook when cluster != NULL
- Add test_cluster_persist.c: compile-time struct check + config defaults + workdir path derivation
- Register test in controller_test.c and meson.build

All controller tests pass (1/1 OK, 50s).
2026-05-07 13:07:47 +00:00
8f796b3109 security communication with openssl and simplify netio 2025-11-18 11:26:36 +08:00
9f91f3df80 eliminate initial configure(use command line options) 2025-11-06 16:12:02 +08:00
Rongsong Shen
41f56fa6ed update controller code 2025-11-05 16:58:57 +08:00