Commit graph

19 commits

Author SHA1 Message Date
f0d6074d4e feat(ha-commit-engine): engine instantiation + sync-append path switch (task 5.3)
cluster_t embeds commit_engine_t; init in cluster_init_phase2 after the
vtaskmgr guard (idempotence-guarded — phase2 can run twice), wiring
repl_set_commit_engine + ha.repl back-pointer before any standby can
register; fini after iomgr/vtask shutdown. persist_wal_sync_append
passes written_lsn; engine-wired path drops -ENODEV and the
committed_lsn store (engine owns advancement); LSN-0 first-entry
routes to legacy per the accepted interim. p2 -ENODEV pin updated to
the Req 6.3 contract. 34/34 + 6/6 green through honest delegation;
census exactly at the 3 validated baselines; production binary links.
2026-06-12 00:16:22 +00:00
3d076ccda6 fix(ha): fix config replication to standbys; add replConfigPush handler
- config_repl.c: add handle_repl_config_push — standby handler for
  OPCode_replConfigPush. Verifies SHA-256, saves blob to disk via
  conf_save_config, reloads via load_primary_config, ACKs primary.
- config_repl.h: declare handle_repl_config_push.
- cluster.c: dispatch OPCode_replConfigPush to handle_repl_config_push;
  populate c->myhost_info.name from myconf->name (ctrl-1/ctrl-2/ctrl-3)
  instead of the static "controller" placeholder.
- ha.c: fix raft_cb_become_leader push_config_to_standbys call to pass
  real config bytes and tag read from disk (was NULL/0 — standbys got
  empty push and never saved config).
- typemap.h: add raw_sha256/raw_bytes/raw_bytes_len fields to
  repl_config_push_req_t (mirrors upgrade_config_req_t pattern).
- request.c: pre-extract List(UInt8) sha256/bytes from OPCode_replConfigPush
  before capn_free() in xce_decode(); free them in xce_free_request().
- helper.c: add all HA/replication opcode names to xce_request_op_name
  so "unknown opcode 30" becomes "replConfigPush" in logs.
- cluster_def.h: add embedded myhost_info field to cluster_t struct.
- ha.h: declare ha_start_primary_heartbeat; remove dead
  ha_find_and_connect_to_primary declaration.
2026-06-01 05:56:16 +00:00
c42e1e0cb4 refactor(controller): extract init_controller_tunables() for perf params
Add ctrl_tunables struct to cluster_t with per-table fields:
  jobmgr_nhash/delta, workermgr_nhash/delta, qmgr_nhash/delta,
  resmgr_nhash/delta/inst, xstrings_nhash

init_controller_tunables() sets defaults (jobs=512, others=128)
and is called from cluster_init_phase1 so values are available
before cluster_init_phase2. Phase2 reads from c->tunables instead
of cluster_config or local #defines.

Each table gets its own values since they hold different item counts:
jobs can scale to hundreds, workers/queues/resources are typically
single-digit to tens.
2026-05-30 01:14:40 +00:00
2c375cf476 feat: add config replication fields to myconf_t, cluster_t, ha_state_t, persist_t; create byteorder.h 2026-05-13 14:15:30 +00:00
c627f69d50 feat: wire ha_state_t into cluster_t + HA startup/shutdown lifecycle (P1-5) 2026-05-07 14:55:48 +00:00
b52fbef36d feat: wire persist_init/recover/start/stop into cluster lifecycle (P0-F)
- Add `persist_t persist` field to cluster_t (cluster_def.h includes persist.h)
- Add `workdir` and `name` fields to myconf_t; main.c passes them through
- cluster_init: call persist_set_workdir, persist_init, persist_recover after apiserver_init
- cluster_start: call persist_start (launches fsync thread + registers WAL hooks)
- cluster_fini: call persist_stop + persist_fini before delaymgr/vtaskmgr teardown
- persist_init: store cluster back-pointer (replaces (void)c stub)
- persist_start: register persist_job/queue/worker_hook_fun via xce_add_hook when cluster != NULL
- Add test_cluster_persist.c: compile-time struct check + config defaults + workdir path derivation
- Register test in controller_test.c and meson.build

All controller tests pass (1/1 OK, 50s).
2026-05-07 13:07:47 +00:00
045d9e3287 feat: add HOOK_DETAIL_JOB_STOP/RESUME; fix xce_reply ordering after AFTER_EVENT hooks (P0-4a)
- cluster_def.h: add struct tag `cluster_s` to cluster_t to resolve
  conflicting typedef with persist.h forward declaration
- cluster_def.h: add HOOK_DETAIL_JOB_STOP and HOOK_DETAIL_JOB_RESUME enum
  values after HOOK_DETAIL_JOB_RUSAGE_UPDATE
- job.c on_job_stop(): replace mark_job_update() call with direct
  xce_run_hooks(BEFORE/AFTER, HOOK_DETAIL_JOB_STOP) so WAL writer
  records stop events with the correct detail value
- job.c on_job_resume(): same pattern with HOOK_DETAIL_JOB_RESUME
- job.c do_clr_preason(): fix typo — second hook call was HOOK_BEFORE_EVENT
  instead of HOOK_AFTER_EVENT (double-before, zero-after bug)
- join.c handle_worker_join(): move xce_reply() to after HOOK_AFTER_EVENT
  so client ACK is not sent before WAL write completes (durability fix)
- testsuite/test_hooks.c: new TDD tests for JOB_STOP/RESUME enum values
- testsuite/controller_test.c + meson.build: register test_hooks suite

Reply ordering audit results:
  job.c handle_jobsubmit: reply before AFTER_EVENT — intentional (jobid
    returned to client before scheduler notification; WAL hook belongs in
    BEFORE_EVENT for this path, handled in Task 13)
  job.c do_set_preason: reply before set_preason() loop — intentional
    fire-and-forget (no AFTER_EVENT hook at top level)
  job.c do_clr_preason: reply before loop — same fire-and-forget pattern
  job.c do_jobcontrol_op: no AFTER_EVENT hook, reply only — OK
  join.c handle_worker_join: FIXED (reply moved after AFTER_EVENT)
  worker.c on_worker_update/dead: reply after all hooks — OK
  cluster.c: AFTER_EVENT hooks are internal (no reply) — OK
2026-05-07 12:08:52 +00:00
43dc12a36c save works 2026-03-06 18:20:30 +08:00
3f74f14687 save works 2026-03-04 18:18:53 +08:00
acfbec875b save works 2026-03-03 16:49:26 +08:00
b748e30d20 save works 2026-02-28 17:55:50 +08:00
e7dc09c604 fix memory leak in controller and use lock protect apiserver data structure 2026-01-26 15:35:45 +08:00
44fc5b23e9 push more to api server 2026-01-22 17:23:18 +08:00
8d78f0cbeb improve resmgr/jobmgr/qmgr/workermgr 2025-12-18 17:37:34 +08:00
2362b32825 improve resmgr 2025-12-18 16:22:32 +08:00
1f1e4fdaba improve resmgr 2025-12-18 16:20:41 +08:00
cb0c7fe9e7 merge job rusage and pending reason support 2025-12-16 14:36:28 +08:00
Rongsong Shen
41f56fa6ed update controller code 2025-11-05 16:58:57 +08:00
Rongsong Shen
346768a144 use nix build & meson & zlog 2025-11-03 18:19:59 +08:00