Commit graph

16 commits

Author SHA1 Message Date
9b6a91a89e feat: implement restore_job/worker_from_capnp for WAL recovery
WAL hooks now write real Cap'n Proto payloads instead of stub uint32_t:
- WAL_JOB_SUBMIT: full xce_job_t serialized via encode_Job/write_Job
- WAL_WORKER_CTRL: [uint32_t detail][xce_worker_t capnp] via encode_Worker
- Other job events (START/FINISH/FAIL/STOP etc): uint64_t jobid only

restore_job(): deserializes Cap'n Proto xce_job_t, guards against duplicate
replay with key_find check, then calls create_job()

worker_from_capnp(): skips 4-byte detail prefix, deserializes xce_worker_t,
guards against duplicate replay, then calls restore_worker()

join.c: pass worker_t* (looked up by idx via vget) to WORKER_JOIN hook
so persist hook can serialize worker data without touching workermgr RTTI
2026-05-08 07:49:27 +00:00
3030012f51 feat: JOB_STAT_TRANSIENT, job_should_retain, worker_should_retain, dead_since (P0-5a, P0-8, P0-13)
Add terminal-state constants and GC retention predicates for the
checkpoint xzarray_thaw() GC pass:

- types.h: add JOB_STAT_TRANSIENT (0x400), XCE_WORKER_HIDDEN (0x00200000),
  QUEUE_STAT_HIDDEN (0x0100)
- persist.c: implement job_is_terminal(), job_should_retain(),
  worker_should_retain() — placed here so the unit test binary
  (which links only persist.c) can access them without RTTI deps
- persist.h: declare the three GC predicates; include xce/types.h
- worker.c: set dead_since = time(NULL) in mark_worker_unavail()
  when dead_since is not already set
- cluster.c: add TODO P0-8b marker on zarray_del endpoint cleanup
- meson.build: register test_gc_retention.c; raise test timeout 60→120s
- testsuite/test_gc_retention.c: 19 tests covering job_is_terminal,
  job_should_retain (TRANSIENT TTL, terminal TTL, non-terminal always
  retained), and worker_should_retain (HIDDEN/UNAVAIL/healthy/zero
  dead_since); all pass (63 total, 0 failures)
2026-05-07 12:52:07 +00:00
045d9e3287 feat: add HOOK_DETAIL_JOB_STOP/RESUME; fix xce_reply ordering after AFTER_EVENT hooks (P0-4a)
- cluster_def.h: add struct tag `cluster_s` to cluster_t to resolve
  conflicting typedef with persist.h forward declaration
- cluster_def.h: add HOOK_DETAIL_JOB_STOP and HOOK_DETAIL_JOB_RESUME enum
  values after HOOK_DETAIL_JOB_RUSAGE_UPDATE
- job.c on_job_stop(): replace mark_job_update() call with direct
  xce_run_hooks(BEFORE/AFTER, HOOK_DETAIL_JOB_STOP) so WAL writer
  records stop events with the correct detail value
- job.c on_job_resume(): same pattern with HOOK_DETAIL_JOB_RESUME
- job.c do_clr_preason(): fix typo — second hook call was HOOK_BEFORE_EVENT
  instead of HOOK_AFTER_EVENT (double-before, zero-after bug)
- join.c handle_worker_join(): move xce_reply() to after HOOK_AFTER_EVENT
  so client ACK is not sent before WAL write completes (durability fix)
- testsuite/test_hooks.c: new TDD tests for JOB_STOP/RESUME enum values
- testsuite/controller_test.c + meson.build: register test_hooks suite

Reply ordering audit results:
  job.c handle_jobsubmit: reply before AFTER_EVENT — intentional (jobid
    returned to client before scheduler notification; WAL hook belongs in
    BEFORE_EVENT for this path, handled in Task 13)
  job.c do_set_preason: reply before set_preason() loop — intentional
    fire-and-forget (no AFTER_EVENT hook at top level)
  job.c do_clr_preason: reply before loop — same fire-and-forget pattern
  job.c do_jobcontrol_op: no AFTER_EVENT hook, reply only — OK
  join.c handle_worker_join: FIXED (reply moved after AFTER_EVENT)
  worker.c on_worker_update/dead: reply after all hooks — OK
  cluster.c: AFTER_EVENT hooks are internal (no reply) — OK
2026-05-07 12:08:52 +00:00
f905f891a8 feat: implement persist_recover with snapshot load and WAL replay (P0-4)
- Add load_snapshot(): reads P0 sentinel ("XCESNAP\0" + lsn), validates magic,
  extracts checkpoint_lsn
- Add replay_wal(): scans seg-*.wal files in LSN order, skips entries <= from_lsn,
  verifies CRC chain, stops on mismatch, applies entries via apply_wal_entry() stub
- Add persist_recover(): find latest .capnp snapshot, load it, replay post-checkpoint
  WAL, retention pass (no-op in P0)
- Fix trim_wal_segments(): compare against active segment start LSN (not checkpoint_lsn)
  to avoid deleting the currently-open segment and losing post-checkpoint WAL entries
- Add restore_job() stub in job.c (P0-12a placeholder)
- Add worker_from_capnp() stub in worker.c (P0-12 placeholder)
- Declare both new functions in cluster.h and worker.h
- Add 5 recovery tests: empty state, snapshot lsn loading, WAL replay, corrupt
  sentinel rejection, latest-snapshot selection; all 23 controller tests pass
2026-05-07 11:41:33 +00:00
dffa44d59b chore: remove dead code from job.c/cluster.c (Part 1)
- job.c: remove unused #define NJOBS (100) constant
- job.c: remove unused global uint64_t jobid = 100
- job.c: remove #if 0 dead block inside load_jobs()
- cluster.c: expand cluster_build_groups() stub with TODO comment
2026-05-07 08:31:29 +00:00
3f74f14687 save works 2026-03-04 18:18:53 +08:00
acfbec875b save works 2026-03-03 16:49:26 +08:00
f4b1ba4cd7 save works 2026-03-02 12:14:30 +08:00
b748e30d20 save works 2026-02-28 17:55:50 +08:00
a46ea822c1 working on scheduler and fix issue #17 2025-12-25 16:24:56 +08:00
cb0c7fe9e7 merge job rusage and pending reason support 2025-12-16 14:36:28 +08:00
e001d4955c fix memory leak 2025-12-10 15:25:05 +08:00
e497dd1b69 job submit and support run in worker 2025-11-28 17:24:31 +08:00
8f796b3109 security communication with openssl and simplify netio 2025-11-18 11:26:36 +08:00
9ff857db9f reformat code and job array 2025-11-07 16:14:27 +08:00
Rongsong Shen
41f56fa6ed update controller code 2025-11-05 16:58:57 +08:00