lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat: implement restore_job/worker_from_capnp for WAL recovery

Browse source 
WAL hooks now write real Cap'n Proto payloads instead of stub uint32_t:
- WAL_JOB_SUBMIT: full xce_job_t serialized via encode_Job/write_Job
- WAL_WORKER_CTRL: [uint32_t detail][xce_worker_t capnp] via encode_Worker
- Other job events (START/FINISH/FAIL/STOP etc): uint64_t jobid only

restore_job(): deserializes Cap'n Proto xce_job_t, guards against duplicate
replay with key_find check, then calls create_job()

worker_from_capnp(): skips 4-byte detail prefix, deserializes xce_worker_t,
guards against duplicate replay, then calls restore_worker()

join.c: pass worker_t* (looked up by idx via vget) to WORKER_JOIN hook
so persist hook can serialize worker data without touching workermgr RTTI
This commit is contained in:
Rongsong Shen 2026-05-08 07:49:27 +00:00
parent b80ca3deaa
commit 9b6a91a89e
4 changed files with 239 additions and 42 deletions
Download patch file Download diff file Expand all files Collapse all files

43
src/controller/job.c
View file

@ -1,9 +1,11 @@
#include <errno.h>
#include <signal.h>
#include <stdbool.h>
#include <string.h>
#include <time.h>
#include "cluster.h"
#include "controller.capnp.h"
#include "jobj.h"
#include "persist.h"
#include "preason.h"
@ -63,15 +65,40 @@ int load_jobs(void* c)
return 0;
}
/* P0 stub — full deserialization wired in Task 15 (cluster wiring).
* Inserts a job at its original jobid without allocating a new one.
* Returns -EEXIST if jobid already present (safety guard for replay). */
int restore_job(cluster_t* c, uint64_t jobid, const void* capnp_payload, size_t len)
/* Reconstruct a job from a WAL_JOB_SUBMIT Cap'n Proto payload.
* Uses the jobid embedded in the serialized xce_job_t (not the caller's arg).
* Returns -EEXIST if the job is already present (idempotent replay guard). */
int restore_job(cluster_t* c, uint64_t jobid_unused,
const void* capnp_payload, size_t len)
{
(void)c;
(void)jobid;
(void)capnp_payload;
(void)len;
(void)jobid_unused;
if (!capnp_payload || len == 0)
return -EINVAL;
struct capn c_ctx;
capn_init_mem(&c_ctx, (const uint8_t*)capnp_payload, len, 0);
Job_ptr jp;
jp.p = capn_getp(capn_root(&c_ctx), 0, 1);
struct Job cj;
xce_job_t j;
memset(&j, 0, sizeof(j));
read_Job(&cj, jp);
decode_Job(&j, &cj);
capn_free(&c_ctx);
/* Safety guard: skip if already present (WAL replay may see duplicates
* when recovery restarts from a segment boundary). */
job_t* existing = NULL;
lock(c->jobmgr, RTTI_LOCK_READ);
int found = key_find(c->jobmgr, (void*)j.jobid, (void**)&existing);
unlock(c->jobmgr);
if (found > 0 && existing != NULL)
return -EEXIST;
job_t* job = create_job(c, &j);
if (!job)
return -errno;
return 0;
}