feat(scheduler-core): implement snapshot save and load (task 8.1)

Adds xce_sched_core_save_snapshot()/xce_sched_core_load_snapshot(),
split across snapshot.c/snapshot_wire.c/snapshot_load.c (800-line cap)
along real save/load/shared-helper boundaries. Serializes the full
core state (config, jobs, queues, workers, resources, resvals, users,
projects, prng_seed, internal_state) into/out of sched_snapshot_data_t
(task 7.3's committed C mirror of the capnp SnapshotData struct) as
plain in-memory structures -- no wire encoding here, that's a later
task. After load, rebuilds the picker's private state by re-invoking
its state-change hooks (queue updates, then job pushes) against the
restored core state, matching xce_core_push_events()'s own hook-firing
order; picker-private state itself is never stored in the snapshot.

Adds a new prng_seed field to xce_sched_core_t -- requirements.md/
design.md both presuppose it but no prior task added it. Seeded to 0,
no PRNG consumer wired yet (deliberately out of scope).

Cross-core behavioral-equivalence proven directly: two independent
cores, each with its own registered picker, save on one and load into
a freshly-initialized other, both driven through an identical real
session, decision sequences asserted identical in order.

Found and fixed two real bugs along the way: a missing prng_seed
restore in this task's own new code, and a genuine guard bug in
already-committed core.c (sched_get_user()/sched_get_project()
treating key_find()'s expected -ENOENT as fatal, silently breaking
get-or-create for any new name) that also affects the live production
path (make_sched_job(), JOBS_PUSH) -- a narrowly-scoped two-line fix,
independently traced to the primitive level and confirmed correct by
review.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019dw5WnN3DXStaEaPYfACU1

.wolf/buglog.json

@@ -14487,5 +14487,185 @@
     ],
     "occurrences": 1,
     "last_seen": "2026-07-31T04:50:00Z"
+  },
+  {
+    "id": "bug-1489",
+    "timestamp": "2026-08-03T04:00:00Z",
+    "error_message": "gcc: 'unknown type name xce_sched_core_t; did you mean xce_sched_ctx_t?' at core.h:92 (xce_sched_core_init prototype) when compiling scheduler.h consumers (handler.c, main.c, etc).",
+    "file": "src/scheduler/include/core.h",
+    "root_cause": "core.h defined `struct xce_sched_core_s { ... };` (the full body) but relied entirely on plugin.h's forward declaration (`typedef struct xce_sched_core_s xce_sched_core_t;`) for the typedef name -- core.h itself never declared it. scheduler.h includes core.h BEFORE plugin.h, so in that (the only real) include order, xce_sched_core_t was unresolved by the time core.h's own function prototypes referenced it.",
+    "fix": "Added `typedef struct xce_sched_core_s xce_sched_core_t;` directly in core.h, immediately before the struct body. An identical repeated typedef (same underlying type) is valid C11/gnu17, matching this codebase's existing cluster_t idiom (myconf.h/persist.h/ha.h/cluster_def.h all forward-declare independently) -- so the fix works regardless of which of core.h/plugin.h is included first.",
+    "tags": [
+      "scheduler-core",
+      "task-4.1",
+      "core-extraction",
+      "typedef-forward-decl",
+      "include-order"
+    ],
+    "related_bugs": [],
+    "occurrences": 1,
+    "last_seen": "2026-08-03T04:00:00Z"
+  },
+  {
+    "id": "bug-1490",
+    "timestamp": "2026-08-03T04:00:00Z",
+    "error_message": "ld: undefined reference to 'reset_sched_obj' when linking xce_scheduler (referenced from plugins/limit.c's limit_job_election()).",
+    "file": "src/scheduler/src/schedobj.c",
+    "root_cause": "While rewriting schedobj.c wholesale to relocate the CRUD accessors (sched_get_job/sched_add_job/etc, scheduler-core spec task 4.1) into core.c, the reset_sched_obj() function (unrelated to the relocation, a small xlist/zarray cleanup helper called by plugins/limit.c) was accidentally omitted from the rewritten file -- a transcription miss, not an intentional removal.",
+    "fix": "Re-added reset_sched_obj() verbatim (unchanged body) to schedobj.c, right after make_sched_obj() where it originally lived.",
+    "tags": [
+      "scheduler-core",
+      "task-4.1",
+      "core-extraction",
+      "link-error",
+      "whole-file-rewrite-risk"
+    ],
+    "related_bugs": [
+      "bug-1489"
+    ],
+    "occurrences": 1,
+    "last_seen": "2026-08-03T04:00:00Z"
+  },
+  {
+    "id": "bug-1491",
+    "timestamp": "2026-08-03T06:33:00Z",
+    "error_message": "AddressSanitizer: SEGV on unknown address 0x000000000000 in dataq_deq_head (lib/dataq.c:15), called from dataq_fini <- xce_sched_core_fini (core.c:126), when a test called xce_sched_core_init() then xce_sched_core_fini() back-to-back on a fresh xce_sched_core_t.",
+    "file": "src/scheduler/testsuite/test_core_jbuf_capacity_pin.c",
+    "root_cause": "xce_sched_core_init() memset()s the whole xce_sched_core_t, leaving core->pend_updates[0]/[1] NULL -- those two dataq_t* are only allocated+dataq_init()d by scheduler.c sched_init() AFTER calling xce_sched_core_init() (see core.c file header comment: pend_updates storage lives in xce_sched_core_t but its dataq_init/killer wiring stays driver-owned in scheduler.c). xce_sched_core_fini() unconditionally calls dataq_fini(core->pend_updates[0]) and [1] with no NULL guard, so calling fini() right after init() without going through sched_init()'s extra setup step dereferences NULL inside dataq_deq_head.",
+    "fix": "Not a production bug (xce_sched_core_fini()'s real caller, scheduler.c sched_fini(), always runs after sched_init() has populated pend_updates) -- fixed the test instead: test_jbuf_capacity_is_runtime_one() (task 4.2) calls only xce_sched_core_init() and reads core.jbuf.ncap, deliberately never calling xce_sched_core_fini() on a fixture built by init() alone. Matches this testsuite directory's established \"fixtures aren't torn down\" convention (test_picker_core_loop_boundary.c's header comment).",
+    "tags": [
+      "scheduler-core",
+      "task-4.2",
+      "core-init-fini",
+      "segv",
+      "null-deref",
+      "test-authoring-gotcha",
+      "pend_updates"
+    ],
+    "related_bugs": [
+      "bug-1489",
+      "bug-1490"
+    ],
+    "occurrences": 1,
+    "last_seen": "2026-08-03T06:33:00Z"
+  },
+  {
+    "id": "bug-1492",
+    "timestamp": "2026-08-04T01:04:00Z",
+    "error_message": "plugin_end_session() checks plugin->begin_session_fn != NULL but calls plugin->end_session_fn(plugin) -- a guard/call field mismatch",
+    "file": "src/scheduler/src/plugin.c",
+    "root_cause": "Pre-existing bug predating the scheduler-core spec (git blame: commit 638173dec, 2026-02-26). plugin_end_session()'s per-plugin loop guards on the wrong struct field before invoking the hook, so a plugin registering only end_session_fn (not begin_session_fn) never has it called; a plugin registering only begin_session_fn (not end_session_fn) would crash via a NULL function-pointer call if this branch were ever reached for it. Discovered incidentally during scheduler-core task 5.2's independent review while verifying the plugin_begin_session/plugin_end_session per-session bookkeeping fan-out is preserved in the new normal-mode driver loop -- not introduced by task 5.2, confirmed untouched by its diff.",
+    "fix": "NOT FIXED -- out of scope for task 5.2, flagged for a dedicated follow-up. The one-line fix is straightforward: change the guard to `plugin->end_session_fn != NULL` (mirroring plugin_begin_session()'s own correct `plugin->begin_session_fn != NULL` guard for its `begin_session_fn` call).",
+    "tags": [
+      "scheduler-core",
+      "task-5.2",
+      "plugin-c",
+      "guard-call-mismatch",
+      "pre-existing",
+      "legacy-fcfs-picker",
+      "not-fixed"
+    ],
+    "related_bugs": [],
+    "occurrences": 1,
+    "last_seen": "2026-08-04T01:04:00Z"
+  },
+  {
+    "id": "bug-1493",
+    "timestamp": "2026-08-04T12:35:00Z",
+    "error_message": "gcc: 'expected declaration specifiers before sched_snapshot_data_t' cascading into ~20 errors across typemap.h/controller.capnp.h/controller.capnp.c when building lib/controller.capnp.c (scheduler-core task 7.3 SnapshotData C struct).",
+    "file": "src/include/typemap.h",
+    "root_cause": "A doc comment on sched_snapshot_data_t read '...raw uint8_t*/size_t pattern...' -- the literal substring '*/' inside the comment text (no space between the uint8_t* pointer-star and the following /size_t) prematurely closed the C block comment. Everything after that point (the rest of the comment prose plus the following struct's opening brace) was parsed as real C tokens, corrupting the sched_snapshot_data_t typedef and cascading into every generated function signature that references it.",
+    "fix": "Reworded the comment to avoid the accidental */ sequence ('uint8_t-pointer + size_t pattern' instead of 'uint8_t*/size_t pattern'). General gotcha: never write 'TypeA*/TypeB' (pointer-star immediately followed by slash) inside a /* */ C comment -- always put a space or word between the * and the /.",
+    "tags": [
+      "scheduler-core",
+      "task-7.3",
+      "capnp-schema",
+      "typemap.h",
+      "comment-syntax",
+      "build-break"
+    ],
+    "related_bugs": [],
+    "occurrences": 1,
+    "last_seen": "2026-08-04T12:35:00Z"
+  },
+  {
+    "id": "bug-1494",
+    "timestamp": "2026-08-04T00:00:00Z",
+    "error_message": "clang-format -i produced bizarre whole-file +1-space indentation drift (starting right after the top #include block, persisting to EOF) in a newly-written test_snapshot_save_load.c, with kill_sched_job() alone expanded to multiple lines while identical sibling one-liners stayed collapsed.",
+    "file": "src/scheduler/testsuite/test_snapshot_save_load.c",
+    "root_cause": "Same class of bug as bug-1493 (task 7.3): the file's own top-of-file header comment contained the literal substring '(kill_sched_*/plugin_unregister/...)' -- the pointer-star-like '*' immediately followed by '/' prematurely closed the /* */ block comment. Everything after that point was parsed as real C code by clang-format's lexer, which is what produced the indentation drift (a strong diagnostic signal, not just a formatting quirk) -- gcc itself would also have failed to compile this.",
+    "fix": "Reworded 'kill_sched_*/plugin_unregister/sched_worker_release_host_ctx/sched_resolve_job_picker)' to 'kill_sched_* family, plugin_unregister, sched_worker_release_host_ctx, sched_resolve_job_picker)', removing the accidental */ sequence. General lesson (reinforcing bug-1493): a sudden, file-wide clang-format indentation shift starting at a specific line is a strong signal to grep that file for a stray '*/' inside a comment before assuming it's a clang-format config issue -- verified via a small python scanner that walks the file tracking in-comment state and flags any '*/' encountered while not already inside a comment.",
+    "tags": [
+      "scheduler-core",
+      "task-8.1",
+      "comment-syntax",
+      "clang-format",
+      "testsuite"
+    ],
+    "related_bugs": [
+      "bug-1493"
+    ],
+    "occurrences": 1,
+    "last_seen": "2026-08-04T00:00:00Z"
+  },
+  {
+    "id": "bug-1495",
+    "timestamp": "2026-08-04T00:00:00Z",
+    "error_message": "sched_get_user()/sched_get_project() (core.c) always returned NULL for any user/project name not already present in core->users/core->projects, even though both are documented and relied upon (core_events.c JOBS_PUSH, schedobj.c make_sched_job, and task 8.1's own snapshot_load.c) as get-or-create. Surfaced as job->user/job->project being silently NULL for every job referencing a brand-new user/project, discovered while writing test_snapshot_save_load.c's cross-core equivalence test (RED phase: TEST_ASSERT_NOT_NULL(sched_get_project(&core_a, \"proj1\")) failed immediately after a successful JOBS_PUSH).",
+    "file": "src/scheduler/src/core.c",
+    "root_cause": "Both functions call key_find() then do `if (rc < 0) { return NULL; }` before falling through to the make_sched_user()/make_sched_project() create path. key_find()'s actual not-found signal is a NEGATIVE errno (-ENOENT, confirmed via direct debug instrumentation: rc=-2 for an absent key, rc=1 for a present one) -- NOT rc==0 as the surrounding code assumed. Since -ENOENT < 0, the guard incorrectly treated the EXPECTED not-found outcome as a fatal error and returned NULL before ever reaching the create fallback, making get-or-create unreachable for any genuinely new name. This is a pre-existing bug from earlier tasks (4.1/4.3), not introduced by task 8.1, but it directly breaks task 8.1's own snapshot_load.c users/projects restoration onto a freshly-initialized core (which always starts empty, so every name is 'new').",
+    "fix": "Changed the guard in both functions from `if (rc < 0)` to `if ((rc < 0) && (rc != -ENOENT))`, so only a genuine (non-ENOENT) error short-circuits before the create fallback. Two-line change per function, core.c lines ~341 and ~416. Verified: full scheduler suite 23/23 (up from 22/22 baseline, no regressions), plus the new test's own field-level assertions (users/projects created correctly, priority preserved) now pass.",
+    "tags": [
+      "scheduler-core",
+      "task-8.1",
+      "core.c",
+      "get-or-create",
+      "kvmgr",
+      "key_find",
+      "sched_get_user",
+      "sched_get_project",
+      "regression-fix"
+    ],
+    "related_bugs": [],
+    "occurrences": 1,
+    "last_seen": "2026-08-04T00:00:00Z"
+  },
+  {
+    "id": "bug-1496",
+    "timestamp": "2026-08-04T00:00:00Z",
+    "error_message": "xce_sched_core_load_snapshot() never restored core->prng_seed from the snapshot -- a freshly loaded core silently kept xce_sched_core_init()'s default (0) instead of the value xce_sched_core_save_snapshot() had captured. Caught by test_snapshot_save_load.c's field-level assertion: TEST_ASSERT_EQUAL_UINT64(0xDEADBEEFULL, core_b.prng_seed) failed with Expected 3735928559 Was 0 after a successful save+load round trip.",
+    "file": "src/scheduler/src/snapshot_load.c",
+    "root_cause": "save_snapshot() (snapshot.c) correctly writes `out->prng_seed = core->prng_seed;`, but the corresponding `core->prng_seed = in->prng_seed;` assignment was simply missing from load_snapshot()'s field-copy sequence -- an incomplete round trip for a field the function's own header comment (snapshot.c, 'PRNG seed resolution') explicitly documents as in scope (Req 5 AC 21's 'PRNG state' checklist row).",
+    "fix": "Added `core->prng_seed = in->prng_seed;` to xce_sched_core_load_snapshot() (snapshot_load.c), placed after the jobs-restoration loop and before the internal_state restoration block. Verified via the same field-level assertion, now passing.",
+    "tags": [
+      "scheduler-core",
+      "task-8.1",
+      "snapshot_load.c",
+      "prng_seed",
+      "round-trip-gap"
+    ],
+    "related_bugs": [],
+    "occurrences": 1,
+    "last_seen": "2026-08-04T00:00:00Z"
+  },
+  {
+    "id": "bug-1497",
+    "timestamp": "2026-08-04T00:00:00Z",
+    "error_message": "ASan SEGV inside dup_val() -> makeString() -> hash_string() while building test_snapshot_save_load.c's cross-core equivalence test fixtures.",
+    "file": "src/scheduler/testsuite/test_snapshot_save_load.c",
+    "root_cause": "Test-side only, not production code: three test-owned xce_val_t resval fixtures were left with `.which` at its memset-zero default. XCE_VAL_STRING happens to be enum value 0 (xce/types.h), so the zeroed `.which` was silently interpreted as \"this is a string\" -- dup_val()/makeString() then reinterpreted the fixture's raw ivalue32 int as a char* pointer and hash_string() dereferenced it, crashing.",
+    "fix": "Explicitly set `.which = XCE_VAL_NUMERIC` on all three test-owned xce_val_t fixtures in test_snapshot_save_load.c before use -- never rely on struct-zero-init to select a union discriminant when a non-zero-valued enum member (XCE_VAL_NUMERIC) is the intended default.",
+    "tags": [
+      "scheduler-core",
+      "task-8.1",
+      "test_snapshot_save_load.c",
+      "xce_val_t",
+      "zero-init-union-discriminant",
+      "test-only",
+      "asan-segv"
+    ],
+    "related_bugs": [],
+    "occurrences": 1,
+    "last_seen": "2026-08-04T00:00:00Z"
   }
-]
+]