Append-only UInt64 epoch-millis field for the responder snapshot timestamp
(Req 11.1, wire-format setup). Hand-authored xce_ctrlstatus_t typemap gains a
matching responder_time_ms member; codec round-trips it. Controller suite 1/1 OK.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
EARS requirements for a controller status query (XCE lsid analog): primary identity
(unknown when unelected), answered-by node+role (distinct from cluster view),
configured peers+count, committed config version + content checksum (catch same-version
divergence), term/commit-apply/fenced/cluster-name, failover-aware delivery, test-client
command, and wait-primary switching its probe to it. C tier only.
Conflation fix: gate every kill + durability assertion on cluster health (3 nodes
up + serving + rejoin settle); assert_no_loss probes serveability first and reports
INCONCLUSIVE (not DATA LOSS) when unserveable; confirm-settle re-check distinguishes
catch-up lag from persistent loss; post-kill election SLA configurable (60s default).
DATA LOSS now reported ONLY for a job persistently absent from a SERVING cluster.
The Task 9 tests were marked [x] but were stub/absent. Convert the
correctness-critical ones to real white-box tests, registered as meson tests
under --suite scheduler (leaks-off per convention):
- test_handlers.c (new) — #include ../src/handler.c; covers 9.2/9.3: each
notify/dataset handler enqueues the correct op / skips NULL-empty / -EINVAL.
Asserts the 5.6 fix (update_job -> UPDATE_JOB_FIELDS, not bulk UPDATE_JOB).
- test_job_update.c — add 9.5 (real sched_update_existing_queue, replacing the
former local stub); already covers 9.4 (job) and 9.7 (rusage).
- meson.build — build + register all three scheduler test executables.
Suite GREEN: sched_test 15/15, sched_job_update_test 10/10,
sched_handlers_test 6/6; ASAN-clean.
Honest remaining: 9.6 dequeue worker-resvals APPLY is PARTIAL (enqueue tested;
apply is a dormant path, R-3); 9.8 is a documented compile-time absence check.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The spec/design placed the scheduler-op name lookup as xce_sched_op_name in
lib/helper, but xce_sched_op_t is a scheduler-only type and src/lib must not
depend on scheduler headers — so it is correctly a static sched_op_name() in
scheduler.c. Code kept as-is (correct); docs amended to match:
- Task 1 intro + 1.1/1.3 + 5.7 + 9.1: scheduler-layer placement + layering note.
- design.md Component 8/9, file table, risks: same reconciliation.
Only xce_notify_event_name (int-typed, lib-level) lives in lib/helper.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
5.5 (Req 6 / G-5): the JOB_RUSAGE notify path called the plugin but never
stored the usage data — xce_sched_job_t had no rusage field (UB-6.1 unmet).
- Add xce_job_rusages_t* rusages to xce_sched_job_t.
- New sched_apply_job_rusage() stores the report ON the job (ownership
transferred), replacing any prior via free_JobRusages_ptr; -ENOENT unknown,
-EINVAL NULL. Dequeue NULLs the request pointer on success; kill_sched_job
releases job->rusages.
5.8 (G-3): sched_pend_update_killer left five new-op payloads on empty
break cases — a per-message leak since ownership is transferred on enqueue.
- Free each with the exact generated deep-freer: free_QueueConf_ptr,
free_Resource_ptr[], free_ResourceVal_ptr[], free_ResValList_ptr,
free_JobRusages_ptr. Safe: handlers nullify on enqueue, makeResource copies
via makeInstance, dequeue copies values (no aliasing).
Tests: sched_job_update_test 7/7 (3 update + 4 rusage), regression
sched_test 15/15, ASAN clean (no bad-free/double-free).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The notify schedUpdateJob path shared XCE_SCHED_OP_UPDATE_JOB with the
dataset bulk-load, so the dequeue routed both to sched_update_jobs()
(create/replace) — a notify update to an existing job corrupted/duplicated
live job state and never returned -ENOENT (Req 3 UB-3.1/UB-3.2 unmet;
sched_update_existing_job was dead code copying only state).
- Add distinct op XCE_SCHED_OP_UPDATE_JOB_FIELDS (+ jobfieldsupdate union),
enqueued only by the notify handler, dequeued to sched_update_existing_job.
- Move sched_update_existing_job to schedobj.c; update all mutable fields
(state, priority, queue, resreq) in place; -ENOENT unknown, -EINVAL NULL.
- Dataset bulk-create path unchanged. Killer frees the new payload.
- New white-box test sched_job_update_test (3/3); regression sched_test 15/15.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per design invariant (a controller exits only on crash or admin kill), replace the
self-abort recovery terminal (tasks 3.3/4.3/4.4) with a loud, permanent, non-promotable
fence that stays alive and keeps retrying catch-up (self-heals on peer return; no manual
admin recovery). Records the never-self-terminate invariant as project-wide guidance in
src/controller/CLAUDE.md + .kiro/steering/ha-async-io.md.
