The Task 9 tests were marked [x] but were stub/absent. Convert the
correctness-critical ones to real white-box tests, registered as meson tests
under --suite scheduler (leaks-off per convention):
- test_handlers.c (new) — #include ../src/handler.c; covers 9.2/9.3: each
notify/dataset handler enqueues the correct op / skips NULL-empty / -EINVAL.
Asserts the 5.6 fix (update_job -> UPDATE_JOB_FIELDS, not bulk UPDATE_JOB).
- test_job_update.c — add 9.5 (real sched_update_existing_queue, replacing the
former local stub); already covers 9.4 (job) and 9.7 (rusage).
- meson.build — build + register all three scheduler test executables.
Suite GREEN: sched_test 15/15, sched_job_update_test 10/10,
sched_handlers_test 6/6; ASAN-clean.
Honest remaining: 9.6 dequeue worker-resvals APPLY is PARTIAL (enqueue tested;
apply is a dormant path, R-3); 9.8 is a documented compile-time absence check.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
5.5 (Req 6 / G-5): the JOB_RUSAGE notify path called the plugin but never
stored the usage data — xce_sched_job_t had no rusage field (UB-6.1 unmet).
- Add xce_job_rusages_t* rusages to xce_sched_job_t.
- New sched_apply_job_rusage() stores the report ON the job (ownership
transferred), replacing any prior via free_JobRusages_ptr; -ENOENT unknown,
-EINVAL NULL. Dequeue NULLs the request pointer on success; kill_sched_job
releases job->rusages.
5.8 (G-3): sched_pend_update_killer left five new-op payloads on empty
break cases — a per-message leak since ownership is transferred on enqueue.
- Free each with the exact generated deep-freer: free_QueueConf_ptr,
free_Resource_ptr[], free_ResourceVal_ptr[], free_ResValList_ptr,
free_JobRusages_ptr. Safe: handlers nullify on enqueue, makeResource copies
via makeInstance, dequeue copies values (no aliasing).
Tests: sched_job_update_test 7/7 (3 update + 4 rusage), regression
sched_test 15/15, ASAN clean (no bad-free/double-free).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The notify schedUpdateJob path shared XCE_SCHED_OP_UPDATE_JOB with the
dataset bulk-load, so the dequeue routed both to sched_update_jobs()
(create/replace) — a notify update to an existing job corrupted/duplicated
live job state and never returned -ENOENT (Req 3 UB-3.1/UB-3.2 unmet;
sched_update_existing_job was dead code copying only state).
- Add distinct op XCE_SCHED_OP_UPDATE_JOB_FIELDS (+ jobfieldsupdate union),
enqueued only by the notify handler, dequeued to sched_update_existing_job.
- Move sched_update_existing_job to schedobj.c; update all mutable fields
(state, priority, queue, resreq) in place; -ENOENT unknown, -EINVAL NULL.
- Dataset bulk-create path unchanged. Killer frees the new payload.
- New white-box test sched_job_update_test (3/3); regression sched_test 15/15.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire up 6 previously-stubbed message handlers in the scheduler:
resource dataset, resval dataset, job update notify, queue update
notify, resval update notify, and job rusage notify. Remove the
unreachable pushmsg branch (vtask_entry rewrites all pushmsg to
datamsg before the handler fires).
Changes by component:
- helper.h/helper.c: add xce_notify_event_name() for symbolic log names
- scheduler.h: add 5 new xce_sched_op_t values and matching union fields
- plugin.h/plugin.c: add plugin_job_rusage_hook_t and plugin_job_rusage_update()
- schedobj.c: add sched_update_existing_queue() with NULL/-ENOENT guards
- scheduler.c: wire 5 new dequeue cases; add sched_update_existing_job()
and static sched_op_name(); emit WARN log on handler failure
- handler.c: implement sched_handle_resources/resvals (dataset) and
sched_handle_sched_update_{job,queue,resvals}/job_rusage (notify);
remove dead sched_handle_jobpush/workerpush and pushmsg branch;
update TRACE logs to use symbolic name helpers
- test.c: 5 new unit tests covering notify event names, enum values,
and queue update NULL guard (15/15 pass)
Three identical static copies (controller, worker, scheduler) replaced
with a single xce_parse_peers() in src/lib/cluster.c.
The libxce version is the canonical implementation: stores the logical
name from 'name=host:port' syntax so daemons on the same dev host can
identify themselves by name rather than IP address.
Worker (xcecd):
- Add xce_cluster_config_t cluster_cfg field to controller_connector_ctx_t
(controller_connector.h includes xce/cluster.h)
- Populate cc_ctx->cluster_cfg from parsed cluster_cfg after xcecd_start()
sets cc_ctx (main.c, removes the P3 placeholder)
- Replace xcecd_join_cluster() single-shot connect with a candidate-iterating
loop: tries each candidate, calls xcecli_stop() before the next attempt,
falls back to the original single-host path when candidates_count == 0
Scheduler (xce_scheduler):
- Add xce_cluster_config_t cluster_cfg field to xce_scheduler_t
(scheduler.h includes xce/cluster.h)
- Add sched_set_cluster_cfg() setter
- Replace the hardcoded xcecli_start/xcecli_join pair in sched_init() with
the same candidate-iterating pattern; original single-host path preserved
- main.c calls sched_set_cluster_cfg() before sched_init() when --peers given
(removes the P3 placeholder comment)
checking_connection() already calls xcecd_join_cluster() on retry, giving
the worker free reconnect-after-failover behaviour for no additional code.
Worker (P2.5-5):
- Added parse_peers() static helper to src/worker/src/main.c
- Switched getopt to getopt_long to support --peers and --ca-cert long options
- When --peers is given: parses into xce_cluster_config_t, calls
xce_cluster_config_validate(), logs INFO with candidate count/identity/ca_cert
- If validation fails: prints clear error and exits(1)
- WARN logged when candidates_count > 3
- Backward compatible: no --peers falls back to existing single-controller path
Scheduler (P2.5-6):
- Added parse_peers() static helper to src/scheduler/src/main.c
- Extended argp struct with --peers (-P) and --ca-cert (-C) options
- Same validation/logging pattern as worker
- Backward compatible: no --peers falls back to existing single-controller path
Tests (new):
- src/lib/testsuite/test_cluster_parse.c: 8 unit tests for parse_peers logic
(single entry, multiple entries, no-name format, empty/null/missing-port errors,
name-stripped correctly, MAX_HA_CANDIDATES cap respected)
- Registered in test.c and meson.build
- All 8 tests GREEN; 3 pre-existing failures in unrelated GC/CSE tests unchanged
Two correctness fixes found during final Phase 6 code review:
F1 (CRITICAL): GC permanently disabled — staging_merge_complete bootstrapping deadlock.
sched_begin_session reset staging_merge_complete to 0 but never set it back to 1,
so sched_end_session's guard (!staging_merge_complete) always fired, preventing GC
from ever running. select-node memory leaked indefinitely across scheduling sessions.
Fix: set staging_merge_complete=1 at the end of sched_begin_session after all
staging work (sched_account_jobs etc.) completes.
F2 (HIGH): recompact_node_ids() omitted from GC protocol in sched_end_session.
The protocol in resreq.h requires gc_sweep → recompact_node_ids. Without
recompaction, sel_node_next_id grows monotonically across sessions and
select_eval_ctx_t arrays (sized by max node_id) grow without bound.
Fix: call recompact_node_ids() after gc_sweep_select_nodes.
Also adds test_staging_merge_complete_gc_guard to document the bootstrap
invariant: sched_begin_session must leave staging_merge_complete=1 so the
GC guard in sched_end_session does not fire.
- simple_num/simple_str: guard h->numvals/strvals NULL and idx out-of-range before array access
- eval_select_simple SEL_RESNAME: guard host->numvals NULL and idx bounds before deref
- eval_select_inner SEL_RESNAME (memoizing path): same NULL/bounds guards for numvals and strvals
- build_host_eval_ctx: return valid empty context (num_res=0, NULL arrays) instead of NULL when num_res<=0
- sched_meet_host_selection: remove empty_ctx stack hack; always call build_host_eval_ctx which now handles the zero-resource case cleanly
Replace the always-return-1 stub with a real implementation that:
- Returns 1 immediately when s == NULL or s->root == NULL (no constraint)
- Builds a host_eval_ctx_t from w->resvals and calls eval_select_simple
- Uses a zero-resource stack context when resvals.caps == 0 so constant
sub-expressions (SEL_TRUE, SEL_FALSE, SEL_ANY) still evaluate correctly
- On OOM (build_host_eval_ctx returns NULL), conservatively passes the host
- Maps eval_select_simple result: >0 → 1 (match), <=0 → 0 (no-match)
Add #define UNIT_TEST 1 to test.c, remove redundant #ifdef guard, and
add three new TDD tests covering NULL select, SEL_TRUE, and SEL_FALSE.
All 6 tests pass.
- Fix issue to xce_get() api cause client side crash
- add parse resreq section
- add rusage parser
- parse resreq with basic support alternative & compound, still miss select
