lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(scheduler,controller): worker resource values always display as 0.00 (bug-1568)

Browse source 
xce_scheduler -i -s <replay>: "info worker <name>" showed
total=0.00 available=0.00 for every resource, even though the live
cluster (xce_test resval/workerlist) reported real non-zero values
for the same worker.

Two independent root causes, both live-traced end-to-end against a
real deployed cluster (value-level TRACE at every hop: controller
push, scheduler apply, snapshot save, snapshot load/decode):

1. src/scheduler/src/core_events.c: XCE_CORE_EVENT_WORKERS_PUSH
   called sched_add_worker() -- a plain key_add(), add-only --
   unconditionally on every push. A worker connecting triggers
   multiple WORKERS_PUSH events in quick succession (an early push
   before the controller finishes writing real capacity values,
   carrying stale/zero data, followed moments later by a push
   carrying the real values). The FIRST push always won the race;
   every push after it for the same worker name silently failed
   -EEXIST and its resource values were discarded -- not just at
   join, for the rest of the process's lifetime (unlike jobs, there
   was no "update an existing worker" event path at all). Fixed via
   a new core_upsert_worker() (mirrors the existing
   core_upsert_queue() add-or-update precedent): looks up the
   worker by name first, updates status+resvals in place if found,
   else builds+adds a new one as before.

2. src/scheduler/src/repl_info.c: repl_info_worker_one()/
   repl_info_resource_one() checked `xce_val_t.which ==
   XCE_VAL_NUMERIC` -- but XCE_VAL_NUMERIC (xce/types.h, value 1)
   is a resource-DEFINITION value-type constant
   (xce_resource_t.valtype's own enum), a completely different enum
   from xce_val_t.which's real domain, Value_u_which
   (controller.capnp.h, Value_u_fvalue=5). Every producer in this
   codebase sets xce_val_t.which to Value_u_fvalue, never
   XCE_VAL_NUMERIC, so the comparison was always false -- every
   genuinely-numeric resval displayed as 0.00 unconditionally,
   independent of fix (1). Corrected all 4 occurrences to
   Value_u_fvalue.

Both fixed via TDD (RED test before each fix, GREEN after):
test_core_events.c's test_workers_push_twice_updates_existing_
worker_in_place (two pushes, same name, different values -- both
must apply) and test_repl_breakpoints_inspection.c's
test_info_worker_reports_resource_values (a real numeric resval
must display, not fall back to 0.00).

Live-validated by reproducing the user's exact original scenario
end-to-end (deploy a real cluster, record, `xce_scheduler -i -s
<replay>`, `info worker <name>`) -- now shows slots=22.00,
maxmem=33182089216.00 (30.9GB), mem=17151664128.00 (16GB), matching
the user's own live xce_test resval/workerlist numbers exactly.

Full scheduler suite green (41/41, excluding pre-existing unrelated
bug-1560); controller suite green (1102/1104, the same pre-existing
bug-1561 pair, confirmed unrelated to this change).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-08-17 10:35:34 +00:00
parent 9f539583d3
commit aabddf6025
Signed by: rshen
SSH key fingerprint: SHA256:8Vf2pZ3h6rJa7EhGK+WaAQ9Mta/apZvteW6g9U/WKKk
9 changed files with 446 additions and 38 deletions
Download patch file Download diff file Expand all files Collapse all files

15
src/controller/dataex.c
View file

@ -73,6 +73,21 @@ int collect_resval(var resmgr, zarray_t* array, xlist_t* l)
memcpy(nv->val, src, sizeof(xce_val_t));
}
/* bug-1568 instrumentation: the actual value read out of `array`
* (w->host.resvals for a per-worker call, c->sharedinst for a
* shared-resource call) right here -- the true "about to be
* wire-packaged" point shared by BOTH push paths
* (xcesched_push_workers -> collect_resvals_workers -> this
* function, and sched_worker_change -> this function directly).
* If this shows a real non-zero fvalue but the scheduler's own
* core_build_worker TRACE (core_events.c) shows 0 for the same
* resource, the loss is in the wire encode/decode or the
* scheduler-side resolution, not here or upstream. */
if (src->which == Value_u_fvalue) {
TRACE("%s resource '%s' idx %d -> fvalue=%f", __func__, r->name,
r->idx, src->u.fvalue);
}
xlist_store(l, nv);
}

27
src/controller/sched.c
View file

@ -264,6 +264,25 @@ static int xcesched_push_workers(cluster_t* c)
workers.workers = (xce_worker_t**)wlist.data;
workers.count = workers.n_total;
/* bug-1568 instrumentation: this is the FULL initial worker sync sent
* once, at on_scheduler_join() -- distinct from sched_worker_change()'s
* per-event incremental push (this file, above), which only fires
* AFTER a scheduler is already connected. A scheduler that connects
* before this controller's own resource_sync has completed its first
* poll cycle would see 0 resvals here even though this push path
* itself is working correctly. */
if (workers.n_total > 0) {
int total_resvals = 0;
for (i = 0; i < workers.n_total; i++) {
xce_worker_t* w = workers.workers[i];
total_resvals += (w != NULL) ? w->nresvals_to_sched : 0;
}
TRACE("%s initial sync: %d worker(s), %d total resval(s)", __func__,
workers.n_total, total_resvals);
}
rc = xce_push_data_req(c->sched.ep, c->myhost, &data, check_controller, c,
100);
@ -507,6 +526,14 @@ static int sched_worker_change(int type, int situation, void* data,
if (c->sched.ep != NULL) {
switch (situation) {
case HOOK_AFTER_EVENT:
/* bug-1568 instrumentation: the controller-side count actually
* being pushed to the scheduler right now -- compare against
* core_build_worker()'s "WORKERS_PUSH carries N resval(s)"
* TRACE (scheduler/src/core_events.c) in the scheduler's own
* log for the same worker/timestamp to confirm the wire push
* itself is not where a resource value gets lost. */
TRACE("%s worker %s pushing %d resval(s) to scheduler", __func__,
w->host.name, resvalist.count);
xlist_init(&wlist, 2);
xlist_store(&wlist, w);
rc = xcesched_push_workerlist(c, &wlist);

19
src/controller/worker.c
View file

@ -97,6 +97,25 @@ static int update_val_res(cluster_t* c, worker_t* w, xce_resource_t* r,
memcpy(v, val, sizeof(*v));
/* bug-1568 instrumentation: the actual VALUE being written into
* w->host.resvals[idx] (or c->sharedinst[idx] for a shared resource)
* -- earlier instrumentation (core_events.c/snapshot_wire.c/
* snapshot_load.c/sched.c) only ever counted entries flowing through
* the pipeline, never inspected their CONTENT; this is the true
* source write, so a 0.0 logged here for e.g. 'slots'/'maxmem' means
* the zero is genuine at the point of writing, not lost downstream. */
if (val->which == Value_u_fvalue) {
TRACE("%s resource '%s' worker '%s' idx %d -> fvalue=%f", __func__,
r->name, (r->flags & XCE_RESF_SHARED) ? "(shared)"
: w->host.name,
idx, val->u.fvalue);
} else {
TRACE("%s resource '%s' worker '%s' idx %d -> which=%d (non-numeric)",
__func__, r->name,
(r->flags & XCE_RESF_SHARED) ? "(shared)" : w->host.name, idx,
val->which);
}
return 0;
}

166
src/scheduler/src/core_events.c
View file

@ -275,11 +275,74 @@ static void core_apply_config_queues(xce_sched_core_t* core,
}
}
/*
* core_apply_worker_resvals - resolves w->resvals_to_sched[] against
* core->resmgr and applies each into `worker`'s resvals zarray. Shared by
* core_build_worker() (fresh worker, `worker->resvals` just zarray_init'd
* and empty) and core_upsert_worker()'s update-in-place branch (`worker`
* already has entries from a prior push) -- safe for both without a
* separate wipe step, since zarray_put() already frees any existing value
* at a given index (via the zarray's own killer) before overwriting it
* (zarray.c).
*/
static void core_apply_worker_resvals(xce_sched_core_t* core,
xce_sched_worker_t* worker,
xce_worker_t* w)
{
int i;
/* bug-1568 instrumentation: how many resvals this push actually
* carried for this worker, before any per-entry resolution below --
* a live recording session's logs pin down whether a worker resource
* value ever left the controller's push (this line) vs got dropped
* resolving against this scheduler's own resmgr (the applied-count
* TRACE below). */
TRACE("%s worker '%s' WORKERS_PUSH carries %d resval(s)", __func__,
w->name ? w->name : "(null)", w->nresvals_to_sched);
for (i = 0; i < w->nresvals_to_sched; i++) {
xce_name_val_t* nv = w->resvals_to_sched[i];
xce_resource_t* r = NULL;
xce_val_t* val = NULL;
if ((nv == NULL) || (nv->name == NULL)) {
continue;
}
if ((key_find(core->resmgr, (void*)nv->name, (void**)&r) < 0)
|| (r == NULL)) {
WARN("%s unknown resource '%s' in worker resvals, skipping",
__func__, nv->name);
continue;
}
/* bug-1568 instrumentation: the value as RECEIVED over the wire,
* right before it's applied -- compare against
* src/controller/dataex.c's collect_resval() TRACE (the matching
* "about to be wire-packaged" point) for the same resource/
* worker/timestamp to isolate the wire encode/decode step. */
if (nv->val != NULL && nv->val->which == Value_u_fvalue) {
TRACE("%s worker '%s' resource '%s' received fvalue=%f",
__func__, w->name ? w->name : "(null)", nv->name,
nv->val->u.fvalue);
}
MOVE_PTR(val, nv->val);
zarray_put(&(worker->resvals), r->idx, val);
}
/* bug-1568 instrumentation: how many actually landed in
* worker->resvals -- compare against the carried-count TRACE above to
* spot a resolution-time drop (unknown-resource WARN above) vs a
* clean pass-through. */
TRACE("%s worker '%s' applied %d resval(s) to worker->resvals",
__func__, w->name ? w->name : "(null)", worker->resvals.count);
}
static xce_sched_worker_t* core_build_worker(xce_sched_core_t* core,
xce_worker_t* w)
{
xce_sched_worker_t* worker;
int i;
if ((core == NULL) || (w == NULL)) {
return NULL;
@ -299,29 +362,71 @@ static xce_sched_worker_t* core_build_worker(xce_sched_core_t* core,
zarray_init(&(worker->resInUse), 128,
(zarray_item_killer_t)kill_sched_resval);
for (i = 0; i < w->nresvals_to_sched; i++) {
xce_name_val_t* nv = w->resvals_to_sched[i];
xce_resource_t* r = NULL;
xce_val_t* val = NULL;
if ((nv == NULL) || (nv->name == NULL)) {
continue;
}
if ((key_find(core->resmgr, (void*)nv->name, (void**)&r) < 0)
|| (r == NULL)) {
WARN("%s unknown resource '%s' in worker resvals, skipping",
__func__, nv->name);
continue;
}
MOVE_PTR(val, nv->val);
zarray_put(&(worker->resvals), r->idx, val);
}
core_apply_worker_resvals(core, worker, w);
return worker;
}
/*
* core_upsert_worker - shared add-or-update logic for one xce_worker_t
* wire payload (WORKERS_PUSH), mirrors core_upsert_queue()'s add-or-update
* shape (lookup-then-branch; WORKERS_PUSH itself has no picker hook, Req 1
* AC 2, so unlike core_upsert_queue() this never calls into the picker).
*
* bug-1568: a worker connecting triggers multiple WORKERS_PUSH events in
* quick succession (live-confirmed against a real deploy: an early push
* before the controller finished writing the worker's real capacity
* values, carrying stale/zero data, followed moments later by a push
* carrying the real values). WORKERS_PUSH previously always built a
* brand-new xce_sched_worker_t (core_build_worker()) and called
* sched_add_worker() -- a plain key_add(), add-only -- unconditionally,
* so every push after the first for an already-registered worker name
* silently failed -EEXIST and its resource values were discarded;
* core->workers[name] then never advanced past whatever the FIRST push
* happened to carry, for the rest of the process's lifetime (there was no
* "update an existing worker" path at all, unlike jobs, which have
* XCE_CORE_EVENT_JOB_STATE_CHANGE/JOB_UPDATE for in-place mutation of an
* already-added job).
*
* Looks up `w->name` in core->workers; if found, updates status + resvals
* in place via core_apply_worker_resvals(); otherwise builds+adds a new
* one via the original core_build_worker()/sched_add_worker() path.
*/
static void core_upsert_worker(xce_sched_core_t* core, xce_worker_t* w)
{
xce_sched_worker_t* worker;
if ((w == NULL) || (w->name == NULL)) {
WARN("%s NULL worker payload", __func__);
return;
}
worker = sched_get_worker(core, w->name);
if (worker == NULL) {
int add_rc;
worker = core_build_worker(core, w);
if (worker == NULL) {
WARN("%s failed to build new worker '%s'", __func__, w->name);
return;
}
add_rc = sched_add_worker(core, worker);
if (add_rc < 0) {
/* Should not happen -- sched_get_worker() just confirmed this
* name doesn't exist yet, and xce_core_push_events() has no
* concurrent callers for the same core (single drain point,
* driver.c). Logged defensively anyway, matching this file's
* "log and continue" discipline for the rest of the batch. */
WARN("%s sched_add_worker('%s') unexpectedly failed (rc=%d)",
__func__, worker->name ? worker->name : "(null)", add_rc);
}
} else {
worker->status = w->status;
core_apply_worker_resvals(core, worker, w);
}
}
/*
* xce_core_push_events - see core.h's doc comment for the full contract
* (Req 5 AC 4 / AC 8). Per event type:
@ -342,8 +447,13 @@ static xce_sched_worker_t* core_build_worker(xce_sched_core_t* core,
* on_config(picker, config) if the hook is non-NULL.
* JOBS_PUSH core_build_job() + sched_add_job() per job, then
* on_job_push(picker, job) once PER JOB in the batch.
* WORKERS_PUSH core_build_worker() + sched_add_worker() per worker.
* No picker hook (Req 1 AC 2: worker updates are not
* WORKERS_PUSH core_upsert_worker() per worker: sched_get_worker()
* lookup by name; update status+resvals in place if
* found (bug-1568 fix -- a worker connecting sends
* multiple WORKERS_PUSH events, so an add-only path
* here silently dropped every push after the first),
* else core_build_worker() + sched_add_worker(). No
* picker hook (Req 1 AC 2: worker updates are not
* forwarded to the picker).
* RESOURCES_PUSH makeResource()/key_find()-update-in-place per
* resource (mirrors scheduler.c's sched_update()
@ -464,17 +574,7 @@ int xce_core_push_events(xce_sched_core_t* core, xce_core_event_t* events,
int w;
for (w = 0; w < ev->u.workers_push.nworkers; w++) {
xce_sched_worker_t* worker
= core_build_worker(core, ev->u.workers_push.workers[w]);
if (worker == NULL) {
WARN("%s WORKERS_PUSH: failed to build worker (batch "
"index %d)",
__func__, w);
continue;
}
sched_add_worker(core, worker);
core_upsert_worker(core, ev->u.workers_push.workers[w]);
}
break;
}

16
src/scheduler/src/repl_info.c
View file

@ -2,6 +2,7 @@
#include <stdlib.h>
#include <string.h>
#include "controller.capnp.h"
#include "repl.h"
#include "repl_internal.h"
#include "resreq.h"
@ -320,8 +321,13 @@ static void repl_info_worker_one(xce_sched_sim_driver_t* sim,
if ((avail_v == NULL) || (rname == NULL)) {
continue;
}
avail = (avail_v->which == XCE_VAL_NUMERIC) ? avail_v->u.fvalue : 0.0;
used = ((used_v != NULL) && (used_v->which == XCE_VAL_NUMERIC))
/* bug-1568: xce_val_t.which holds a Value_u_which tag
* (controller.capnp.h), NOT an XCE_VAL_* resource-definition
* value-type constant (xce/types.h, a different enum for
* xce_resource_t.valtype) -- comparing against the wrong one here
* made every genuinely-numeric resval silently fall back to 0.0. */
avail = (avail_v->which == Value_u_fvalue) ? avail_v->u.fvalue : 0.0;
used = ((used_v != NULL) && (used_v->which == Value_u_fvalue))
? used_v->u.fvalue
: 0.0;
fprintf(stdout, "[sim] resource %s: total=%.2f available=%.2f\n", rname,
@ -427,11 +433,13 @@ static void repl_info_resource_one(xce_sched_sim_driver_t* sim,
}
avail_v = (xce_val_t*)zarray_get(&(w->resvals), r->idx);
used_v = (xce_val_t*)zarray_get(&(w->resInUse), r->idx);
if ((avail_v != NULL) && (avail_v->which == XCE_VAL_NUMERIC)) {
/* bug-1568: same Value_u_fvalue vs XCE_VAL_NUMERIC type-tag
* mismatch as repl_info_worker_one() above. */
if ((avail_v != NULL) && (avail_v->which == Value_u_fvalue)) {
available += avail_v->u.fvalue;
total += avail_v->u.fvalue;
}
if ((used_v != NULL) && (used_v->which == XCE_VAL_NUMERIC)) {
if ((used_v != NULL) && (used_v->which == Value_u_fvalue)) {
total += used_v->u.fvalue;
}
}

38
src/scheduler/src/snapshot_load.c
View file

@ -203,10 +203,46 @@ int xce_sched_core_load_snapshot(xce_sched_core_t* core,
continue;
}
/* bug-1568 instrumentation: the value as decoded from the
* .replay file, right before it's applied -- compare
* against snapshot_wire.c's build_wire_worker() TRACE (the
* matching "about to be serialized" point) for the same
* resource/worker to isolate the capnp
* encode/write/read/decode round-trip specifically. */
if (nv->val != NULL && nv->val->which == Value_u_fvalue) {
TRACE("%s worker '%s' resource '%s' decoded fvalue=%f",
__func__, w->name, nv->name, nv->val->u.fvalue);
}
zarray_put(&(w->resvals), r->idx, dup_val(nv->val));
}
sched_add_worker(core, w);
/* bug-1568 instrumentation: the snapshot's own recorded count
* (ww->nresvals_to_sched, what build_wire_worker() captured at
* save time -- see that function's matching TRACE,
* snapshot_wire.c) vs. how many actually landed in
* w->resvals here -- a gap means resources this replay file
* genuinely recorded were dropped resolving against this
* (possibly different) core's resmgr, not lost earlier at
* save time. */
TRACE("%s worker '%s' snapshot-restored %d/%d resval(s)",
__func__, w->name, w->resvals.count,
ww->nresvals_to_sched);
{
int add_rc = sched_add_worker(core, w);
/* bug-1568 instrumentation: same unchecked-return concern
* as core_events.c's WORKERS_PUSH handler -- a negative
* rc here means this worker (with whatever resvals were
* just restored above) was silently discarded instead of
* becoming part of core->workers. */
if (add_rc < 0) {
WARN("%s worker '%s' sched_add_worker failed (rc=%d) "
"-- this worker's restored state was discarded",
__func__, w->name, add_rc);
}
}
}
}

26
src/scheduler/src/snapshot_wire.c
View file

@ -229,6 +229,21 @@ xce_worker_t* build_wire_worker(xce_sched_worker_t* w,
rname = find_resource_name_by_idx(resources, nresources, i);
if (rname == NULL) {
/* bug-1568: unlike every other analogous skip in this
* codebase (core_events.c's core_build_worker "unknown
* resource" WARN, snapshot_load.c's own load-side WARN), this
* skip was previously silent -- a worker resval whose idx no
* longer resolves against `resources` (this snapshot's own
* fresh walk of core->resmgr, taken moments before this
* call, xce_sched_core_save_snapshot()) is dropped from the
* snapshot with zero diagnostic trace, which is exactly how
* a real resource value can vanish between "the scheduler
* genuinely holds it" (core_events.c's own TRACE) and "the
* .replay file captured it" with no visible cause. */
WARN("%s worker '%s' resval at idx %d has no matching "
"resource in core->resmgr, skipping (snapshot will be "
"missing this value)",
__func__, (w->name != NULL) ? w->name : "(null)", i);
continue;
}
@ -254,5 +269,16 @@ xce_worker_t* build_wire_worker(xce_sched_worker_t* w,
/* worker->resInUse: not captured -- see snapshot.c header
* "internal_state contents" (session-transient, always recomputed). */
/* bug-1568 instrumentation: the scheduler's own LIVE w->resvals count
* (source of truth at the instant `record start` fires) vs. how many
* actually made it into this snapshot's resvals_to_sched -- a gap
* between the two (n < w->resvals.count) means resources this
* scheduler process genuinely held were dropped resolving against
* `resources`/`nresources` right here (unknown-idx skip above), not
* lost earlier in the controller->scheduler push or later in
* snapshot load. */
TRACE("%s worker '%s' snapshot-captured %d/%d resval(s)", __func__,
(w->name != NULL) ? w->name : "(null)", n, w->resvals.count);
return ww;
}

97
src/scheduler/testsuite/test_core_events.c
View file

@ -515,6 +515,102 @@ static void test_non_picker_events_mutate_state_but_never_call_hooks(void)
TEST_ASSERT_NOT_NULL(sched_get_worker(&core, "hostA"));
}
/*
* bug-1568 regression: a worker connecting triggers multiple WORKERS_PUSH
* events in quick succession (live-confirmed against a real deploy -- an
* early push before the controller finishes writing a worker's real
* capacity values, carrying stale/zero data, followed shortly after by a
* push carrying the real values). Before this fix, WORKERS_PUSH always
* built a brand-new xce_sched_worker_t and called sched_add_worker() (a
* plain key_add(), add-only) unconditionally -- so the SECOND push for an
* already-registered worker name silently failed -EEXIST and its resource
* values were discarded, leaving core->workers[name] stuck at whatever the
* FIRST push happened to carry, forever. Two WORKERS_PUSH events for the
* SAME worker name, with DIFFERENT resource values, must both be applied
* -- the worker's resvals end up at the SECOND push's values, not the
* first, and no duplicate/orphaned worker object is created.
*/
static void test_workers_push_twice_updates_existing_worker_in_place(void)
{
xce_sched_core_t core;
xce_core_event_t event;
xce_resource_t wire_resource;
xce_worker_t wire_worker;
xce_worker_t* worker_ptrs[1];
xce_val_t val1;
xce_name_val_t nv1;
xce_name_val_t* nv1_ptrs[1];
xce_val_t val2;
xce_name_val_t nv2;
xce_name_val_t* nv2_ptrs[1];
xce_sched_worker_t* w;
xce_resource_t* r = NULL;
xce_val_t* applied;
int rc;
fixture_init(&core, &g_full_plugin);
memset(&wire_resource, 0, sizeof(wire_resource));
wire_resource.name = "mem";
TEST_ASSERT_NOT_NULL(makeResource(core.resmgr, &wire_resource));
TEST_ASSERT_TRUE(key_find(core.resmgr, (void*)"mem", (void**)&r) >= 0);
TEST_ASSERT_NOT_NULL(r);
/* First push: worker "hostA" joins with mem=100.0. */
memset(&val1, 0, sizeof(val1));
val1.which = Value_u_fvalue;
val1.u.fvalue = 100.0;
memset(&nv1, 0, sizeof(nv1));
nv1.name = "mem";
nv1.val = &val1;
nv1_ptrs[0] = &nv1;
memset(&wire_worker, 0, sizeof(wire_worker));
wire_worker.name = "hostA";
wire_worker.status = 0;
wire_worker.nresvals_to_sched = 1;
wire_worker.resvals_to_sched = nv1_ptrs;
worker_ptrs[0] = &wire_worker;
memset(&event, 0, sizeof(event));
event.type = XCE_CORE_EVENT_WORKERS_PUSH;
event.u.workers_push.nworkers = 1;
event.u.workers_push.workers = worker_ptrs;
rc = xce_core_push_events(&core, &event, 1);
TEST_ASSERT_EQUAL_INT(0, rc);
w = sched_get_worker(&core, "hostA");
TEST_ASSERT_NOT_NULL(w);
applied = (xce_val_t*)zarray_get(&(w->resvals), r->idx);
TEST_ASSERT_NOT_NULL(applied);
TEST_ASSERT_EQUAL_FLOAT(100.0f, (float)applied->u.fvalue);
/* Second push: SAME worker name, mem now really 200.0 (the "arrived
* late with the real value" push bug-1568 traced live). */
memset(&val2, 0, sizeof(val2));
val2.which = Value_u_fvalue;
val2.u.fvalue = 200.0;
memset(&nv2, 0, sizeof(nv2));
nv2.name = "mem";
nv2.val = &val2;
nv2_ptrs[0] = &nv2;
wire_worker.resvals_to_sched = nv2_ptrs;
rc = xce_core_push_events(&core, &event, 1);
TEST_ASSERT_EQUAL_INT(0, rc);
/* Still exactly one "hostA" worker -- updated in place, not a second,
* silently-discarded object leaving the first one stale. */
w = sched_get_worker(&core, "hostA");
TEST_ASSERT_NOT_NULL(w);
applied = (xce_val_t*)zarray_get(&(w->resvals), r->idx);
TEST_ASSERT_NOT_NULL(applied);
TEST_ASSERT_EQUAL_FLOAT(200.0f, (float)applied->u.fvalue);
}
/* JOB_RUSAGE's on_job_rusage hook is OPTIONAL (Req 5 AC 4) -- a picker
* without it must still apply the state mutation and must not error or
* crash. */
@ -704,6 +800,7 @@ int main(void)
RUN_TEST(test_push_events_fires_each_hook_once_per_event_with_picker_idle);
RUN_TEST(test_jobs_push_fires_on_job_push_once_per_job);
RUN_TEST(test_non_picker_events_mutate_state_but_never_call_hooks);
RUN_TEST(test_workers_push_twice_updates_existing_worker_in_place);
RUN_TEST(test_job_rusage_applies_state_when_hook_is_null);
RUN_TEST(test_config_push_decomposes_queues_resources_and_resvals);
RUN_TEST(test_push_events_rejects_with_ebusy_when_session_active_no_mutation);

80
src/scheduler/testsuite/test_repl_breakpoints_inspection.c
View file

@ -657,6 +657,85 @@ static void test_info_queue_reports_status_and_queued_jobs(void)
printf("test_info_queue_reports_status_and_queued_jobs: PASS\n");
}
/*
* bug-1568 (display-layer piece): repl_info_worker_one()/
* repl_info_resource_one() (repl_info.c) checked
* `avail_v->which == XCE_VAL_NUMERIC` to decide whether a worker's
* xce_val_t resval held a real number -- but XCE_VAL_NUMERIC (xce/types.h,
* value 1) is a RESOURCE-DEFINITION value-type constant (xce_resource_t.
* valtype), a completely different enum from xce_val_t.which's own real
* domain, Value_u_which (controller.capnp.h) -- every producer in this
* codebase (update_val_res/core_apply_worker_resvals/snapshot_load.c/etc.)
* sets xce_val_t.which to Value_u_fvalue (value 5), never XCE_VAL_NUMERIC.
* The comparison was therefore ALWAYS false for a genuinely-numeric
* resval, so `info worker`/`info resource` printed total=0.00/
* available=0.00 for every worker resource, unconditionally -- live-
* confirmed against a real deployed cluster: the value survived the full
* controller-push -> scheduler-apply -> snapshot-save -> snapshot-load
* round trip completely intact (traced at every hop), yet still displayed
* as 0.00, isolating the bug to this display-layer type-tag mismatch
* alone.
*/
static void test_info_worker_reports_resource_values(void)
{
xce_sched_sim_driver_t sim;
xce_core_event_t events[2];
xce_resource_t wire_resource;
xce_worker_t wire_worker;
xce_worker_t* worker_ptrs[1];
xce_val_t val;
xce_name_val_t nv;
xce_name_val_t* nv_ptrs[1];
char* captured;
int rc;
memset(&sim, 0, sizeof(sim));
rc = xce_sched_core_init(&(sim.core));
TEST_ASSERT_EQUAL_INT(0, rc);
memset(&wire_resource, 0, sizeof(wire_resource));
wire_resource.name = "mem";
TEST_ASSERT_NOT_NULL(makeResource(sim.core.resmgr, &wire_resource));
memset(&val, 0, sizeof(val));
val.which = Value_u_fvalue;
val.u.fvalue = 42.5;
memset(&nv, 0, sizeof(nv));
nv.name = "mem";
nv.val = &val;
nv_ptrs[0] = &nv;
memset(&wire_worker, 0, sizeof(wire_worker));
wire_worker.name = "hostA";
wire_worker.status = 0;
wire_worker.nresvals_to_sched = 1;
wire_worker.resvals_to_sched = nv_ptrs;
worker_ptrs[0] = &wire_worker;
memset(events, 0, sizeof(events));
events[0].type = XCE_CORE_EVENT_WORKERS_PUSH;
events[0].u.workers_push.nworkers = 1;
events[0].u.workers_push.workers = worker_ptrs;
rc = xce_core_push_events(&(sim.core), events, 1);
TEST_ASSERT_EQUAL_INT(0, rc);
captured = capture_dispatch(&sim, "info worker hostA");
TEST_ASSERT_NOT_NULL_MESSAGE(
strstr(captured, "resource mem: total=42.50 available=42.50"),
"a worker's real numeric resource value must be displayed, not "
"the XCE_VAL_NUMERIC/Value_u_fvalue type-tag mismatch's 0.00 "
"(bug-1568)");
TEST_ASSERT_NULL_MESSAGE(strstr(captured, "total=0.00 available=0.00"),
"must not fall back to the zero default for a "
"genuinely-set numeric value");
free(captured);
printf("test_info_worker_reports_resource_values: PASS\n");
}
static void test_info_summary_reports_counts(void)
{
xce_sched_sim_driver_t sim;
@ -1055,6 +1134,7 @@ int main(void)
RUN_TEST(test_info_job_reports_state_queue_priority);
RUN_TEST(test_info_job_unknown_reports_not_found);
RUN_TEST(test_info_queue_reports_status_and_queued_jobs);
RUN_TEST(test_info_worker_reports_resource_values);
RUN_TEST(test_info_summary_reports_counts);
RUN_TEST(test_info_session_reports_position_after_synthetic_events);