Implementation Tasks: resval-id-decoupling

Generated: 2026-05-12T07:45:00-07:00 Feature: resval-id-decoupling Status: Tasks Generated

Task Overview

7 major tasks, 19 sub-tasks. All 6 requirements covered.

Dependencies: Task 1 must complete first (headers + schema unblock everything). Tasks 2 and 3 can proceed in parallel after Task 1. Tasks 4, 5, 6 can proceed in parallel after Tasks 2 and 3. Task 7 last.

Tasks

1. Update shared headers and capnp schema

Change the wire type and C struct types that carry resource values across subsystem boundaries. This is the structural foundation — all downstream tasks depend on these types compiling correctly.

1.1 In src/schema/controller.capnp, change Worker.resvals @5 and Conf4Sched.resvals @6 from List(ValueWithIdx) to List(NameValue). Preserve field ordinal numbers; only the element type changes. Requirements: 1, 2, 3, 4
1.2 In src/include/xce/types.h, change xce_worker_t.resvals_to_sched from xce_val_with_idx_t** to xce_name_val_t**. Requirements: 2, 3
1.3 In src/include/typemap.h, change xce_conf4sched_t.resvals from xce_val_with_idx_t** to xce_name_val_t**. Requirements: 1, 4
1.4 Run nix develop --command ninja -C src/build to regenerate controller.capnp.[ch]. Confirm the generated encode_Conf4Sched and decode_Conf4Sched now call encode_NameValue_list / decode_NameValue_list instead of their ValueWithIdx counterparts. Expect compiler errors at this stage from old call sites — that is expected and resolved in subsequent tasks.

2. (P) Replace `free_resval_with_idx_list` with `free_name_val_list`

Swap the heap-cleanup utility function to match the new element type. Old callers in the controller will be updated in Task 4.

2.1 In src/lib/utils.c, add free_name_val_list(xce_name_val_t*** vals, int count): iterate the array, free each entry's name, free val->u.svalue if string-typed, free val, free the entry, then null and free the array pointer. Mirror the structure of the existing free_resval_with_idx_list.
2.2 In src/include/utils.h, remove the free_resval_with_idx_list declaration and add extern void free_name_val_list(xce_name_val_t***, int);.
2.3 In src/lib/utils.c, delete the free_resval_with_idx_list implementation. Requirements: 5

3. (P) Add `collect_resval` and remove `collect_resval_with_idx`

Replace the index-keyed collector in the controller with a name-keyed one. The key design inversion: iterate resmgr (names) rather than the zarray_t (slots), using r->idx to look up values. This guarantees every emitted record carries a valid name.

3.1 In src/controller/dataex.c, add collect_resval(var resmgr, zarray_t* array, xlist_t* l): acquire a read lock on resmgr, iterate with iter_make/iter_next/iter_destroy, for each xce_resource_t* r call zarray_get(array, r->idx) to get the value (skip if NULL), allocate an xce_name_val_t, copy name (strdup) and value (strdup svalue or memcpy), store in the list. Release the resmgr lock. Return l->count. The array lock is the caller's responsibility (consistent with old pattern). Requirements: 1, 2, 5
3.2 In src/controller/dataex.c, rewrite collect_shared_resvals to lock c->sharedinst.lock, call collect_resval(c->resmgr, &c->sharedinst, l), then unlock. The unused fn parameter is retained for signature consistency. Requirements: 1
3.3 In src/controller/dataex.c, update collect_resvals_workers to accept var resmgr as a new first parameter. Replace the collect_resval_with_idx call with mylock_lock(&w->resvals.lock) / collect_resval(resmgr, &w->resvals, &lst) / mylock_unlock. Update the call site in collect_workers to pass c->resmgr. Requirements: 2
3.4 In src/controller/dataex.c, delete both collect_resval_with_idx_ (static) and collect_resval_with_idx. In src/controller/include/cluster.h, remove the collect_resval_with_idx declaration. Requirements: 5

4. (P) Update controller call sites

Update all controller sites that populate or clean up resvals_to_sched / conf4sched->resvals to use the new type and free function. Tasks 2 and 3 must be complete before this task.

4.1 In src/controller/sched.c — xcesched_push_configs: change the cast from (xce_val_with_idx_t**) to (xce_name_val_t**) when assigning xreslist.data to conf4sched->resvals, and replace free_resval_with_idx_list with free_name_val_list. Requirements: 1
4.2 In src/controller/sched.c — xcesched_push_workerlist and its worker-loop cleanup: add mylock_lock/mylock_unlock around collect_resval, change cast to (xce_name_val_t**), replace all free_resval_with_idx_list calls with free_name_val_list. Requirements: 2
4.3 In src/controller/sched.c:226, src/controller/reply.c:66, and src/controller/opget.c:128, replace each free_resval_with_idx_list call with free_name_val_list. Requirements: 5
4.4 In src/controller/apiserver.c — apiserver_worker_change: add mylock_lock/mylock_unlock around collect_resval, change cast to (xce_name_val_t**), replace free_resval_with_idx_list with free_name_val_list at both cleanup sites (lines 51 and 397–399). Requirements: 3

5. (P) Update scheduler — shared resource value initialisation

Change sched_init_sharedInst to resolve resource values by name via the scheduler's own resmgr instead of using the raw index received from the controller. Task 1 must be complete before this task.

5.1 In src/scheduler/src/scheduler.c — sched_init_sharedInst: replace the idx-direct loop with a name-lookup loop. For each xce_name_val_t* nv in sched->config->resvals: skip if nv or nv->name is NULL; call key_find(sched->resmgr, nv->name, &r); if not found emit WARN and continue; then MOVE_PTR(val, nv->val) and zarray_put(&sched->sharedInst, r->idx, val). Do not change the call order in sched_init_config — sched_init_resources must remain before sched_init_sharedInst. Requirements: 1, 6

6. (P) Update scheduler — per-worker resource value initialisation

Change make_sched_worker to resolve worker resource values by name. Task 1 must be complete before this task.

6.1 In src/scheduler/src/schedobj.c — make_sched_worker: replace the idx-direct loop with a name-lookup loop. For each xce_name_val_t* nv in w->resvals_to_sched: skip if nv or nv->name is NULL; call key_find(sched->resmgr, nv->name, &r); if not found emit WARN and continue; then MOVE_PTR(val, nv->val) and zarray_put(&worker->resvals, r->idx, val). The existing sched pointer (cast from void* arg) is already available at the top of the function. Requirements: 2, 6

7. Update Go API server to consume worker resource values

Update the Go API server frontend to read resvals_to_sched (now xce_name_val_t**) from received worker data and expose resource values in the HostInfo model. Previously the Go layer ignored this field entirely.

7.1 In src/frontend/apiserver/models/hosts.go, add ResourceValues []models.ResourceDistributionInfo (or ResourceValues []ResourceDistributionInfo if within the same package) to HostInfo. Requirements: 3
7.2 In src/frontend/apiserver/backend/convert_pure.go, add a resvals field ([]models.ResourceDistributionInfo) to rawWorkerData, and update rawWorkerToModel to assign info.ResourceValues = r.resvals. Requirements: 3
7.3 In src/frontend/apiserver/backend/convert.go, add cNameValToWorkerResval(workerName string, nv *C.xce_name_val_t) *models.ResourceDistributionInfo: return nil if nv or nv.val is nil (with a WARN log); set ResourceName from nv.name, Location from workerName, IsDynamic: true; decode the value type using the same xce_val_which switch as cResvalToDistribution. Requirements: 3
7.4 In src/frontend/apiserver/backend/convert.go, update cWorkerToModel to iterate worker.nresvals_to_sched entries in worker.resvals_to_sched, call cNameValToWorkerResval for each, collect non-nil results, and store them in rawWorkerData.resvals. Requirements: 3

8. Build, verify, and clean up

Confirm the full system compiles and passes tests with no regressions. All prior tasks must be complete.

8.1 Run a full clean build: nix develop --command bash -c "cd src && ninja -C build". Resolve any remaining compiler errors (expected: none if tasks 1–7 complete). Confirm controller.capnp.[ch] uses NameValue codecs for both Conf4Sched.resvals and Worker.resvals.
8.2 Run the scheduler and controller test suites: nix develop --command bash -c "cd src/frontend && make test" and any C-level tests under src/lib/testsuite/. Confirm all tests pass. Note: xcecli/common/output failure (undefined: parseJobTime) is pre-existing and unrelated to this feature.
8.3 Verify no remaining references to collect_resval_with_idx or free_resval_with_idx_list in src/ (excluding comments or git history): grep -r "collect_resval_with_idx\|free_resval_with_idx_list" src/. Expected: zero matches. Requirements: 5

Requirements Coverage

Requirement	Tasks
1	1.1, 1.3, 3.2, 4.1, 5.1
2	1.1, 1.2, 3.3, 4.2, 4.3, 6.1
3	1.1, 1.2, 4.4
4	1.1, 1.3
5	2.1, 2.2, 2.3, 3.4, 4.3, 7.3
6	5.1, 6.1

9.5 KiB Raw Blame History Unescape Escape