Breakpoint 2 of the admin-plane gaps (YAML flag/resourcemap wiring), plus
the queue payload ownership fix that the t1 controller segfault of
2026-10-23 12:41:13 traced to.
1. YAML resourcemap wiring (admin-plane-gaps 1.10):
- schema: Resource.distributions @8 / hostBased @9 uncommented --
resourcemap had no wire destination at all before this.
- resobj.c: resobj_init's #if 0 stub replaced with a real deep copy
(distributions + hbased_defined); resobj_destroy frees them. Deep
copy is mandatory: the source is myconf->cluster_config, which
keeps owning and later frees its tree.
- controller: reconcile_config_resources + seed_sharedinst_from_config
(cluster.c) seed c->sharedinst for SHARED|GLOBAL resources whose
distribution carries a value (the license-pool shape), from both
config_activate_committed paths, ahead of the conf4sched push.
- xceadmin: mapping.py maps resourcemap -> distributions (only the
location:[all]+value shape sets SHARED|GLOBAL), build.py renders
shared=true/distributions; golden expected.bin regenerated.
2. xcesched_push_configs resval double free (t1 12:39:52 abort): the
seeded push freed the resval entries AND the backing array via
free_name_val_list, then xlist_fini freed the same array again.
Detach xreslist before the FREEUP/xlist_fini block (same ownership
class bug-1506 fixed in sched_worker_change). Latent since inception:
every pre-1.10 push ran with nresvals==0 and never armed it.
3. queue payload deep-copy (t1 12:41:13 encode_Threshold_list segfault):
queue_init (create) MOVED the definition's users/hosts/thresholds/
window payloads out, and queue_refresh_def MOVED them out again --
but reconcile_config_queues runs MORE THAN ONCE over the same
cluster_config object (boot: pre-election activation AND phase2's
build_queues), so the second pass refreshed live queues from the
drained definition: thresholds became the poisoned (count>0,
array=NULL) pair, the generated encode_Threshold_list has no NULL
guard, and the next queue/worker push died on the NULL base.
Both paths now deep-copy (queue_init and queue_refresh_def); the
definition keeps its payloads, (NULL/0) genuinely means "no list in
this config", and reconcile is idempotent. New dup_group_config
helper in utils.c (the generated code has no GroupConfig dup).
Mirrors resobj_init's distributions deep-copy and worker.c
apply_join_state's dup pattern.
4. UT: test_reconcile_thresholds_survive_second_pass drives a triple
reconcile of a threshold-carrying queue and asserts definition
intactness plus a real encode_QueueConf -> wire -> decode round
trip; new test_config_resource_reconcile.c covers seed/deep-copy/
non-seed classes and idempotent re-seed.
Suites: controller_unit_test 1348/1348, sched_push 4, sched_worker_change 3,
apiserver_push 2, host_runusage 11, xcelib 759, xcealloc_oom 11,
99 scheduler binaries, all worker + harness suites, xceadmin pytest 375.
t1 E2E: boot double-reconcile clean, scheduler rejoin + full conf4sched
push applied, recovered crash-era PEND job reached DONE, 30-job saturation
flow (24 RUN + 6 PEND -> DONE), live gen-12 config activation with
re-push to the connected scheduler -- no crash, all four daemons healthy.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Add workerResyncWindowMs/workerResyncTimeoutMs to xce.schema.json
(minimum:0, matching the optional-scalar-field precedent -- these
aren't `required` with a floor like heartbeatInterval, so minimum:1
would be wrong) and to mapping.py's YAML->capnp field table. Regenerate
both golden config-build fixtures (task 1.1 grew ClusterConfig's wire
layout by 2 fields; full/expected.bin and minimal/expected.bin each
grew by exactly 8 bytes). full's fixture YAML now sets real values
(900000ms/15000ms); minimal deliberately left untouched (no xce.yaml)
per its own locked-down "hosts+queues only" boundary test -- its
expected.bin still regenerates to reflect the new default-0 fields.
Two new tests assert specific decoded values, not just "build succeeds".
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y4M5MJM2zK6nU43da267Ku
Cross-reference warnings ("references undefined entry") were genuine
gaps in the example configs, not validator bugs: src/configs/hosts.yaml
had a typo'd resource ref (mg -> mem) and referenced a nonexistent
host3/hostgroup pair; queues.yaml referenced hgroup1/hgroup2/ugroup1
instead of the real hg1/ug1 names; resources.yaml was missing the
r1m/r15s/tmp built-in worker load-index resources
(worker/src/resource_sync.c's init_builtin_load_indices()). Added the
missing host3 host, hg2 hostgroup, ug2 usergroup, and three resource
entries; fixed the typo'd refs.
"schema-valid but has no ClusterConfig destination" warnings for
resources.yaml's type/order/consumable/release fields were a real gap
in xcecommon/mapping.py's _build_resource, not an inherent limitation
-- the encoding (XCE_VAL_*/xce_order_t/XCE_RESF_CONSUMABLE/
XCE_RESF_RELEASE) is documented in include/xce/types.h and actively
read by the scheduler; the old docstring's claim that it was
undocumented was wrong. Only resourcemap remains genuinely unmapped
(its capnp destination fields are commented out).
Updated test_mapping.py/test_config_build.py for the new mapping
behavior and regenerated the full golden fixture's expected.bin
(legitimate representation-drift from the mapping fix, not a bug).
tests/fixtures/{minimal,full}/configs/*.yaml + checked-in expected.bin
golden files, added to test_config_build.py. Catches representation
drift (bytes changing even when decoded content stays correct) that
the existing structural/decode-based assertions can't detect.
config build pipeline: scan the 7 known YAML filenames, validate
each present file, run cross-file business-rule checks, map to
ClusterConfig, serialize. Missing files become empty sections;
schema-valid-but-unmapped fields and undefined references warn to
stderr instead of failing. write_output() handles -o <path>/-o -
and returns the hex SHA-256 digest (printing it is the CLI's job).
