From 48a5a0b4e68df69d68d3217ab773383c3eaf618b Mon Sep 17 00:00:00 2001 From: Rongsong Shen Date: Wed, 22 Jul 2026 09:58:04 +0000 Subject: [PATCH] feat(admin-config-tool): task 5.4 - golden-file build determinism tests tests/fixtures/{minimal,full}/configs/*.yaml + checked-in expected.bin golden files, added to test_config_build.py. Catches representation drift (bytes changing even when decoded content stays correct) that the existing structural/decode-based assertions can't detect. --- .kiro/specs/admin-config-tool/tasks.md | 3 +- .../tests/fixtures/full/configs/hosts.yaml | 39 ++++ .../tests/fixtures/full/configs/limits.yaml | 49 +++++ .../tests/fixtures/full/configs/queues.yaml | 94 +++++++++ .../fixtures/full/configs/resources.yaml | 199 ++++++++++++++++++ .../tests/fixtures/full/configs/scaler.yaml | 13 ++ .../tests/fixtures/full/configs/users.yaml | 8 + .../tests/fixtures/full/configs/xce.yaml | 6 + src/xceadmin/tests/fixtures/full/expected.bin | Bin 0 -> 5304 bytes .../tests/fixtures/minimal/configs/hosts.yaml | 8 + .../fixtures/minimal/configs/queues.yaml | 3 + .../tests/fixtures/minimal/expected.bin | Bin 0 -> 448 bytes src/xceadmin/tests/test_config_build.py | 62 +++++- 13 files changed, 481 insertions(+), 3 deletions(-) create mode 100644 src/xceadmin/tests/fixtures/full/configs/hosts.yaml create mode 100644 src/xceadmin/tests/fixtures/full/configs/limits.yaml create mode 100644 src/xceadmin/tests/fixtures/full/configs/queues.yaml create mode 100644 src/xceadmin/tests/fixtures/full/configs/resources.yaml create mode 100644 src/xceadmin/tests/fixtures/full/configs/scaler.yaml create mode 100644 src/xceadmin/tests/fixtures/full/configs/users.yaml create mode 100644 src/xceadmin/tests/fixtures/full/configs/xce.yaml create mode 100644 src/xceadmin/tests/fixtures/full/expected.bin create mode 100644 src/xceadmin/tests/fixtures/minimal/configs/hosts.yaml create mode 100644 src/xceadmin/tests/fixtures/minimal/configs/queues.yaml create mode 100644 src/xceadmin/tests/fixtures/minimal/expected.bin diff --git a/.kiro/specs/admin-config-tool/tasks.md b/.kiro/specs/admin-config-tool/tasks.md index 3f099e76..a6366e42 100644 --- a/.kiro/specs/admin-config-tool/tasks.md +++ b/.kiro/specs/admin-config-tool/tasks.md @@ -121,7 +121,7 @@ - _Boundary: xcecommon.schema, xcecommon.validation_ - _Depends: 2.1, 2.2, 1.3_ -- [ ] 5.4 (P) Golden-file test config build determinism +- [x] 5.4 (P) Golden-file test config build determinism - Building from a minimal fixture, a full fixture covering all seven file types, and a set of intentionally invalid fixtures each produce the expected result - Building the same fixture twice produces byte-identical output and an identical checksum - _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 1.10, 1.11, 1.12, 9.1, 9.2, 9.3_ @@ -158,3 +158,4 @@ - **4.2**: `deploy-ha.sh` resolves both `xceadmin` (`$SCRIPT_DIR/../xceadmin/.venv/bin/xceadmin`) and the YAML configs source (`$SCRIPT_DIR/../configs`) relative to the script's own location (`$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)`), not the caller's CWD — matches the project's other relative-path scripts and keeps the deploy working regardless of where it's invoked from. `--build-config` is parsed out of `"$@"` before positional-arg assignment so it can appear anywhere in the argument list, not just a fixed position. Verified live against the real `xceadmin` build and `src/configs/`: missing `cluster.bin` auto-builds (8.4), an existing one with no flag is left byte-for-byte untouched — same mtime, same SHA (8.5), and `--build-config` against an existing `cluster.bin` forces a rebuild that reproduces byte-identical content (8.3, confirming 9.1's determinism guarantee holds through this path too). - **5.1, 5.2**: Already fully satisfied by task 2.3's `test_mapping.py` and task 2.4's `test_wire.py` respectively — same requirement lists, same acceptance bullets (in-memory-only mapper coverage + the explicit two-field case for 5.1; a fixed known byte sequence + real loopback round-trips for 5.2). No new tests written; would have been pure duplication. - **5.3**: Found and fixed a real gap while writing `test_schema_bundling.py` — `xcecommon.validation.validate_yaml()`'s `json.loads(schema_path.read_text())` call (loading the *bundled JSON validation schema itself*, not the operator's YAML) had no error handling, so a corrupted bundled JSON schema crashed with a raw `json.JSONDecodeError` instead of a clean `ValidationError` — the same "crash vs. clean error" class of bug task 2.2 already fixed for `controller.capnp`, just on the JSON-schema side, and not caught by task 2.1 since that task's tests only exercised operator-YAML failure modes, never a corrupted *schema* file. Fixed by wrapping it and raising `ValidationError(..., constraint="schema-corrupted")`, matching the existing missing-schema handling's pattern one function above it. **Gotcha for any future task touching `xcecommon`/`xceadmin` source**: editing a source file does NOT propagate to a running `.venv` until reinstalled — always re-run `xce-build.sh xceadmin` after a source edit before trusting a bare `.venv/bin/python -m pytest` result; a stale venv (especially one left in `release`/pinned mode by a prior profile run) will silently test the *old* code and produce a false failure (or false pass) that looks like a real bug. +- **5.4**: `tests/fixtures/{minimal,full}/configs/*.yaml` + checked-in `expected.bin` golden files added to `test_config_build.py` (not a new file — design.md's File Structure Plan lists the SAME requirement set for `test_config_build.py` under both 3.1 and 5.4). `full/configs/*.yaml` is a byte-for-byte copy of `src/configs/*.yaml`; its golden SHA (`860cfd4f59ef...`) independently matches the tag `deploy-ha.sh` produced in task 4.2's live verification against the same source files — an unplanned but welcome additional confirmation of Requirement 9's cross-context determinism guarantee. Golden-file tests catch representation drift (bytes change even though decoded content is still semantically correct) that purely structural/decode-based assertions elsewhere in the suite cannot detect; regenerate `expected.bin` deliberately (via `build_cluster_config()`, sanity-checked by decoding it back) whenever a real, intentional field-mapping change lands — never hand-edit the binary. diff --git a/src/xceadmin/tests/fixtures/full/configs/hosts.yaml b/src/xceadmin/tests/fixtures/full/configs/hosts.yaml new file mode 100644 index 00000000..d9d57167 --- /dev/null +++ b/src/xceadmin/tests/fixtures/full/configs/hosts.yaml @@ -0,0 +1,39 @@ +--- +hosts: + - hostname: default + type: "x86_64-linux-gnu" + model: "x86_64-linux-gnu" + max_slot: -1 + description: "default host" + resource: + - name: mg + - name: maxswap + value: 10000 + sched_stop: + - resname: r1m + sched: 3.5 + stop: 4.5 + - resname: r15s + sched: 4.0 + stop: 10.0 + dispatch_window: + - start_time: "9:00" + end_time: "18:00" + - start_time: "21:00" + end_time: "7:00" + + - hostname: worker2 + model: "!" + type: "!" + max_slot: 256 + resource: + - name: gstring + value: "This is a G string" + + +hostgroups: + - groupname: hg1 + members: + - all + exclude_members: [h3] + diff --git a/src/xceadmin/tests/fixtures/full/configs/limits.yaml b/src/xceadmin/tests/fixtures/full/configs/limits.yaml new file mode 100644 index 00000000..127312fd --- /dev/null +++ b/src/xceadmin/tests/fixtures/full/configs/limits.yaml @@ -0,0 +1,49 @@ +--- +limits: + # cap total slots for ug1 across the whole cluster + - name: ug1_slots + users: + members: + - ug1 + resname: slots + value: 10 + + # per-user slot cap inside ug2 (each member gets up to 4, not a shared pool) + - name: ug2_slots_per_user + users: + members: + - ug2 + exclude_members: + - user3 + resname: slots + value: 4 + per_user: true + + # queue-scoped memory cap, percentage of total mem resource + - name: night_mem_pct + queues: + members: + - night + resname: mem + value: 50 + percent: true + + # host-scoped cap, applied per host in hg1 + - name: hg1_slots_per_host + hosts: + members: + - hg1 + resname: slots + value: 20 + per_host: true + + # combined scope: ug1 members on the normal queue + - name: ug1_normal_slots + users: + members: + - ug1 + queues: + members: + - normal + resname: slots + value: 8 diff --git a/src/xceadmin/tests/fixtures/full/configs/queues.yaml b/src/xceadmin/tests/fixtures/full/configs/queues.yaml new file mode 100644 index 00000000..9bafd74c --- /dev/null +++ b/src/xceadmin/tests/fixtures/full/configs/queues.yaml @@ -0,0 +1,94 @@ +--- +queues: + - name: xce + description: "xce queue" + priority: 30 + fairshare: + - user: default + share: 100 + hosts: + members: + - all + + - name: normal + description: "normal queue" + priority: 40 + nice: 10 + fairshare: + - user: default + share: 100 + hosts: + members: + - all + + + - name: night + description: "night queue" + priority: 30 + nice: 10 + res_req: "mem>1000 rusage[mem=1000]" + gpu_req: "num=1:mode=shared:gvendor=NVIDIA:gmodel=A100" + fairshare: + - user: ug1 + share: 1000 + - user: others + share: 100 + cpulimit: 100 + memlimit: 1000 + runlimit: 20 + runtimeest: 30 + dispatch_window: + - start_time: "19:00" + end_time: "8:30" + run_window: + - start_time: "5:19:00" + end_time: "1:8:30" + - start_time: "20:00" + end_time: "8:30" + sched_stop: + - resname: tmp + sched: 2000 + stop: 1000 + - resname: res1 + sched: 1.0 + stop: 2.5 + reservation: 100 + preemption: preemptive + backfill: true + exclusive: false + rerunable: true + pre_exec: "/path/to/preexec.sh" + post_exec: "/path/to/postexec.sh" + job_starter: "/path/to/jobstarter.sh" + users: + members: + - ug1 + - ug2 + exclude_members: + - user3 + hosts: + members: + - hgroup1 + - hgroup2 + exclude_members: + - host3 + + - name: short + description: "for short jobs" + priority: 45 + cpulimit: 15 + fairshare: + - user: ug1 + share: 1000 + - user: default + share: 100 + reservation: 10 + preemption: preemptable + backfill: true + users: + members: + - ugroup1 + hosts: + members: + - hgroup1 + diff --git a/src/xceadmin/tests/fixtures/full/configs/resources.yaml b/src/xceadmin/tests/fixtures/full/configs/resources.yaml new file mode 100644 index 00000000..f12b9f82 --- /dev/null +++ b/src/xceadmin/tests/fixtures/full/configs/resources.yaml @@ -0,0 +1,199 @@ +--- +resources: + # built-in: job slot count (consumed by running jobs, decreases as slots fill) + - name: slots + type: numeric + order: decrease + consumable: true + description: "job slot count (built-in)" + + # built-in: dynamic memory load index reported by workers + - name: mem + type: numeric + interval: 60 + consumable: false + order: decrease + description: "available memory in MB (built-in dynamic load index)" + + # built-in host attributes, registered per-worker by + # controller/worker.c add_hostattrs_resvals() + - name: type + type: string + description: "host OS/arch type (built-in host attribute)" + + - name: model + type: string + description: "host hardware model (built-in host attribute)" + + - name: cpuf + type: numeric + description: "CPU speed factor (built-in host attribute)" + + - name: maxcpus + type: numeric + description: "max CPU count (built-in host attribute)" + + - name: maxmem + type: numeric + description: "max memory in MB (built-in host attribute)" + + - name: maxswap + type: numeric + description: "max swap in MB (built-in host attribute)" + + - name: maxtmp + type: numeric + description: "max tmp disk space in MB (built-in host attribute)" + + - name: ndisks + type: numeric + description: "disk count (built-in host attribute)" + + # dynamic hostbased resource, external load index + - name: dynres1 + type: numeric + interval: 60 + order: decrease + consumable: true + release: true + description: "user defined dynamic resource" + resourcemap: + - location: + - default + + + # dynamic share resource, 2 instances + - name: dynres2 + type: numeric + interval: 60 + consumable: false + order: decrease + description: "user defined dynamic resource" + resourcemap: + - location: + - host1 + - host2 + - location: + - host5 + - host6 + + + # static share resource, all host share 100 res. + - name: staticres4 + type: numeric + interval: 0 + consumable: true + order: decrease + resourcemap: + - location: + - all + value: 100 + + # static share resource, host1 and host2 share the 100 res. + - name: staticres1 + type: numeric + interval: 0 + consumable: true + order: decrease + resourcemap: + - value: 100 + location: + - host1 + - host2 + + # static share resource, each of host1 and host2 shares the 100 res. + - name: staticres2 + type: numeric + interval: 0 + consumable: false + order: decrease + resourcemap: + - value: 100 + location: + - host1 + - value: 100 + location: + - host2 + + # static hostbased resource. + - name: staticres3 + type: numeric + interval: 0 + consumable: true + order: decrease + resourcemap: + - value: 100 + location: + - default + + - name: res1 + type: numeric + interval: 0 + consumable: true + order: decrease + resourcemap: + - location: + - all + value: 100 + + - name: sres1 + type: numeric + interval: 0 + consumable: true + order: decrease + + + # boolean resource + - name: bigmem + type: boolean + order: na + resourcemap: + - value: true + location: + - host1 + - value: true + location: + - host2 + - value: true + location: + - host3 + + # dynamic string resource + - name: strres1 + type: string + interval: 60 + consumable: false + order: na + description: "string resources" + resourcemap: + - location: + - default + + # static string resource + - name: strres2 + type: string + description: "string resources" + resourcemap: + - value: v1 + location: + - host1 + - value: v2 + location: + - host2 + - value: v3 + location: + - host3 + - name: host_quota + description: quota on each host + type: numeric + order: decrease + consumable: true + interval: 0 + resourcemap: + - value: 10 + location: + - default + + - name: gstring + description: string resource + type: string diff --git a/src/xceadmin/tests/fixtures/full/configs/scaler.yaml b/src/xceadmin/tests/fixtures/full/configs/scaler.yaml new file mode 100644 index 00000000..9f904ea1 --- /dev/null +++ b/src/xceadmin/tests/fixtures/full/configs/scaler.yaml @@ -0,0 +1,13 @@ +--- +scalers: + # script-based cloud autoscaler provider + - name: aws-asg + type: script + conf_path: /etc/xce/scalers/aws-asg.conf + script_path: /opt/xce/scalers/aws_asg.sh + script_option: "--region us-east-1" + + # minimal provider, no conf file or options + - name: k8s-pool + type: script + script_path: /opt/xce/scalers/k8s_pool.sh diff --git a/src/xceadmin/tests/fixtures/full/configs/users.yaml b/src/xceadmin/tests/fixtures/full/configs/users.yaml new file mode 100644 index 00000000..b8841ee2 --- /dev/null +++ b/src/xceadmin/tests/fixtures/full/configs/users.yaml @@ -0,0 +1,8 @@ +--- +#define usergroup share +usergroups: + - groupname: ug1 + members: + - user1 + - user2 + diff --git a/src/xceadmin/tests/fixtures/full/configs/xce.yaml b/src/xceadmin/tests/fixtures/full/configs/xce.yaml new file mode 100644 index 00000000..05a0fef2 --- /dev/null +++ b/src/xceadmin/tests/fixtures/full/configs/xce.yaml @@ -0,0 +1,6 @@ +--- +cluster_name: eda1 +heartbeat_interval: 30 +inactive_heartbeat_count: 3 +fetch_job_interval: 10 +netio_timeout: 30 diff --git a/src/xceadmin/tests/fixtures/full/expected.bin b/src/xceadmin/tests/fixtures/full/expected.bin new file mode 100644 index 0000000000000000000000000000000000000000..13aa3878e34d7490954818310278374fc7dfde21 GIT binary patch literal 5304 zcmZQzU|^WW1R@z2m>D=2P?~QL9`YY_2}^h zQU~%ci2j5ne$dq$LfsFdv8hKl0_5%}bU{3PP`r|&o|lP%ffV(B8A((BmJyOZ@i-Yp z>Lw#3zn}=>;T&az#4j!-=;6PS5t2SZbQdm7Na8D@>OnLy>K8)8XDK5C!z?6oa53jX z%?Hs<3=9m#Ir$~U3=9le`AG^OwnB1#X!Pg0)ULmD2FEKYWSs^DsF-0LWFD11CY(zR31&|<;Rh@3LpkXIAYZWc3(+u0hm@O$t_Sw$t=!RC@x4$P9@HSJdlQB1_ls? z9@iM2V_;xNsmv=%EjDB*Elw>`NJ&l0%u7vy$8k|=aeir0GALpp>Wm1gD=tYa$xJRv zEj9rM0|T57uH_gId{89@N}n+G#t`*j$AHA4jHJwTaQGLO6oEyGONuh{(iLDX10`*+ zMu@x-11@<`aK{&x=9eUbl{0{u3i){osfo!M;DCXMr$Y=tH-|w0T>o=I6rr^NK!O_? zAoVPW4Z>?c1Oqd}7AOs-un>E|VvJCAhzddjERG=7GN6`AE5K6B3>%;{n8HHr0E;m~ z)gjCWwP(OmP+}zmq?`oNpmxO>usAcr8z{}o$iQIEz`y{KU&O$`0P^oD1_lNXu#t=m z3Q!uXoPmKs7si0nrD)}F0a!gVLj#nCD#M^U!19bxb&L!Z;M9nr3`2mGfq{V!NfcxZ zs2!A;11fEhB%sW^{G!}M>{3t_5Q=1TGt)ClK;=$quAQNQfq_C%X>nqDYIJUDt}Tck z3o({KzaX(BL%$?nzn~~JwIVfHuQ&rNgAmUzE^@1`7k{o8itYOi!<_zK=BTf1DS`O9zo$+3Jq66>LB3`iA$JT5Df|!kpDm>6DZz6 zG$h^Lp!SX>0|Ntyc4S~+P-kFZ0HuFWJk@|zGeQyo0|UcW21vUK zL<>RrFj^d}7DAi=Q;ZBLscDI&IVB7g7H08gCb~J9d8HM)>3OAKX$A(I;vn~eQZC4! zAb-HrgTfKyZV(L$pWJkaOm1QYs5)l=bKyC%mFboPW7#l>3f(Rtcj>Im{FUn3WGGb7Kios|| zx?lpAGobJQr5{i`6jo1$WMmdAWELwVD!3~^t4^p15DMZSRt8mQe1d3D{|Q8c`YWJx z@rZ$e!IlBizIA0_VDMsKVDLn%??B?OAVz}8#PVX@#Nu>@;^d;tf)WM>{nV0V{fgvN z{o>@roYbOXeW(Jxp_`X&Q5+9a2sIE?o*?IgRt5%!AO;484zTN)8D>D~1yCBfJ_n^I zPD6PT> zO8t!Bd<#;l3+-o^K;>fzmNheIPv`oXQBv*Evx68YtZZrKdpYIZ%D2>4x#3 z{s=hngA9OTQ2GLuH&8CPJ^+TZP(1;QcW}6ZS`P@-)MR1v7sws3dJNSr6RLKkeP`ty$LH-7rkDgAkxdT-GLCl8fCYM&ga5n%bv@d`F literal 0 HcmV?d00001 diff --git a/src/xceadmin/tests/fixtures/minimal/configs/hosts.yaml b/src/xceadmin/tests/fixtures/minimal/configs/hosts.yaml new file mode 100644 index 00000000..feea3662 --- /dev/null +++ b/src/xceadmin/tests/fixtures/minimal/configs/hosts.yaml @@ -0,0 +1,8 @@ +hosts: + - hostname: worker1 + type: x86_64-linux-gnu + max_slot: 8 +hostgroups: + - groupname: all_hosts + members: + - worker1 diff --git a/src/xceadmin/tests/fixtures/minimal/configs/queues.yaml b/src/xceadmin/tests/fixtures/minimal/configs/queues.yaml new file mode 100644 index 00000000..d091cc35 --- /dev/null +++ b/src/xceadmin/tests/fixtures/minimal/configs/queues.yaml @@ -0,0 +1,3 @@ +queues: + - name: normal + priority: 30 diff --git a/src/xceadmin/tests/fixtures/minimal/expected.bin b/src/xceadmin/tests/fixtures/minimal/expected.bin new file mode 100644 index 0000000000000000000000000000000000000000..d2ac36de35b046828f61abec68a36c58e8aa0d8b GIT binary patch literal 448 zcmZQzU|=u@5ey6r%nTe*7GX-1fq@|%Nm(ib1A{yR14APN1A{%1IE={xHlGv5Wsrlj zAPhwY1_mn#6HLm0NoEEWC=I5t5D26Q7Y^Tv7}bV`N}paAII!D9