b38ef6de63
The external-license-resources E2 fixture builder crr_build_config
registers five resources and four distributions, writing resources[4],
dists[3] and vals[4]. All four caller test functions declared only
resources[4]/dists[3]/vals[4], so every build_config call wrote past
the caller's stack arrays -- deterministic ASAN stack-buffer-overflow,
the whole controller suite (137 fixtures) red since 2fcacac2.
Resize the four caller arrays to match the builder's sizing
(resources[5]/dists[4]/vals[5]) with a bug-1977 comment on the sizing
contract. No production code touched.
Verified: src/tools/xce-build.sh controller (asan) 6/6 OK, controller
suite 91s green; buglog record .wolf/bugs/bug-1977.md.
Co-Authored-By: Claude Code <noreply@anthropic.com>
197 lines
24 KiB
JSON
197 lines
24 KiB
JSON
{
|
|
"version": 1,
|
|
"generated": "by scripts/buglog.py from .wolf/bugs/*.md -- do not edit",
|
|
"bugs": [
|
|
{
|
|
"id": "bug-1971",
|
|
"status": "fixed",
|
|
"status_updated": "2026-10-22",
|
|
"status_note": "two-choke archive in xce-build.sh; test harness green",
|
|
"component": "build",
|
|
"severity": "wrong-result",
|
|
"timestamp": "2026-10-22T07:50:00Z",
|
|
"last_seen": "2026-10-22T07:50:00Z",
|
|
"occurrences": 2,
|
|
"error_message": "next xce-build.sh run destroys the prior testlog — flaky-failure evidence unrecoverable\n\n**Observed:** (t2, 2026-10-22) a full `make build` (asan profile) failed on a controller test case; a following `make clean` + `make build` passed 134/134. The failing run's `meson-logs/testlog.{txt,json,junit.xml}` had been destroyed twice over: `setup_build`'s unconditional `rm -rf \"$BUILD_DIR\"` deletes the whole build dir (meson-logs included) before the retry, and meson itself overwrites `testlog.*` on every `meson test` invocation regardless. Nothing in the surviving tree identifies which case failed: the worker suite's daemon logs (`/tmp/test_xcecd_resync.log` and rotations) showed no failure signature, and no `sched_test`/Go test output is retained anywhere.\n\nReconstruction from filesystem timestamps alone (`/tmp/test_*.log` rotations, `meson-log.txt` build options line, `/usr/local/go` ctime, bash history) established the two-run timeline but could not name the failed case.\n\nThe same evidence-loss shape hit t1 earlier (2026-10-21, the two controller flakes fixed in 5b9d7270) — there the testlog happened to still exist because no clean rebuild ran in between; that was luck, not design.\n\n**Expected:** when a build/test run fails, its meson-logs (testlog.txt/json/junit.xml) and any per-case fixtures it kept (the mkdtemp sweep contract deliberately preserves FAILED cases' `/tmp/xce_*` dirs) survive at least until the next run starts — so a flaky failure can be diagnosed from disk afterwards without re-running and hoping to reproduce.\n\n**Impact:** intermittent/flaky test failures (the exact class this project keeps hitting: 5b9d7270's two flakes, bug-1521's random heap corruption, the t2 2026-10-22 case) leave zero durable evidence. Each recurrence costs a full rebuild cycle and often fails to reproduce. Also buries the mkdtemp \"failed cases keep their dirs for post-mortem\" contract, since the next `make build` never looks for those dirs and a clean build gives the appearance of none existing.\n\n**Found by:** user report (t2 build failure, 2026-10-22) + post-mortem on the t2 filesystem.",
|
|
"file": "src/tools/xce-build.sh",
|
|
"root_cause": "`setup_build()` in `src/tools/xce-build.sh` unconditionally `rm -rf \"$BUILD_DIR\"` on every non-incremental invocation, and `meson test` overwrites `$BUILD_DIR/meson-logs/testlog.*` on every run. xce-build.sh never copies failure state anywhere durable, and nothing else in the tree does either.",
|
|
"fix": "Two chokes in `src/tools/xce-build.sh` (this commit):\n\n1. `archive_previous_failure()` — runs at script start, before `setup_build`'s\n wipe. If the marker `.meson_test_failed_rc` (written by the previous run's\n EXIT trap) exists, the previous run's `meson-logs` are MOVED to\n `$SRC_DIR/testlogs.archive/<BUILD_DIR>/prev-<stamp>-rc<N>/` — outside the\n build dir, so the wipe can't touch them. It also sweeps aged (>5 min, i.e.\n not a concurrent session's) `/tmp/xce_*` FAILED-case fixture DIRS into\n `prev-<stamp>-fixtures/`, and caps the archive at the newest 10 entries\n per build dir.\n2. `finalize_failure_evidence()` — the EXIT trap (replaces the old\n lock-release trap, which becomes its last statement). On rc≠0 with\n RUN_TESTS=1, COPIES `meson-logs` to\n `testlogs.archive/<BUILD_DIR>/this-<stamp>-rc<N>/` and writes the marker.\n Copy-not-move: meson-logs still belongs to the live build dir for this\n run's own inspection; the next run moves the copy out at its start.\n Configure/compile failures (RUN_TESTS unset) archive nothing — those are\n deterministic and self-explanatory on rerun; it's the testlog-bearing\n flaky failures that need preservation.\n\nAdditionally `-type d` added to BOTH `/tmp/xce_*` sweeps (this one and\nbug-924's `prune_stale_tmp_fixtures`): mkdtemp fixtures are always\ndirectories (`/tmp/xce_<label>_XXXXXX`), but the bare name also matches\nregular files — the untyped find live-moved daemon debug logs\n(`/tmp/xce_scheduler.log`, `/tmp/xce_strace*.log`) into the archive on\n2026-10-22, and the bug-924 variant `rm -rf`s its matches.\n\nVerification:\n- Test: `src/tools/test-failure-evidence-archive.sh` — 6 scenarios /\n 11 assertions exercising the real function bodies (sourced with surgical\n stubbing): failed run archives as this-*-rc3 + marker; success archives\n nothing; next run moves prior failure to prev-*-rc3 + consumes marker;\n aged fixture dirs move / fresh stay / regular files never touched;\n retention caps at newest 10; RUN_TESTS=0 archives nothing. All green\n (11 passed, 0 failed, bash, 2026-10-22).\n- Suites: n/a — this bug lives in the build script, not in C code; no meson\n suite exercises it. `bash -n` syntax check green.\n- Real wipe path not exercised by the harness test (it stubs meson);\n validated by inspection: the archive root is `$SRC_DIR/testlogs.archive/`,\n which `rm -rf \"$BUILD_DIR\"` cannot reach.\n- Live: first t2/sandbox failing run after this commit leaves\n `testlogs.archive/xce.build/this-*-rc*` with the testlog — that is the\n evidence this bug asks for.",
|
|
"tags": [
|
|
"test",
|
|
"flaky",
|
|
"observability",
|
|
"build"
|
|
],
|
|
"related_bugs": [],
|
|
"fix_commits": [
|
|
"68c3ccd7"
|
|
],
|
|
"record": ".wolf/bugs/bug-1971.md"
|
|
},
|
|
{
|
|
"id": "bug-1972",
|
|
"status": "fixed",
|
|
"status_updated": "2026-10-26",
|
|
"status_note": "",
|
|
"component": "controller",
|
|
"severity": "crash",
|
|
"timestamp": "2026-10-26T09:46:58Z",
|
|
"last_seen": "2026-10-26T09:46:58Z",
|
|
"occurrences": 4,
|
|
"error_message": "test_job_live_gc asserted destruction by reading freed memory\n\n**Observed:**\n`controller_unit_test` aborts under ASAN at `test_sweep_purges_expired_done_job`\n(test_job_live_gc.c:169):\n\n```\n==27834==ERROR: AddressSanitizer: heap-use-after-free on address 0x5110000da290\nREAD of size 8 at 0x5110000da290 thread T0\n #0 job_get_refcount ../lib/jobj.c:715\n #1 test_sweep_purges_expired_done_job test_job_live_gc.c:169\nfreed by:\n #3 job_dereference ../lib/jobj.c:705\n #5 job_live_gc_sweep ../controller/job.c:4777\n```\n\n**Expected:** a correct sweep dereferences the purged job down to refcount 0\nand destroys it; the test must verify destruction without touching the freed\nobject.\n\n**Impact:** controller suite aborts (killed by SIGABRT); the whole\n`controller test` meson entry fails. Test-only — the production sweep\n(job.c:4765-4777, double dereference: collection ref + table's makeJob ref)\nis correct.\n\n**Found by:** ASAN, first full-suite run after d028c8c3 (the archives' earlier\nruns all predate that commit).",
|
|
"file": "src/controller/testsuite/test_job_live_gc.c",
|
|
"root_cause": "The assertion `TEST_ASSERT_EQUAL_UINT32(0, job_get_refcount(expired))` runs\nAFTER `job_live_gc_sweep` has already dereferenced the job to 0 and freed the\nentire RTTI object (header + private + data in one calloc block). Reading the\nrefcount through the dangling `job_t*` is itself the use-after-free: the\nstronger the sweep does its job, the more surely the test crashes. The same\npattern is absent from test_apply_pump.c's refcount test because there the\njob is never destroyed.",
|
|
"fix": "Move the refcount probe BEFORE the sweep and invert its meaning: assert\nrefcount == 1 (table's seed reference, nothing else) before `job_live_gc_sweep`,\nthen verify destruction by absence — `jlg_find_job(81001) == NULL` plus the\nidempotent second sweep (both already present). `expired` is never dereferenced\nafter the sweep; a comment marks the last safe point.\n\nVerification:\n- Test: test_sweep_purges_expired_done_job (same file) — RED before (deterministic\n ASAN abort 4/4), GREEN after (rc=0, 1354/1354 controller unity tests, zero\n ASAN reports).\n- Suites: controller_unit_test 1354 tests 0 failures (ASAN, detect_leaks=0);\n sched_repl_trace_baseline_test 4/4 with regenerated fixtures (separate issue,\n bug-free fixture staleness from fc33a5da's schema change, handled in the same\n working tree).\n- Live: n/a (test-only defect; production sweep unchanged).",
|
|
"tags": [
|
|
"test-only",
|
|
"uaf",
|
|
"gc"
|
|
],
|
|
"related_bugs": [],
|
|
"fix_commits": [
|
|
"f780a166"
|
|
],
|
|
"record": ".wolf/bugs/bug-1972.md"
|
|
},
|
|
{
|
|
"id": "bug-1973",
|
|
"status": "fixed",
|
|
"status_updated": "2026-10-27",
|
|
"status_note": "",
|
|
"component": "controller",
|
|
"severity": "wrong-result",
|
|
"timestamp": "2026-10-27T03:27:47Z",
|
|
"last_seen": "2026-10-27T03:27:47Z",
|
|
"occurrences": 1,
|
|
"error_message": "cluster_build_resources aborted at first EEXIST: new resource defs never registered\n\n**Observed:**\nController log on t1, after pushing a cluster.bin whose resources section ADDED\nthe new `extlic` definition (external-license-resources E2 Phase 1):\n\n```\n[ERROR] [log_update_resval_failure] update_resval `extlic': No such file or directory\n```\n\nThe xceesd daemon joined and pushed extlic values, but the controller had\nno `extlic` registration at all (update_resval key_find → -ENOENT).\n\n**Expected:** a config activation that adds a new resource definition must\nregister it in the running controller's resmgr, so daemon pushes land.\n\n**Impact:** every newly-added resource definition was silently inert until\nthe controller process was restarted (boot-time registration path did run).\nFor E2 this meant the whole extres-daemon chain was dead on a live controller.\n\n**Found by:** E2 Phase 1 live E2E on t1 (spec .kiro/specs/external-license-resources design.md §3.5 checklist).",
|
|
"file": "src/controller/cluster.c, src/controller/testsuite/test_config_resource_reconcile.c",
|
|
"root_cause": "`cluster_build_resources()` looped over the decoded config's resource list\ncalling `makeResource()` for each. `makeResource_` (src/lib/resobj.c:244-247)\nreturns NULL with errno=EEXIST when the name is already registered — which is\nthe COMMON case at activation (boot already registered some definitions). The\nold loop treated that rc as fatal and aborted the whole loop, so every\ndefinition AFTER the first already-registered one was never registered.",
|
|
"fix": "Rewrite the loop on top of bug-1854's `resourceApplyDef()` (lookup+add under\none write lock, EEXIST-tolerant): RES_APPLY_ADDED/SAME proceed normally;\nRES_APPLY_CHANGED updates the registered object's definition fields\n(valtype/ordtype/flags/interval) IN PLACE — none of them touches the\nregistration slot (idx) or accumulated accounting; mirrors\nreconcile_config_queues' in-place refresh precedent (second live bug, see\nbug-1974 — kept here as one loop). Negative rc still propagates.\n\nVerification:\n- Test: `test_reconcile_resources_registers_new_defs_past_eexist` +\n `test_reconcile_resources_hot_applies_changed_definition`\n (src/controller/testsuite/test_config_resource_reconcile.c); RED before the\n fix (lic1 unregistered / flags stayed 0x00), GREEN after.\n- Suites: controller testsuite 1358 tests, 0 failures (build.dbg).\n- Live: t1 E2E — config gen=15 push with extlic; xceesd push landed in\n sharedinst (extlic=4), rusage[extlic=3] job RUNNED; second job PENDed at\n free=1; release re-admitted it.",
|
|
"tags": [
|
|
"config",
|
|
"reconcile",
|
|
"E2",
|
|
"external-license"
|
|
],
|
|
"related_bugs": [
|
|
"bug-1854"
|
|
],
|
|
"fix_commits": [
|
|
"2fcacac2"
|
|
],
|
|
"record": ".wolf/bugs/bug-1973.md"
|
|
},
|
|
{
|
|
"id": "bug-1974",
|
|
"status": "fixed",
|
|
"status_updated": "2026-10-27",
|
|
"status_note": "",
|
|
"component": "controller",
|
|
"severity": "wrong-result",
|
|
"timestamp": "2026-10-27T03:35:12Z",
|
|
"last_seen": "2026-10-27T03:35:12Z",
|
|
"occurrences": 1,
|
|
"error_message": "config activation kept stale resource definition (flags=0): daemon pushes refused with -EPERM\n\n**Observed:**\nController log on t1 after re-pushing a corrected cluster.bin (extlic mapped\nwith `location: [external]` → flags SHARED|GLOBAL|EXTERNAL):\n\n```\n[ERROR] [log_update_resval_failure] update_resval `extlic': Operation not permitted\n```\n\n**Expected:** update_val_res' SHARED branch routes a host==\"\" resval for a\nSHARED-flagged resource into c->sharedinst; with correct flags the push lands.\n\n**Impact:** the extres daemon's pushes were permanently refused on a controller\nwhose registration predated the flag fix — the stale shape was frozen for the\nprocess lifetime, so re-pushing a corrected config did not heal it.\n\n**Found by:** E2 Phase 1 live E2E on t1.",
|
|
"file": "src/controller/cluster.c, src/controller/worker.c, src/xceadmin/src/xcecommon/mapping.py",
|
|
"root_cause": "Two compounding factors:\n1. **Producer**: the DEPLOYED xceadmin venv (/opt/xce-denix/share/xce/\n xceadmin-venv) carries a stale mapping.py with NO resourcemap handling\n (83-line diff vs tree) — cluster bins it builds carry flags=0 for every\n resource. Ops issue, fixed by building with the tree venv\n (src/build.dbg/xceadmin/.venv/bin/xceadmin); the deployed venv still needs\n redeployment.\n2. **Controller**: even with a corrected bin, `cluster_build_resources()`\n treated an already-registered name as \"nothing to do\" (bug-1854's\n RES_APPLY_CHANGED was reported but never applied), so the stale flags=0\n shape survived every activation.",
|
|
"fix": "`cluster_build_resources()` now applies RES_APPLY_CHANGED by updating the\nregistered object's definition fields (valtype/ordtype/flags/interval) in\nplace under the resmgr write lock — none of them touches the registration\nslot (idx) or accumulated accounting; mirrors reconcile_config_queues'\n\"updated by configuration activation\" in-place refresh precedent. (Same loop\nas bug-1973's fix.) Producer side tracked separately as ops follow-up:\nredeploy the xceadmin venv.\n\nVerification:\n- Test: `test_reconcile_resources_hot_applies_changed_definition`\n (src/controller/testsuite/test_config_resource_reconcile.c); RED before\n (flags stayed 0x00), GREEN after (0x1C, idx preserved).\n- Suites: controller testsuite 1358 tests, 0 failures (build.dbg).\n- Live: t1 — bin rebuilt with tree venv (gen=15) + fix deployed; xceesd push\n landed (extlic=4 in `resources distribution list`), rusage[extlic=3] job RUN.",
|
|
"tags": [
|
|
"config",
|
|
"flags",
|
|
"E2",
|
|
"external-license"
|
|
],
|
|
"related_bugs": [
|
|
"bug-1854",
|
|
"bug-1973"
|
|
],
|
|
"fix_commits": [
|
|
"2fcacac2"
|
|
],
|
|
"record": ".wolf/bugs/bug-1974.md"
|
|
},
|
|
{
|
|
"id": "bug-1975",
|
|
"status": "fixed",
|
|
"status_updated": "2026-10-27",
|
|
"status_note": "",
|
|
"component": "controller",
|
|
"severity": "wrong-result",
|
|
"timestamp": "2026-10-27T03:40:05Z",
|
|
"last_seen": "2026-10-27T03:40:05Z",
|
|
"occurrences": 1,
|
|
"error_message": "xceesd idle on dead connection after controller restart (stable value: no re-push)\n\n**Observed:**\nOld xceesd (pid 162609) sat idle on a dead connection for 50+ minutes after\nthe controller was restarted:\n\n```\n(no socket() attempts; strace showed only collector forks + epoll_wait;\n /proc/net socket table had no trace of the controller connection)\n```\n\nLast log line was the boot-time `esd full re-push: 0/1 source(s)` — no\nfurther activity.\n\n**Expected:** within one poll cycle of the controller's restart the daemon\nmust re-establish the connection and re-push its value (the new primary has\nno record of the daemon or its value).\n\n**Impact:** after any controller restart/failover with a stable license\nvalue, the controller restarts with extlic read 0 (seed skipped for EXTERNAL\nby design) and no daemon re-push — every rusage[extlic] job PENDs forever\nwith NO_RESOURCE until the license value happens to change.\n\n**Found by:** E2 Phase 1 live E2E on t1 (controller restart step).",
|
|
"file": "src/esd/src/esd.c, src/lib/client.c, src/lib/client_internal.h",
|
|
"root_cause": "Three mechanisms conspired:\n1. `esd_poll_round()` debounces unchanged values (`last_value == v` → skip\n push) — with a stable value it pushes NOTHING.\n2. A reconnect inside the client library only fires while INSIDE a push's\n failover call — no pushes, no reconnect.\n3. `esd_on_reconnect` (the rejoin hook) only runs on a successful rotate;\n with no push attempt it never runs.",
|
|
"fix": "`esd_run()`'s main loop now checks `xcecli_is_connection_broken()`\n(bug-1760's pure predicate — the transport's RDHUP handler sets\nXCE_CTX_ERR asynchronously) and forces one `esd_push_all()` when it latches.\nThe write attempt drives the rotate/reconnect; `esd_on_reconnect` keeps the\nfull re-push contract for the failover path.\n\nVerification:\n- Test: covered by the live contract (client-side predicate already unit\n tested by bug-1760's suite); esd-side loop is integration-shaped (the\n trigger is a transport callback), so live E2E is the proof.\n- Suites: controller testsuite 1358 tests, 0 failures (build.dbg).\n- Live: t1 — controller killed and restarted; esd logged\n `connection broken -- forcing a full re-push to reconnect`, rotated to the\n new primary, re-pushed; `xcecli resources distribution list` shows extlic=4\n and a fresh rusage[extlic=3] job RUNS.",
|
|
"tags": [
|
|
"esd",
|
|
"external-license",
|
|
"reconnect",
|
|
"E2"
|
|
],
|
|
"related_bugs": [
|
|
"bug-1760",
|
|
"bug-1973",
|
|
"bug-1974"
|
|
],
|
|
"fix_commits": [
|
|
"2fcacac2"
|
|
],
|
|
"record": ".wolf/bugs/bug-1975.md"
|
|
},
|
|
{
|
|
"id": "bug-1976",
|
|
"status": "fixed",
|
|
"status_updated": "2026-10-28",
|
|
"status_note": "",
|
|
"component": "controller",
|
|
"severity": "wrong-result",
|
|
"timestamp": "2026-10-28T05:37:09Z",
|
|
"last_seen": "2026-10-28T05:26:00Z",
|
|
"occurrences": 1,
|
|
"error_message": "controller dispatch commit drops allocs_shared — vres usage never charged, limits unenforced\n\n**Observed:**\nE2 Phase 3 t1 E2E: 4 projA jobs with `-R \"rusage[extlic=1]\"` against a\nper-project limit `extlic_share` (scope projA, cap 3.0) — ALL FOUR ran\ninstead of the 4th PENDing with reason limit_global. Live gdb on the\nscheduler showed `sharedInUse[36/37/38]` (the vres instances) = 0.0 with\n3 projA jobs RUN: the vres charge never landed.\n\n**Expected:** the 4th projA job PENDs (limit_global); killing a RUN job\nre-admits it; projB unaffected (own per-project instance).\n\n**Impact:** every scheduler-side global limit (the limit vres mechanism)\nwas advisory only — no quota was ever enforced, for ANY shared resource\ncarried via allocs_shared, since dispatch-time adoption overwrote the\nscheduler's own charged accounts with a shared-stripped copy.\n\n**Found by:** spec task (external-license-resources E2 Phase 3 E2E).",
|
|
"file": "src/controller/job.c, src/controller/testsuite/test_dispatch_commit.c",
|
|
"root_cause": "The controller's dispatch-time commit (`dispatch_a_job_()`, job.c)\nrebuilt `j->job.allocs` from the scheduler's dispatch payload but\nmigrated ONLY the per-host records (`MOVE_PTR(j->job.allocs->allocs,\nallocs)`): `nres_shared` stayed 0 and the payload's `allocs_shared`\nwas neither moved nor freed (a leak on top). That rebuilt record is\nwhat `sched_job_update_fun()` re-sends to the scheduler as ground\ntruth, and the scheduler's JOB_UPDATE adoption (bug-1860 branch,\ncore_events.c) rebuilds `job->accounts` from exactly that wire copy —\nso `accounts_shared == NULL`, `sched_account_jobs` charges nothing to\nthe vres, and `unsatisfy_` sees cap ≥ 0 + want forever. The scheduler's\nown commit (`sched_commit_decision`, scheduler.c:1806-1812) built the\naccounts correctly — the controller's \"ground truth\" clobbered it.",
|
|
"fix": "job.c dispatch-time commit: migrate the shared records alongside the\nhost records — `MOVE_PTR(j->job.allocs->allocs_shared,\nalloc->allocs_shared); j->job.allocs->nres_shared =\nalloc->nres_shared; alloc->nres_shared = 0;` (MOVE, not copy — the\ncaller frees the payload; also closes the leak).\n\nVerification:\n- Test: `test_dc_success_migrates_shared_allocs`\n (src/controller/testsuite/test_dispatch_commit.c). RED before the fix\n (\"Expected 2 Was 0\" on nres_shared), GREEN after.\n- Suites: full meson test 136/136 OK (build.dbg), including controller\n unit suite 1361/0.\n- Live: t1 E2E — 3 projA RUN / 4th PEND, projB RUN (own instance), kill\n one RUN job → pending projA re-admitted, external free pool\n (fake_lic) enforced independently; all daemons healthy throughout.",
|
|
"tags": [
|
|
"external-license-resources",
|
|
"limits",
|
|
"shared-resources",
|
|
"dispatch",
|
|
"e2-phase3"
|
|
],
|
|
"related_bugs": [
|
|
"bug-1860"
|
|
],
|
|
"fix_commits": [
|
|
"76ddd75f"
|
|
],
|
|
"record": ".wolf/bugs/bug-1976.md"
|
|
},
|
|
{
|
|
"id": "bug-1977",
|
|
"status": "fixed",
|
|
"status_updated": "2026-10-28",
|
|
"status_note": "",
|
|
"component": "controller",
|
|
"severity": "crash",
|
|
"timestamp": "2026-10-28T11:56:58Z",
|
|
"last_seen": "2026-10-28T11:56:58Z",
|
|
"occurrences": 1,
|
|
"error_message": "crr_build_config callers' stack arrays smaller than builder -- ASAN stack overflow\n\n**Observed:**\n```\nERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffc...\nWRITE of size 8 at 0x7ffc... thread T0\n #0 crr_build_config testsuite/test_config_resource_reconcile.c:126 (dists[3] = &d4)\n #1 test_reconcile_resources_registers_and_seeds_shared_pools testsuite/test_config_resource_reconcile.c:161\n```\nEvery test in the file that calls `crr_build_config` crashes the same way;\nthe full controller suite reports 137/137 FAIL (meson reports each fixture\ncrash as its own failure).\n\n**Expected:** all four callers pass arrays large enough for the builder's\nfixtures; the suite stays green.\n\n**Impact:** the whole controller testsuite was unrunnable under ASAN since the\nexternal-license-resources test fixtures landed (commit 2fcacac2) -- any\nregression in the suite was invisible behind the deterministic crash.\n\n**Found by:** ASAN -- full regression build for the license-resource feature\n(`src/tools/xce-build.sh controller`, profile=asan).",
|
|
"file": "src/controller/testsuite/test_config_resource_reconcile.c",
|
|
"root_cause": "`crr_build_config` (test_config_resource_reconcile.c:70) registers FIVE\nresources (lic1/hostres/dyn1/extlic/slots -> writes `resources[4]`) and FOUR\ndistributions (writes `dists[3]`) and FIVE vals (writes `vals[4]`). The four\ncaller test functions declared only\n`resources[4] / dists[3] / vals[4]`, so the builder's writes to index 4 / 3 /\n4 overflowed the callers' stack arrays. When the extlic E2 tests were added to\nthe builder's fixture set (2fcacac2), none of the caller declarations were\nresized. (Earlier builds of this branch compiled only a worker-focused subset\n-- `test_config_resource_reconcile` was not linked -- so the overflow went\nunnoticed until the full controller suite was built.)",
|
|
"fix": "Resize all four caller arrays to match the builder: `resources[5]`,\n`dists[4]`, `vals[5]`, each with a bug-1977 comment pointing at the sizing\ncontract. Files: src/controller/testsuite/test_config_resource_reconcile.c\n(test_reconcile_resources_registers_and_seeds_shared_pools,\ntest_reconcile_resources_registers_new_defs_past_eexist,\ntest_reconcile_resources_hot_applies_changed_definition,\ntest_reconcile_resources_is_idempotent_and_hot_applies).\n\nVerification:\n- Test: whole-file suite `controller - xce:controller test` (meson): RED\n 137/137 before the fix, GREEN after.\n- Suites: `src/tools/xce-build.sh controller` profile=asan: 6/6 OK\n (sched push, sched worker change, apiserver push, host runusage, no-fork\n rule, controller 91s).\n- Live: n/a (testsuite-only defect; no production code touched).",
|
|
"tags": [
|
|
"testsuite",
|
|
"asan"
|
|
],
|
|
"related_bugs": [],
|
|
"fix_commits": [
|
|
"8c1995c2"
|
|
],
|
|
"record": ".wolf/bugs/bug-1977.md"
|
|
}
|
|
]
|
|
}
|