diff --git a/.wolf/buglog.json b/.wolf/buglog.json index 237d5975..c1351304 100644 --- a/.wolf/buglog.json +++ b/.wolf/buglog.json @@ -6955,15 +6955,15 @@ }, { "id": "bug-1553", - "status": "open", - "status_updated": "2026-09-24", - "status_note": "intermittent (~1 in 3) controller_unit_test SIGSEGV; not isolated, re-seen 5bf123cc", + "status": "cannot-reproduce", + "status_updated": "2026-09-30", + "status_note": "0/26 controller_unit_test runs crashed on dc94da20, 2026-09-29; likely fixed by a later race fix (bug-1733?)", "component": "controller", "severity": "crash", "timestamp": "2026-08-13T07:35:00Z", "last_seen": "2026-09-19T09:00:00Z", "occurrences": 1, - "error_message": "controller_unit_test (combined 1095-test Unity binary) segfaults intermittently (~1 in 3 full runs…\n\ncontroller_unit_test (combined 1095-test Unity binary) segfaults intermittently (~1 in 3 full runs observed) somewhere between test_drive_async_config.c and the next testsuite file (test_configcompare.c/test_slowlane.c area) -- no assertion failure, no ASAN report captured before the crash, process just dies (exit 139/SIGSEGV)", + "error_message": "controller_unit_test (combined 1095-test Unity binary) segfaults intermittently (~1 in 3 full runs…\n\ncontroller_unit_test (combined 1095-test Unity binary) segfaults intermittently (~1 in 3 full runs observed) somewhere between test_drive_async_config.c and the next testsuite file (test_configcompare.c/test_slowlane.c area) -- no assertion failure, no ASAN report captured before the crash, process just dies (exit 139/SIGSEGV)\n\nUPDATE (2026-09-30): Not seen since August. On 2026-09-29 (dc94da20 + cfa0a941, asan) controller_unit_test ran 26 times -- 20 standalone runs (ASAN_OPTIONS=detect_leaks=0) plus 6 full-suite runs via xce-build.sh -- all exit 0, 1256/1258 Tests 0 Failures, no SIGSEGV. At the recorded ~1-in-3 rate, 20 clean standalone runs in a row has probability ~0.03%. Suspected (not proven): fixed by one of the thread-lifetime/race fixes landed since, most plausibly bug-1733 (42e1eeaa, 2026-08-29: vtask_enq wrote task->owner after publishing the task, a use-after-free that would surface as a bare SIGSEGV outside ASAN), or bug-1839/bug-1851 (controller UAFs, Sep 22/24). Proving which would need a build of an August commit reproducing ~1/3 and bisecting.", "file": "src/controller/testsuite/ (combined binary, crash site not yet isolated -- occurs after test_drive_async_config.c's liveness_reserved_* tests)", "root_cause": "Not yet investigated. Reproduced 1/3 runs both BEFORE and AFTER an unrelated worker.c change (update_resval PERROR->rc logging fix) confirming this is pre-existing and unrelated to that fix -- likely order-dependent shared-process state leakage across Unity tests (global fixtures, xstring table, or ASAN heap state) rather than a bug in any specific test file. Two of three runs completed cleanly with 0 failures each time.", "fix": "Not fixed -- out of scope for the update_resval/mxj-auto-detect bugfix session that surfaced it. Needs isolation: bisect which specific test pair triggers it (likely by running testsuite files in smaller batches or under gdb with a core dump), then root-cause the actual memory-safety issue.", @@ -6976,7 +6976,8 @@ ], "related_bugs": [], "fix_commits": [], - "record": ".wolf/bugs/bug-1553.md" + "record": ".wolf/bugs/bug-1553.md", + "repro_attempts": "0/26 runs, 2026-09-29 (20 standalone ASAN_OPTIONS=detect_leaks=0 + 6 full-suite via xce-build.sh, dc94da20, asan)" }, { "id": "bug-1554", @@ -7762,15 +7763,15 @@ }, { "id": "bug-1580", - "status": "open", - "status_updated": "2026-09-24", - "status_note": "intermittent controller_unit_test \"2 Failures\" with all cases PASS; not investigated", + "status": "cannot-reproduce", + "status_updated": "2026-09-30", + "status_note": "0/26 runs with failures on dc94da20, 2026-09-29; original ':FAIL:' grep cannot match Unity's colored FAIL lines", "component": "controller", "severity": "crash", "timestamp": "2026-08-18T10:39:52Z", "last_seen": "2026-09-19T09:00:00Z", "occurrences": 1, - "error_message": "meson test 'xce:controller / controller test' (controller_unit_test) reports '1104 Tests 2…\n\nmeson test 'xce:controller / controller test' (controller_unit_test) reports '1104 Tests 2 Failures 0 Ignored' / exit status 2, but grep of the full raw testlog for ':FAIL:' or a per-test [FAIL] marker finds ZERO matches -- every individual Unity test case visibly printed PASS. Reproduced deterministically 2/2 times (once via meson test, once via a direct standalone re-run of the binary with ASAN_OPTIONS=detect_leaks=0), unlike bug-1553's documented ~1-in-3 intermittent SIGSEGV (exit 139) signature -- this is exit 2 with no crash indication, a distinct manifestation.", + "error_message": "meson test 'xce:controller / controller test' (controller_unit_test) reports '1104 Tests 2…\n\nmeson test 'xce:controller / controller test' (controller_unit_test) reports '1104 Tests 2 Failures 0 Ignored' / exit status 2, but grep of the full raw testlog for ':FAIL:' or a per-test [FAIL] marker finds ZERO matches -- every individual Unity test case visibly printed PASS. Reproduced deterministically 2/2 times (once via meson test, once via a direct standalone re-run of the binary with ASAN_OPTIONS=detect_leaks=0), unlike bug-1553's documented ~1-in-3 intermittent SIGSEGV (exit 139) signature -- this is exit 2 with no crash indication, a distinct manifestation.\n\nUPDATE (2026-09-30): The original evidence does not hold: controller_unit_test prints colored Unity output, where a failure line is `file:line:test:\\e[41mFAIL\\e[0m: msg`. A grep for `:FAIL:` can never match it, so \"2 Failures but zero FAIL markers\" most likely meant two ordinary test failures the grep could not see, not a counter desync; the original log was not kept, so which tests is unknown. On 2026-09-29 (dc94da20 + cfa0a941, asan) controller_unit_test ran 26 times (20 standalone, 6 via xce-build.sh): every run exit 0, 0 Failures, and the printed per-test results agree with the summary. To look for failures in these logs, grep for `FAIL` (or strip ANSI codes first), not `:FAIL:`.", "file": "src/controller/testsuite/ (combined controller_unit_test binary; discovered running xce-build.sh all for the resource-value-verification bug-1579 follow-up fix -- zero C files touched this session, confirmed via git status)", "root_cause": "Not investigated. Most likely the same class of issue cerebrum.md's Unity-harness note already flags for this codebase (multiple UnitySetTestFile()-sharing test-group files in one binary can desync the aggregate PASS/FAIL counter from the printed per-test lines) rather than a real regression -- no code change in this session could have caused it (only .claude/skills/xce-test-resources/run-resource-verify.sh, a bash script, was edited). Possibly related to / the same underlying fragility as bug-1553 (order-dependent shared Unity/ASAN state across test files in the same binary) but with a different observed symptom (miscounted summary vs. an outright segfault).", "fix": "Not fixed -- out of scope for the resource-value-verification bug-1579 follow-up (bash-script-only boundary). Verified NOT a regression from this session's change (no C files modified) and does not block or relate to the run-resource-verify.sh live validation, which was carried out and passed independently. Needs the same isolation bug-1553 needs: bisect which specific linked test-group file(s) desync the counter.", @@ -7785,7 +7786,8 @@ "bug-1553" ], "fix_commits": [], - "record": ".wolf/bugs/bug-1580.md" + "record": ".wolf/bugs/bug-1580.md", + "repro_attempts": "0/26 runs, 2026-09-29 (20 standalone + 6 full-suite, dc94da20, asan)" }, { "id": "bug-1581", diff --git a/.wolf/bugs/bug-1553.md b/.wolf/bugs/bug-1553.md index 75dd7926..b91bdc30 100644 --- a/.wolf/bugs/bug-1553.md +++ b/.wolf/bugs/bug-1553.md @@ -1,8 +1,8 @@ --- id: "bug-1553" -status: "open" -status_updated: "2026-09-24" -status_note: "intermittent (~1 in 3) controller_unit_test SIGSEGV; not isolated, re-seen 5bf123cc" +status: "cannot-reproduce" +status_updated: "2026-09-30" +status_note: "0/26 controller_unit_test runs crashed on dc94da20, 2026-09-29; likely fixed by a later race fix (bug-1733?)" component: "controller" severity: "crash" timestamp: "2026-08-13T07:35:00Z" @@ -11,6 +11,7 @@ occurrences: 1 files: ["src/controller/testsuite/ (combined binary", "crash site not yet isolated -- occurs after test_drive_async_config.c's liveness_reserved_* tests)"] tags: ["flaky", "segfault", "controller_unit_test", "order-dependent", "unity"] related: [] +repro_attempts: "0/26 runs, 2026-09-29 (20 standalone ASAN_OPTIONS=detect_leaks=0 + 6 full-suite via xce-build.sh, dc94da20, asan)" legacy: true --- @@ -19,6 +20,8 @@ legacy: true ## Description controller_unit_test (combined 1095-test Unity binary) segfaults intermittently (~1 in 3 full runs observed) somewhere between test_drive_async_config.c and the next testsuite file (test_configcompare.c/test_slowlane.c area) -- no assertion failure, no ASAN report captured before the crash, process just dies (exit 139/SIGSEGV) +UPDATE (2026-09-30): Not seen since August. On 2026-09-29 (dc94da20 + cfa0a941, asan) controller_unit_test ran 26 times -- 20 standalone runs (ASAN_OPTIONS=detect_leaks=0) plus 6 full-suite runs via xce-build.sh -- all exit 0, 1256/1258 Tests 0 Failures, no SIGSEGV. At the recorded ~1-in-3 rate, 20 clean standalone runs in a row has probability ~0.03%. Suspected (not proven): fixed by one of the thread-lifetime/race fixes landed since, most plausibly bug-1733 (42e1eeaa, 2026-08-29: vtask_enq wrote task->owner after publishing the task, a use-after-free that would surface as a bare SIGSEGV outside ASAN), or bug-1839/bug-1851 (controller UAFs, Sep 22/24). Proving which would need a build of an August commit reproducing ~1/3 and bisecting. + ## Environment _Not recorded (migrated from buglog.json)._ diff --git a/.wolf/bugs/bug-1580.md b/.wolf/bugs/bug-1580.md index 097245c4..1334ea53 100644 --- a/.wolf/bugs/bug-1580.md +++ b/.wolf/bugs/bug-1580.md @@ -1,8 +1,8 @@ --- id: "bug-1580" -status: "open" -status_updated: "2026-09-24" -status_note: "intermittent controller_unit_test \"2 Failures\" with all cases PASS; not investigated" +status: "cannot-reproduce" +status_updated: "2026-09-30" +status_note: "0/26 runs with failures on dc94da20, 2026-09-29; original ':FAIL:' grep cannot match Unity's colored FAIL lines" component: "controller" severity: "crash" timestamp: "2026-08-18T10:39:52Z" @@ -11,6 +11,7 @@ occurrences: 1 files: ["src/controller/testsuite/ (combined controller_unit_test binary; discovered running xce-build.sh all for the resource-value-verification bug-1579 follow-up fix -- zero C files touched this session", "confirmed via git status)"] tags: ["flaky", "controller_unit_test", "unity", "aggregate-count-mismatch", "pre-existing"] related: ["bug-1553"] +repro_attempts: "0/26 runs, 2026-09-29 (20 standalone + 6 full-suite, dc94da20, asan)" legacy: true --- @@ -19,6 +20,8 @@ legacy: true ## Description meson test 'xce:controller / controller test' (controller_unit_test) reports '1104 Tests 2 Failures 0 Ignored' / exit status 2, but grep of the full raw testlog for ':FAIL:' or a per-test [FAIL] marker finds ZERO matches -- every individual Unity test case visibly printed PASS. Reproduced deterministically 2/2 times (once via meson test, once via a direct standalone re-run of the binary with ASAN_OPTIONS=detect_leaks=0), unlike bug-1553's documented ~1-in-3 intermittent SIGSEGV (exit 139) signature -- this is exit 2 with no crash indication, a distinct manifestation. +UPDATE (2026-09-30): The original evidence does not hold: controller_unit_test prints colored Unity output, where a failure line is `file:line:test:\e[41mFAIL\e[0m: msg`. A grep for `:FAIL:` can never match it, so "2 Failures but zero FAIL markers" most likely meant two ordinary test failures the grep could not see, not a counter desync; the original log was not kept, so which tests is unknown. On 2026-09-29 (dc94da20 + cfa0a941, asan) controller_unit_test ran 26 times (20 standalone, 6 via xce-build.sh): every run exit 0, 0 Failures, and the printed per-test results agree with the summary. To look for failures in these logs, grep for `FAIL` (or strip ANSI codes first), not `:FAIL:`. + ## Environment _Not recorded (migrated from buglog.json)._