chore: update OpenWolf session state + cerebrum learnings (meson parallel-timeout false-positive, /tmp/xce_* cleanup)

This commit is contained in:
Rongsong Shen 2026-07-17 13:07:00 +00:00
parent 3f54ef307c
commit 6592f9cbf1
4 changed files with 722 additions and 8 deletions

View file

@ -464,3 +464,9 @@ Post-failover `expect-job`/`joblist` calls got a genuine, persistent `-XCE_ERR_S
## Do-Not-Repeat (2026-07-15, bug-1440) — a `_test.go` file can NEVER directly `import "C"`, regardless of CGO_ENABLED/nix env/build tags; this is a hard Go toolchain rule, not a build-config problem
`src/frontend/apiserver/backend/wire_dispatch_test.go` (new, `//go:build cgo`) hand-builds real C structs (`xce_job_submit_t`, `xce_data_t`, `xce_request_t`) via `import "C"` directly inside the `_test.go` file itself, to drive the real `C.xce_encode`/`C.xce_decode`/`C.extract_data`/`go_push_handler()` chain. This fails immediately with `use of cgo in test wire_dispatch_test.go not supported` under both `make test-backend` and a manual `CGO_ENABLED=1 go test ./backend/... -tags cgo`. Confirmed by reading the actual installed Go 1.25.5 source in the nix shell (`$(go env GOROOT)/src/cmd/go/internal/modindex/read.go` ~line 589): the module indexer explicitly special-cases `imp.path == "C"` when `isTest` is true and calls `badGoFile(...)`, unconditionally — this is golang/go#13938, a permanent Go toolchain restriction with NO environment/flag workaround. CGO directives (`/* #include ... */` comments) in a test file are NOT what triggers it — only the literal `import "C"` statement is. Since 4 of this package's pre-existing test files carry no build tag at all (`handler_test.go`, `convert_test.go`, `peers_test.go`, `heartbeat_test.go` — they build under both cgo and !cgo), one bad cgo-in-test file poisons the WHOLE `apiserver/backend` package build under CGO_ENABLED=1 (`[setup failed]`), silently taking those otherwise-unrelated tests down with it; the `!cgo`-tagged files (`integration_test.go`, `control_host_test.go`, `stub_funcs_test.go`) and the plain `make test` (CGO_ENABLED=0) path are unaffected since the poisoned file is excluded by its own `//go:build cgo` tag in that mode. **Do-not-repeat**: any new CGO-touching Go test in this codebase must put the `import "C"` and all `C.xxx` calls in a plain (non-`_test.go`-suffixed) `//go:build cgo` helper file in the same package, and have the actual `_test.go` file call into pure-Go-signature wrapper functions only — this is the same `!cgo`/`cgo`-split pattern already established for clib.go/convert.go/convert_pure.go, just also required WITHIN the cgo side to keep the literal `import "C"` out of any `_test.go` filename. Because the toolchain rejects the file before type-checking, a fix along these lines may still surface FURTHER errors (undefined C symbols, type mismatches) that were never reached in this run.
## Do-Not-Repeat (2026-07-17) — `xce-build.sh all` / plain `meson test` can report a suite as TIMEOUT purely from meson's default parallel test scheduling, not a real hang/regression
During bug-1445's post-fix regression pass, `nix develop ./src --command bash src/tools/xce-build.sh all` reported `controller test` as a 300s TIMEOUT under meson's default *parallel* scheduling (multiple ASAN-instrumented binaries — lib/controller/scheduler suites — contending for CPU simultaneously on the sandbox's shared cores). Also independently observed during the controller-status validation pass: a genuine ASAN SEGV in an unrelated test file (`test_persist_truncate.c`) traced to a SECOND, concurrent session running its own build+HA-soak against the SAME worktree/build directory at the same time (confirmed live via `ps aux`). **Do-not-repeat**: before concluding a code regression from an "all"-scope build+test run that times out or crashes in a way that doesn't reproduce per-file, (1) check for a competing process against the same build dir (`ps aux` / `pgrep -af meson\|ninja\|xce_controller`), and (2) rerun the specific suite in isolation, and if still ambiguous rerun the full suite serially (`meson test --num-processes 1` from the build dir) — a serial rerun matching the isolated timing exactly (both incidents here: clean 9/9, controller test ~34s, matching its solo-run time) confirms resource contention, not a real hang or crash, before spending more time chasing a phantom regression.
## Key Learning (2026-07-17) — `/tmp/xce_*` WAL test-fixture dirs accumulate fast across repeated ASAN cycles; same root pattern as bug-924
Found 2089 leftover `/tmp/xce_*` directories (~89GB) during the bug-1445 regression pass, left behind by C testsuite fixtures that `mkdtemp("/tmp/xce_*_XXXXXX")` a workdir and don't always clean it up on every exit path (crash, ASAN abort, or a test killed mid-run). This is disk-pressure buildup of the same shape as bug-924 (overlayfs fdatasync wedge under accumulated pressure), not a new bug — just a maintenance reminder. **Key learning**: periodically `rm -rf /tmp/xce_*` between heavy test cycles in a long-lived sandbox session, especially after any run that included ASAN aborts/kills (which skip the test's own cleanup code) — cheap to do, and left unchecked it can eventually degrade or destabilize unrelated test runs sharing `/tmp`.