--- id: "bug-1604" status: "open" status_updated: "2026-09-24" status_note: "concurrent xce-build.sh runs race on rm -rf of shared BUILD_DIR (no lock)" component: "build" severity: "unknown" timestamp: "2026-08-20T10:24:39Z" last_seen: "2026-09-19T09:00:00Z" occurrences: 1 files: ["src/tools/xce-build.sh"] tags: ["asan", "concurrency", "multi-agent-race", "overlayfs", "rm-rf", "sandbox-flake", "scheduler", "shared-build-dir", "xce-build.sh"] related: ["bug-1561"] legacy: true --- # bug-1604: rm: cannot remove 'xce.build/controller/controller_unit_test.p': Directory not empty (xce-build.sh… ## Description rm: cannot remove 'xce.build/controller/controller_unit_test.p': Directory not empty (xce-build.sh setup_build()'s rm -rf \$BUILD_DIR step aborted the script under set -euo pipefail before meson setup ran) ## Environment _Not recorded (migrated from buglog.json)._ ## Reproduction _Not recorded (migrated from buglog.json)._ ## Root cause CORRECTED after further evidence (see fix note): initial overlayfs-flakiness theory was WRONG. The real cause is xce-build.sh defaulting BUILD_DIR to the same "xce.build" for every component and every invocation, with setup_build() doing an unconditional rm -rf "$BUILD_DIR" + fresh meson setup on every non-incremental run. When two xce-build.sh invocations run concurrently in the SAME repo working directory against DIFFERENT components (observed live: this session's "xce-build.sh scheduler asan" raced against a second, independently-running "xce-build.sh controller asan" process, PID 655158, started ~1 min later in the same cwd), they contend for the identical xce.build/ path -- one process's rm -rf can hit a directory the other process is actively writing into ("Directory not empty"), and later, a second clean-build pass can silently delete and replace a just-verified binary out from under a caller who believes their build is done. Confirmed live: xce_scheduler at src/xce.build/scheduler/xce_scheduler, verified present and fresh via ls -la and 'xce_scheduler --version' immediately after this session's scheduler build/test run, was gone minutes later -- the concurrent controller build's rm -rf had removed and was in the process of regenerating xce.build/ for its own component. ## Fix No code fix applied (out of scope for the build-verification task this session was doing) -- flagging as a real, reproducible hazard rather than env flakiness. Practical mitigation for future sessions/agents: never assume xce.build/ (or any non-incremental xce-build.sh output) is stable once another concurrent xce-build.sh process might be running in the same working directory/worktree -- check 'ps aux | grep xce-build.sh' (or pass a distinct BUILD_DIR / use --incremental) before relying on a binary's presence or mtime surviving past the moment it was checked. Corrects this entry's own original overlayfs-race theory, which was a plausible but incorrect guess made before the concurrent process was discovered. ## Verification _Not recorded (migrated from buglog.json)._ ## References - Related: bug-1561