diff --git a/.claude/skills/xce-build/SKILL.md b/.claude/skills/xce-build/SKILL.md index 8a0f5a40..7dc91eeb 100644 --- a/.claude/skills/xce-build/SKILL.md +++ b/.claude/skills/xce-build/SKILL.md @@ -21,7 +21,7 @@ Args: `[component] [profile|build_dir] [build_dir] [--incremental]` — passed a awk '/MemAvailable/ {print int($2/1024)}' /proc/meminfo # available memory, MB (reclaimable-aware, unlike `free`'s raw "free" column) cut -d' ' -f1 /proc/loadavg # current 1-min load average = existing run-queue length from other activity ``` -2. Pick a per-job memory budget by profile — ASAN+debug is the heaviest (debug info + sanitizer instrumentation per translation unit): **~1536 MB/job for `asan`** (the default profile), **~1024 MB/job for `debug`/`release`**. +2. Pick a per-job memory budget by profile — ASAN+debug is the heaviest (debug info + sanitizer instrumentation per translation unit): **~512 MB/job for `asan`** (the default profile; recalibrated in build/de-nix after the capnp compile-once and scheduler test-archive dedup dropped the measured per-job RSS ceiling to ~125-150MB — the old 1536 was calibrated around the 293MB-peak serial capnp compile that no longer exists), **~1024 MB/job for `debug`/`release`**. 3. Compute, reserving ~15% of available memory as headroom for the OS and other sessions, and dampening the CPU term for how many *live OS processes* one "job" actually costs: - `mem_jobs = floor(available_mem_mb * 0.85 / per_job_mb)` - `PROCESS_FACTOR = 0.5` for `asan`/`debug` profiles, `0.7` for `release` — each in-flight ninja job is not one process: the compiler driver (`cc`/`c++`) forks a child compiler (`cc1`/`cc1plus`) and then an assembler (`as`), and `asan`/`debug` stretch out how long those extra children stay resident (sanitizer instrumentation, full debug-info emission) versus `release`'s leaner, shorter-lived ones. `nproc - current_load_avg` alone treats 1 job ≈ 1 process, so it under-throttles by roughly this factor — and `current_load_avg` is a smoothed 1-minute average, so it lags a fresh burst of forked children rather than pre-empting it. diff --git a/src/tools/xce-build.sh b/src/tools/xce-build.sh index fc77e3c3..63eb346b 100755 --- a/src/tools/xce-build.sh +++ b/src/tools/xce-build.sh @@ -157,6 +157,16 @@ else # discounting nproc-minus-load-average for the compiler driver's forked # cc1/cc1plus + as children (asan/debug keep those resident far longer # than release's leaner ones). + # + # asan's 512MB budget (build/de-nix): the two memory hogs the original + # 1536MB figure was calibrated around are both gone -- the 293MB-peak + # serial capnp compile (批1: now compiled once, still far below the old + # budget) and the 17 scheduler test executables each statically linking + # all of scheduler_srcs (批2: shared sched_srcs_lib, link is the only + # heavy step and lld/ld peak ~125-150MB with -g1 debug info). Measured + # per-job RSS ceiling on t1 post-dedup is ~125-150MB; 512MB leaves ~3x + # headroom for a future pathological TU while letting an 8GB box run ~8 + # concurrent compiles instead of 2-3. default_nproc="$(nproc --all 2>/dev/null || echo 1)" default_mem_mb="$(awk '/MemAvailable/ {print int($2/1024)}' /proc/meminfo 2>/dev/null)" default_load="$(cut -d' ' -f1 /proc/loadavg 2>/dev/null)" @@ -164,7 +174,7 @@ else : "${default_load:=0}" case "$PROFILE" in - asan) default_per_job_mb=1536; default_process_factor=0.5 ;; + asan) default_per_job_mb=512; default_process_factor=0.5 ;; debug) default_per_job_mb=1024; default_process_factor=0.5 ;; release) default_per_job_mb=1024; default_process_factor=0.7 ;; esac