build: recalibrate asan per-job memory budget 1536MB -> 512MB
The original 1536MB/job figure was calibrated around two memory hogs that the build/de-nix work has since removed: the 293MB-peak serial capnp compile (now compiled once via libxce_dep) and the 17 scheduler test executables each statically linking all of scheduler_srcs (now one shared sched_srcs_lib archive; link peaks ~125-150MB with -g1). Measured per-job compile RSS ceiling post-dedup is ~125-150MB; 512MB keeps ~3x headroom while letting an 8GB box run ~8 concurrent compiles instead of 2-3 (t1: mem_jobs goes from ~2-3 to ~9, so the -l ceiling becomes CPU-bound rather than memory-bound). SKILL.md's hand-computed formula is updated to match. Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
parent
e723c9c234
commit
28ccf8d35c
2 changed files with 12 additions and 2 deletions
|
|
@ -21,7 +21,7 @@ Args: `[component] [profile|build_dir] [build_dir] [--incremental]` — passed a
|
||||||
awk '/MemAvailable/ {print int($2/1024)}' /proc/meminfo # available memory, MB (reclaimable-aware, unlike `free`'s raw "free" column)
|
awk '/MemAvailable/ {print int($2/1024)}' /proc/meminfo # available memory, MB (reclaimable-aware, unlike `free`'s raw "free" column)
|
||||||
cut -d' ' -f1 /proc/loadavg # current 1-min load average = existing run-queue length from other activity
|
cut -d' ' -f1 /proc/loadavg # current 1-min load average = existing run-queue length from other activity
|
||||||
```
|
```
|
||||||
2. Pick a per-job memory budget by profile — ASAN+debug is the heaviest (debug info + sanitizer instrumentation per translation unit): **~1536 MB/job for `asan`** (the default profile), **~1024 MB/job for `debug`/`release`**.
|
2. Pick a per-job memory budget by profile — ASAN+debug is the heaviest (debug info + sanitizer instrumentation per translation unit): **~512 MB/job for `asan`** (the default profile; recalibrated in build/de-nix after the capnp compile-once and scheduler test-archive dedup dropped the measured per-job RSS ceiling to ~125-150MB — the old 1536 was calibrated around the 293MB-peak serial capnp compile that no longer exists), **~1024 MB/job for `debug`/`release`**.
|
||||||
3. Compute, reserving ~15% of available memory as headroom for the OS and other sessions, and dampening the CPU term for how many *live OS processes* one "job" actually costs:
|
3. Compute, reserving ~15% of available memory as headroom for the OS and other sessions, and dampening the CPU term for how many *live OS processes* one "job" actually costs:
|
||||||
- `mem_jobs = floor(available_mem_mb * 0.85 / per_job_mb)`
|
- `mem_jobs = floor(available_mem_mb * 0.85 / per_job_mb)`
|
||||||
- `PROCESS_FACTOR = 0.5` for `asan`/`debug` profiles, `0.7` for `release` — each in-flight ninja job is not one process: the compiler driver (`cc`/`c++`) forks a child compiler (`cc1`/`cc1plus`) and then an assembler (`as`), and `asan`/`debug` stretch out how long those extra children stay resident (sanitizer instrumentation, full debug-info emission) versus `release`'s leaner, shorter-lived ones. `nproc - current_load_avg` alone treats 1 job ≈ 1 process, so it under-throttles by roughly this factor — and `current_load_avg` is a smoothed 1-minute average, so it lags a fresh burst of forked children rather than pre-empting it.
|
- `PROCESS_FACTOR = 0.5` for `asan`/`debug` profiles, `0.7` for `release` — each in-flight ninja job is not one process: the compiler driver (`cc`/`c++`) forks a child compiler (`cc1`/`cc1plus`) and then an assembler (`as`), and `asan`/`debug` stretch out how long those extra children stay resident (sanitizer instrumentation, full debug-info emission) versus `release`'s leaner, shorter-lived ones. `nproc - current_load_avg` alone treats 1 job ≈ 1 process, so it under-throttles by roughly this factor — and `current_load_avg` is a smoothed 1-minute average, so it lags a fresh burst of forked children rather than pre-empting it.
|
||||||
|
|
|
||||||
|
|
@ -157,6 +157,16 @@ else
|
||||||
# discounting nproc-minus-load-average for the compiler driver's forked
|
# discounting nproc-minus-load-average for the compiler driver's forked
|
||||||
# cc1/cc1plus + as children (asan/debug keep those resident far longer
|
# cc1/cc1plus + as children (asan/debug keep those resident far longer
|
||||||
# than release's leaner ones).
|
# than release's leaner ones).
|
||||||
|
#
|
||||||
|
# asan's 512MB budget (build/de-nix): the two memory hogs the original
|
||||||
|
# 1536MB figure was calibrated around are both gone -- the 293MB-peak
|
||||||
|
# serial capnp compile (批1: now compiled once, still far below the old
|
||||||
|
# budget) and the 17 scheduler test executables each statically linking
|
||||||
|
# all of scheduler_srcs (批2: shared sched_srcs_lib, link is the only
|
||||||
|
# heavy step and lld/ld peak ~125-150MB with -g1 debug info). Measured
|
||||||
|
# per-job RSS ceiling on t1 post-dedup is ~125-150MB; 512MB leaves ~3x
|
||||||
|
# headroom for a future pathological TU while letting an 8GB box run ~8
|
||||||
|
# concurrent compiles instead of 2-3.
|
||||||
default_nproc="$(nproc --all 2>/dev/null || echo 1)"
|
default_nproc="$(nproc --all 2>/dev/null || echo 1)"
|
||||||
default_mem_mb="$(awk '/MemAvailable/ {print int($2/1024)}' /proc/meminfo 2>/dev/null)"
|
default_mem_mb="$(awk '/MemAvailable/ {print int($2/1024)}' /proc/meminfo 2>/dev/null)"
|
||||||
default_load="$(cut -d' ' -f1 /proc/loadavg 2>/dev/null)"
|
default_load="$(cut -d' ' -f1 /proc/loadavg 2>/dev/null)"
|
||||||
|
|
@ -164,7 +174,7 @@ else
|
||||||
: "${default_load:=0}"
|
: "${default_load:=0}"
|
||||||
|
|
||||||
case "$PROFILE" in
|
case "$PROFILE" in
|
||||||
asan) default_per_job_mb=1536; default_process_factor=0.5 ;;
|
asan) default_per_job_mb=512; default_process_factor=0.5 ;;
|
||||||
debug) default_per_job_mb=1024; default_process_factor=0.5 ;;
|
debug) default_per_job_mb=1024; default_process_factor=0.5 ;;
|
||||||
release) default_per_job_mb=1024; default_process_factor=0.7 ;;
|
release) default_per_job_mb=1024; default_process_factor=0.7 ;;
|
||||||
esac
|
esac
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue