feat(xce-build): discount --jobs for forked compiler/assembler children per profile

nproc - load_avg treats one ninja job as one process, but the compiler
driver forks cc1/cc1plus then as, and asan/debug keep those children
resident far longer than release's leaner ones -- under-throttling by
roughly that factor. Apply a per-profile PROCESS_FACTOR (0.5 for
asan/debug, 0.7 for release) to the CPU term before taking the min with
the memory-budget term.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RLSNJhNoULvVcUhXEFafQC
This commit is contained in:
Rongsong Shen 2026-09-15 06:35:34 +00:00
parent 0f3919f195
commit 89a3dec5d9
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2

View file

@ -22,13 +22,14 @@ Args: `[component] [profile|build_dir] [build_dir] [--incremental]` — passed a
cut -d' ' -f1 /proc/loadavg # current 1-min load average = existing run-queue length from other activity
```
2. Pick a per-job memory budget by profile — ASAN+debug is the heaviest (debug info + sanitizer instrumentation per translation unit): **~1536 MB/job for `asan`** (the default profile), **~1024 MB/job for `debug`/`release`**.
3. Compute, reserving ~15% of available memory as headroom for the OS and other sessions:
3. Compute, reserving ~15% of available memory as headroom for the OS and other sessions, and dampening the CPU term for how many *live OS processes* one "job" actually costs:
- `mem_jobs = floor(available_mem_mb * 0.85 / per_job_mb)`
- `cpu_jobs = max(1, floor(nproc - current_load_avg))` — leaves room for whatever's already running
- `PROCESS_FACTOR = 0.5` for `asan`/`debug` profiles, `0.7` for `release` — each in-flight ninja job is not one process: the compiler driver (`cc`/`c++`) forks a child compiler (`cc1`/`cc1plus`) and then an assembler (`as`), and `asan`/`debug` stretch out how long those extra children stay resident (sanitizer instrumentation, full debug-info emission) versus `release`'s leaner, shorter-lived ones. `nproc - current_load_avg` alone treats 1 job ≈ 1 process, so it under-throttles by roughly this factor — and `current_load_avg` is a smoothed 1-minute average, so it lags a fresh burst of forked children rather than pre-empting it.
- `cpu_jobs = max(1, floor((nproc - current_load_avg) * PROCESS_FACTOR))` — leaves room for whatever's already running, then further discounts for this build's own forked children
- `JOBS = max(1, min(nproc, mem_jobs, cpu_jobs))`
4. State the computed value and its inputs in one line (e.g. `nproc=22, available=20466MB, load=1.2 → --jobs 14`) before launching the build, then pass `--jobs <JOBS>` to `xce-build.sh` alongside the other args.
4. State the computed value and its inputs in one line (e.g. `nproc=22, available=20466MB, load=1.2, factor=0.5 → --jobs 7`) before launching the build, then pass `--jobs <JOBS>` to `xce-build.sh` alongside the other args.
This is a judgment call, not a rigid formula — if you notice other heavy build processes already running (`ps aux | grep -E 'cc1plus|ninja|go build'`) or the available-memory read looks unusually low, bias further down.
This is a judgment call, not a rigid formula — if you notice other heavy build processes already running (`ps aux | grep -E 'cc1plus|cc1 |ninja|go build'`) or the available-memory read looks unusually low, bias further down (lower `PROCESS_FACTOR` toward `0.35` rather than just cutting `JOBS` after the fact, so the memory and CPU terms stay consistent with each other). If the workstation is still visibly overloaded (interactive lag, swapping, fans pinned) partway through a build even though `JOBS` was computed per this formula, that's a signal the actual processes-per-job ratio is running higher than assumed for the current toolchain/profile — drop `PROCESS_FACTOR` further for subsequent builds rather than reaching for a bigger flat cut to `nproc`.
- **No `context: fork`, intentionally.** It was tried (twice) and reproducibly breaks this skill: whenever ARGUMENTS is non-empty (e.g. `controller`), the forked subagent receives no usable task — it responds as if handed only ambient project context, never runs the build, and reports nothing back. Bare `/xce-build` (no args) happens to still "succeed" under `context: fork` only because the empty-arg case is indistinguishable from no task at all failing open into some default — not because forking works. Do not re-add `context: fork` here without first confirming the harness actually threads a non-empty ARGUMENTS payload into the forked subagent's prompt.
- **component** (1st word, default `all`) — `all`, `lib`, `controller`, `harness`, `scheduler`, `worker`, `apiserver`, `xcecli`, `xceadmin`