feat(xce-build): discount --jobs for forked compiler/assembler children per profile
nproc - load_avg treats one ninja job as one process, but the compiler driver forks cc1/cc1plus then as, and asan/debug keep those children resident far longer than release's leaner ones -- under-throttling by roughly that factor. Apply a per-profile PROCESS_FACTOR (0.5 for asan/debug, 0.7 for release) to the CPU term before taking the min with the memory-budget term. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLSNJhNoULvVcUhXEFafQC
This commit is contained in:
parent
0f3919f195
commit
89a3dec5d9
1 changed files with 5 additions and 4 deletions
|
|
@ -22,13 +22,14 @@ Args: `[component] [profile|build_dir] [build_dir] [--incremental]` — passed a
|
|||
cut -d' ' -f1 /proc/loadavg # current 1-min load average = existing run-queue length from other activity
|
||||
```
|
||||
2. Pick a per-job memory budget by profile — ASAN+debug is the heaviest (debug info + sanitizer instrumentation per translation unit): **~1536 MB/job for `asan`** (the default profile), **~1024 MB/job for `debug`/`release`**.
|
||||
3. Compute, reserving ~15% of available memory as headroom for the OS and other sessions:
|
||||
3. Compute, reserving ~15% of available memory as headroom for the OS and other sessions, and dampening the CPU term for how many *live OS processes* one "job" actually costs:
|
||||
- `mem_jobs = floor(available_mem_mb * 0.85 / per_job_mb)`
|
||||
- `cpu_jobs = max(1, floor(nproc - current_load_avg))` — leaves room for whatever's already running
|
||||
- `PROCESS_FACTOR = 0.5` for `asan`/`debug` profiles, `0.7` for `release` — each in-flight ninja job is not one process: the compiler driver (`cc`/`c++`) forks a child compiler (`cc1`/`cc1plus`) and then an assembler (`as`), and `asan`/`debug` stretch out how long those extra children stay resident (sanitizer instrumentation, full debug-info emission) versus `release`'s leaner, shorter-lived ones. `nproc - current_load_avg` alone treats 1 job ≈ 1 process, so it under-throttles by roughly this factor — and `current_load_avg` is a smoothed 1-minute average, so it lags a fresh burst of forked children rather than pre-empting it.
|
||||
- `cpu_jobs = max(1, floor((nproc - current_load_avg) * PROCESS_FACTOR))` — leaves room for whatever's already running, then further discounts for this build's own forked children
|
||||
- `JOBS = max(1, min(nproc, mem_jobs, cpu_jobs))`
|
||||
4. State the computed value and its inputs in one line (e.g. `nproc=22, available=20466MB, load=1.2 → --jobs 14`) before launching the build, then pass `--jobs <JOBS>` to `xce-build.sh` alongside the other args.
|
||||
4. State the computed value and its inputs in one line (e.g. `nproc=22, available=20466MB, load=1.2, factor=0.5 → --jobs 7`) before launching the build, then pass `--jobs <JOBS>` to `xce-build.sh` alongside the other args.
|
||||
|
||||
This is a judgment call, not a rigid formula — if you notice other heavy build processes already running (`ps aux | grep -E 'cc1plus|ninja|go build'`) or the available-memory read looks unusually low, bias further down.
|
||||
This is a judgment call, not a rigid formula — if you notice other heavy build processes already running (`ps aux | grep -E 'cc1plus|cc1 |ninja|go build'`) or the available-memory read looks unusually low, bias further down (lower `PROCESS_FACTOR` toward `0.35` rather than just cutting `JOBS` after the fact, so the memory and CPU terms stay consistent with each other). If the workstation is still visibly overloaded (interactive lag, swapping, fans pinned) partway through a build even though `JOBS` was computed per this formula, that's a signal the actual processes-per-job ratio is running higher than assumed for the current toolchain/profile — drop `PROCESS_FACTOR` further for subsequent builds rather than reaching for a bigger flat cut to `nproc`.
|
||||
|
||||
- **No `context: fork`, intentionally.** It was tried (twice) and reproducibly breaks this skill: whenever ARGUMENTS is non-empty (e.g. `controller`), the forked subagent receives no usable task — it responds as if handed only ambient project context, never runs the build, and reports nothing back. Bare `/xce-build` (no args) happens to still "succeed" under `context: fork` only because the empty-arg case is indistinguishable from no task at all failing open into some default — not because forking works. Do not re-add `context: fork` here without first confirming the harness actually threads a non-empty ARGUMENTS payload into the forked subagent's prompt.
|
||||
- **component** (1st word, default `all`) — `all`, `lib`, `controller`, `harness`, `scheduler`, `worker`, `apiserver`, `xcecli`, `xceadmin`
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue