From 89a3dec5d9b4c497b2eea0b411b848179d473dd6 Mon Sep 17 00:00:00 2001 From: Rongsong Shen Date: Tue, 15 Sep 2026 06:35:34 +0000 Subject: [PATCH] feat(xce-build): discount --jobs for forked compiler/assembler children per profile nproc - load_avg treats one ninja job as one process, but the compiler driver forks cc1/cc1plus then as, and asan/debug keep those children resident far longer than release's leaner ones -- under-throttling by roughly that factor. Apply a per-profile PROCESS_FACTOR (0.5 for asan/debug, 0.7 for release) to the CPU term before taking the min with the memory-budget term. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01RLSNJhNoULvVcUhXEFafQC --- .claude/skills/xce-build/SKILL.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/.claude/skills/xce-build/SKILL.md b/.claude/skills/xce-build/SKILL.md index f14c0ae3..15a0d2fe 100644 --- a/.claude/skills/xce-build/SKILL.md +++ b/.claude/skills/xce-build/SKILL.md @@ -22,13 +22,14 @@ Args: `[component] [profile|build_dir] [build_dir] [--incremental]` — passed a cut -d' ' -f1 /proc/loadavg # current 1-min load average = existing run-queue length from other activity ``` 2. Pick a per-job memory budget by profile — ASAN+debug is the heaviest (debug info + sanitizer instrumentation per translation unit): **~1536 MB/job for `asan`** (the default profile), **~1024 MB/job for `debug`/`release`**. -3. Compute, reserving ~15% of available memory as headroom for the OS and other sessions: +3. Compute, reserving ~15% of available memory as headroom for the OS and other sessions, and dampening the CPU term for how many *live OS processes* one "job" actually costs: - `mem_jobs = floor(available_mem_mb * 0.85 / per_job_mb)` - - `cpu_jobs = max(1, floor(nproc - current_load_avg))` — leaves room for whatever's already running + - `PROCESS_FACTOR = 0.5` for `asan`/`debug` profiles, `0.7` for `release` — each in-flight ninja job is not one process: the compiler driver (`cc`/`c++`) forks a child compiler (`cc1`/`cc1plus`) and then an assembler (`as`), and `asan`/`debug` stretch out how long those extra children stay resident (sanitizer instrumentation, full debug-info emission) versus `release`'s leaner, shorter-lived ones. `nproc - current_load_avg` alone treats 1 job ≈ 1 process, so it under-throttles by roughly this factor — and `current_load_avg` is a smoothed 1-minute average, so it lags a fresh burst of forked children rather than pre-empting it. + - `cpu_jobs = max(1, floor((nproc - current_load_avg) * PROCESS_FACTOR))` — leaves room for whatever's already running, then further discounts for this build's own forked children - `JOBS = max(1, min(nproc, mem_jobs, cpu_jobs))` -4. State the computed value and its inputs in one line (e.g. `nproc=22, available=20466MB, load=1.2 → --jobs 14`) before launching the build, then pass `--jobs ` to `xce-build.sh` alongside the other args. +4. State the computed value and its inputs in one line (e.g. `nproc=22, available=20466MB, load=1.2, factor=0.5 → --jobs 7`) before launching the build, then pass `--jobs ` to `xce-build.sh` alongside the other args. -This is a judgment call, not a rigid formula — if you notice other heavy build processes already running (`ps aux | grep -E 'cc1plus|ninja|go build'`) or the available-memory read looks unusually low, bias further down. +This is a judgment call, not a rigid formula — if you notice other heavy build processes already running (`ps aux | grep -E 'cc1plus|cc1 |ninja|go build'`) or the available-memory read looks unusually low, bias further down (lower `PROCESS_FACTOR` toward `0.35` rather than just cutting `JOBS` after the fact, so the memory and CPU terms stay consistent with each other). If the workstation is still visibly overloaded (interactive lag, swapping, fans pinned) partway through a build even though `JOBS` was computed per this formula, that's a signal the actual processes-per-job ratio is running higher than assumed for the current toolchain/profile — drop `PROCESS_FACTOR` further for subsequent builds rather than reaching for a bigger flat cut to `nproc`. - **No `context: fork`, intentionally.** It was tried (twice) and reproducibly breaks this skill: whenever ARGUMENTS is non-empty (e.g. `controller`), the forked subagent receives no usable task — it responds as if handed only ambient project context, never runs the build, and reports nothing back. Bare `/xce-build` (no args) happens to still "succeed" under `context: fork` only because the empty-arg case is indistinguishable from no task at all failing open into some default — not because forking works. Do not re-add `context: fork` here without first confirming the harness actually threads a non-empty ARGUMENTS payload into the forked subagent's prompt. - **component** (1st word, default `all`) — `all`, `lib`, `controller`, `harness`, `scheduler`, `worker`, `apiserver`, `xcecli`, `xceadmin`