update_job_cpu_stats() (misc.c), run after each usage report:
- each host's cpu_rate = CPU-seconds gained / wall seconds since its
previous report; the job's cores = the sum over hosts
- peak_cpu = highest job cores seen (peak_cpu_time = it rounded, the
wire field is an integer); peak_cpu_duration = the interval that set it
- avg_cpu_efficiency = cpu_time / elapsed / slots, peak_cpu_efficiency =
peak_cpu / slots, mem_efficiency = highest used_mem / rusage[mem]
request (job-wide total, per task by default), all in percent
aggregate_job_runusage() no longer treats cumulative cpu_time as a peak.
New fields host sample_time/cpu_rate and job peak_cpu are appended to
the capnp structs (old WAL entries still decode) and copied by
merge_job_runusage(), so a promoted standby keeps the rate baselines.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Same class as bug-1678/1687/1689: the WAL fact for WAL_JOB_RUSAGE only
carried {jobid, state}, so job.run_usage (client-reported per-host
CPU/mem usage) was never restored on either apply seam after a restart.
Unlike the prior three, xce_job_runusage_t/xce_host_runusage_t had no
existing Cap'n Proto mapping -- a genuine new schema addition
(HostRunusage/JobRunusage in controller.capnp, mirroring JobPendReason's
List(NestedStruct)+$C.maplistcount style), verified against the real
struct definitions in types.h, codegen confirmed via nm on the built
binary rather than assumed.
Two deliberate deviations from the established precedent, found by
reading the real code rather than copying the pattern blindly:
1. Adoption is unconditional, not gated on a state transition -- a
rusage report almost never changes job.state (repeat reports keep
state at RUN), so gating on JOB_TRANSITION_APPLIED like START/FINISH/
DISPATCH do would make the fix dead code for the realistic case.
2. Merge, not free-and-replace. update_host_runusage() builds run_usage
with host_name sourced from makeString() -- an RTTI-managed pointer,
not malloc() -- and the primary's own live apply pump re-applies every
fact it just wrote. A naive free_JobRunusage_ptr()-then-adopt (matching
the prior three fixes exactly) would free() a non-malloc'd pointer on
the very first live rusage report for a new host -- guaranteed heap
corruption. Fixed with merge_job_runusage(): existing hosts get
scalar-only in-place updates (host_name pointer never touched), only
genuinely new hosts adopt their freshly-decoded (safe, malloc'd)
host_name. Proved via a real makeString()-backed test asserting
pointer identity survives, not just value equality. job_destroy()
(src/lib/jobj.c) was found to have this identical latent mismatch,
pre-existing and independent of this fix -- flagged, not fixed here.
Found by an /xce-audit-invariants controller pass (field-complete apply
parity dimension). 8 new tests in test_wal_job_rusage_fact.c; RED/GREEN
independently verified. Full controller suite: 1160 tests, 2 failures --
both the pre-existing, unrelated bug-1561 overlayfs sandbox flake. No
ASAN/heap-corruption findings. Backward compatibility confirmed for
old-format bare-payload entries.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AgYRU3zDTqNGV5Jk8aiKLg
