lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(bug-1736): capture memory_swap/tasks_current, compute peak_cpu_time

Browse source 
The worker's rusage collector already measures and sends
"memory_swap"/"tasks_current" per job, but xce_host_runusage_t had no
field to receive them at all -- a complete dead end, not a naming
mismatch like bug-1735's cpu_time/mem_usage. Added mem_swap/
tasks_current fields (types.h), matching capnp fields on HostRunusage
(controller.capnp, new ordinals @3/@4) for WAL field-completeness, and
wired update_host_runusage() (misc.c) to parse them. Also fixed
merge_job_runusage()'s (job.c) existing-host merge branch, which only
copied cpu_time/mem_usage per host -- the two new fields would
otherwise have been silently dropped on WAL replay/catchup for a host
already seen.

Also computed peak_cpu_time as the running max of aggregate_job_
runusage()'s per-call cpu_time sum across every report a job has ever
received, guarding against a later report reducing it.

Deliberately NOT computed: peak_cpu_duration and the three efficiency
fields (mem_efficiency/avg_cpu_efficiency/peak_cpu_efficiency) -- these
need a "used vs. requested" ratio against a job's resource request (a
resreq expression tree, not a plain core/slot count) and/or an
unspecified duration semantic. A documented scope decision, not an
oversight -- see aggregate_job_runusage()'s doc comment.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019QxaSzHRB7pmYFp4cVnwkA
This commit is contained in:
Rongsong Shen 2026-08-30 15:32:02 +00:00
parent dfcd345ded
commit 5c2c6664e5
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
5 changed files with 169 additions and 26 deletions
Download patch file Download diff file Expand all files Collapse all files

53
src/controller/misc.c
View file

@ -139,21 +139,54 @@ int update_host_runusage(xce_job_runusage_t** runusage, char* host, int count,
rusages->host_runusage[j]->mem_usage = usages[i]->value;
continue;
}
/* bug-1736: the worker already sends these two (collect_job_rusages(),
* xcecd_cgroups.c res_names[]) but xce_host_runusage_t had no field
* to receive them at all -- a complete dead end, not a naming
* mismatch like bug-1735's cpu_time/mem_usage. */
if (strcmp(usages[i]->resname, "memory_swap") == 0) {
rusages->host_runusage[j]->mem_swap = usages[i]->value;
continue;
}
if (strcmp(usages[i]->resname, "tasks_current") == 0) {
rusages->host_runusage[j]->tasks_current = (int)usages[i]->value;
continue;
}
}
return 0;
}
/* bug-1735: derives xce_job_runusage_t's own job-level max_mem/avg_mem/
* used_mem/cpu_time from the per-host host_runusage[] entries
* update_host_runusage() populates -- replaces update_job_rusages()'s
* (job.c) standing "update other fields of job rusage later" FIXME.
/* bug-1735/bug-1736: derives xce_job_runusage_t's own job-level max_mem/
* avg_mem/used_mem/cpu_time/peak_cpu_time from the per-host
* host_runusage[] entries update_host_runusage() populates -- replaces
* update_job_rusages()'s (job.c) standing "update other fields of job
* rusage later" FIXME.
*
* used_mem is the SUM across hosts (a multi-host job's total memory
* footprint); max_mem/avg_mem are the max/average across hosts; cpu_time is
* the SUM across hosts. peak_cpu_time/peak_cpu_duration/mem_efficiency/
* avg_cpu_efficiency/peak_cpu_efficiency are intentionally left untouched --
* bug-1735 was explicitly scope-narrowed to memory + CPU time only; those
* fields are bug-1736 (a design task, no formula exists yet). */
* the SUM across hosts. peak_cpu_time is the running MAX of that per-report
* cpu_time sum across every report this job has ever received (a job's own
* cpu_time can only trend upward within one report but this is called
* repeatedly over the job's life, and a merge/replay/WAL-catchup path could
* in principle deliver reports out of order -- tracking the max explicitly,
* not just trusting "latest == largest", is what makes it a genuine peak
* rather than a duplicate of cpu_time).
*
* bug-1736's OTHER two fields, peak_cpu_duration and the three efficiency
* fields (mem_efficiency/avg_cpu_efficiency/peak_cpu_efficiency), are
* DELIBERATELY left untouched here -- a documented scope decision, not an
* oversight (steering persistence.md's field-completeness rule explicitly
* allows this when documented). Unlike peak_cpu_time, these have no
* unambiguous definition to implement without inventing a formula: the
* efficiency fields need a "used vs. requested" ratio against a job's
* resource request, which is a resreq expression tree (parse_resreq()),
* not a plain core/slot count -- deriving a well-defined denominator from
* that is a real design task, not a two-line fix. peak_cpu_duration's own
* semantics (wall-clock elapsed to the peak? duration the peak was
* sustained?) were never specified anywhere. Both need a product decision
* before a formula can be implemented correctly. */
void aggregate_job_runusage(xce_job_runusage_t* runusage)
{
int i, valid = 0;
@ -190,6 +223,10 @@ void aggregate_job_runusage(xce_job_runusage_t* runusage)
runusage->max_mem = mem_max;
runusage->avg_mem = mem_sum / valid;
runusage->cpu_time = cpu_sum;
if (cpu_sum > runusage->peak_cpu_time) {
runusage->peak_cpu_time = cpu_sum;
}
}
int update_pids(job_pidinfo_t** pidinfo, int npids, int* pids)