lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(bug-1735): job memory/CPU usage always read 0 in scheduler replay

Browse source 
Two independent gaps in misc.c/job.c, both silently zeroing every real
job's resource-usage data:

- update_host_runusage() matched worker rusage report resnames
  "cputime"/"mem", but the real worker's collector
  (collect_job_rusages(), xcecd_cgroups.c) sends "cpu_time_sec"/
  "memory_rss" -- the two never matched, so cpu_time/mem_usage stayed
  at their calloc()'d zero for every non-simulated job. Aligned both
  sides, including xce_test's own simulated-worker fixture.
- update_job_rusages() had a standing FIXME: xce_job_runusage_t's
  job-level max_mem/avg_mem/used_mem/cpu_time were never derived from
  the per-host host_runusage[] entries. Added aggregate_job_runusage()
  (sum/max/avg across hosts) and wired it in right after the per-host
  update.

Added test_host_runusage.c (standalone unit test, mirrors
resval_typeresolve_unit_test's isolation pattern) plus the
controller_unit_test stub aggregate_job_runusage() needs since that
target excludes the real misc.c.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019QxaSzHRB7pmYFp4cVnwkA
This commit is contained in:
Rongsong Shen 2026-08-30 15:07:33 +00:00
parent ccef561031
commit d8a37fa27a
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
7 changed files with 335 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

56
src/controller/misc.c
View file

@ -126,12 +126,16 @@ int update_host_runusage(xce_job_runusage_t** runusage, char* host, int count,
}
for (i = 0; i < count; i++) {
if (strcmp(usages[i]->resname, "cputime") == 0) {
/* bug-1735: these must match the REAL worker's rusage collector
* (collect_job_rusages(), xcecd_cgroups.c res_names[]) -- "cputime"/
* "mem" never matched anything that worker actually sends, so
* cpu_time/mem_usage stayed at their calloc()'d zero forever. */
if (strcmp(usages[i]->resname, "cpu_time_sec") == 0) {
rusages->host_runusage[j]->cpu_time = usages[i]->value;
continue;
}
if (strcmp(usages[i]->resname, "mem") == 0) {
if (strcmp(usages[i]->resname, "memory_rss") == 0) {
rusages->host_runusage[j]->mem_usage = usages[i]->value;
continue;
}
@ -140,6 +144,54 @@ int update_host_runusage(xce_job_runusage_t** runusage, char* host, int count,
return 0;
}
/* bug-1735: derives xce_job_runusage_t's own job-level max_mem/avg_mem/
* used_mem/cpu_time from the per-host host_runusage[] entries
* update_host_runusage() populates -- replaces update_job_rusages()'s
* (job.c) standing "update other fields of job rusage later" FIXME.
* used_mem is the SUM across hosts (a multi-host job's total memory
* footprint); max_mem/avg_mem are the max/average across hosts; cpu_time is
* the SUM across hosts. peak_cpu_time/peak_cpu_duration/mem_efficiency/
* avg_cpu_efficiency/peak_cpu_efficiency are intentionally left untouched --
* bug-1735 was explicitly scope-narrowed to memory + CPU time only; those
* fields are bug-1736 (a design task, no formula exists yet). */
void aggregate_job_runusage(xce_job_runusage_t* runusage)
{
int i, valid = 0;
double mem_sum = 0.0;
double mem_max = 0.0;
int cpu_sum = 0;
if ((runusage == NULL) || (runusage->host_runusage == NULL)) {
return;
}
for (i = 0; i < runusage->num_hosts; i++) {
xce_host_runusage_t* h = runusage->host_runusage[i];
if (h == NULL) {
continue;
}
mem_sum += h->mem_usage;
cpu_sum += h->cpu_time;
if ((valid == 0) || (h->mem_usage > mem_max)) {
mem_max = h->mem_usage;
}
valid++;
}
if (valid == 0) {
return;
}
runusage->used_mem = mem_sum;
runusage->max_mem = mem_max;
runusage->avg_mem = mem_sum / valid;
runusage->cpu_time = cpu_sum;
}
int update_pids(job_pidinfo_t** pidinfo, int npids, int* pids)
{
job_pidinfo_t* info;