A job with no rusage[mem=...] request (reserved=0) was being SKIPPED
entirely during reservation-vs-usage reconciliation, on the mistaken
assumption that "no reservation" meant "nothing needs charging". It
doesn't: this codebase's admission check has no live OS read at
decision time (sched_check_host_satisfy_(), resource.c) -- it compares
the worker's last-reported "mem" value against this scheduler's OWN
tracked resInUse tally, built entirely from summed xce_res_account_t
entries. Skipping an unreserved job left its real, live memory
consumption permanently invisible to that tally -- an actual
over-subscription risk (the scheduler could admit more onto a host
than it can support, believing memory was free that a live process was
genuinely using), not merely a display gap.
Fix: charged = max(reserved, actual) applies to EVERY job, reserved or
not. ensure_host_account_by_resname() (resource.c) now creates a fresh
reserved=0 account entry (resolving residx via key_find() against
core->resmgr, same pattern sched_update_res_accounts() already uses
for a fresh account) instead of skipping when a job has no existing
entry for a resource. Returns NULL (a genuine no-op) only when the
resource isn't registered in the cluster's config at all.
Rewrote resource-accounting.md to correct the mechanism it documented:
"available" is NOT a live OS read (mem vs the similarly-named but
static maxmem was worth naming explicitly too) -- it's worker's
last-reported mem minus this scheduler's own resInUse tally, and that
tally must include every job's real usage regardless of reservation.
Also documented a known, pre-existing property surfaced while
correcting this: since mem's own worker report already reflects real
consumption and the scheduler's charge is a separate layer that
doesn't get cleared on a fresh report, tracking can become
conservative over time (under-utilization risk, never over-
subscription) -- the same direction slots already accepts, and a
separate, larger question than this fix's scope.
Replaced test_reconcile_job_resource_usage_skips_when_no_mem_request
(asserted the now-wrong skip behavior) with two tests: one proving an
unreserved job's real usage is created and charged, one proving a
genuine no-op when "mem" isn't a registered resource at all. Full
scheduler suite: 55/55 passing, zero regressions.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019QxaSzHRB7pmYFp4cVnwkA
Rongsong Shen
f4103e0efd