lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(scheduler): host-scoped limit staged N times per dispatch (bug-1846 part 1)

Browse source 
match_job_host_limit_() (plugins/limit.c) runs on every per-host
allocation attempt via limit_prepare_allocation(), and unconditionally
xlist_store()d the matched vres into obj->hlimits[worker->idx]. That list
is only rebuilt by limit_job_election() on a configver bump, so a job that
kept pending on the same host gained one more copy per attempt.
sched_fill_limit_requests() then staged every copy into
ctx->alloc_req.host, and seqjob's make_host_alloc() built a host alloc
carrying all of them.

Found via temporary per-decision --sim-verify diagnostics (reverted, not
committed) against scheduler-replay-20260923-085356.replay: pc13v host
allocs held mem + slots + ~70 identical limit entries (72) instead of 3.
After the fix the replay side is always 3 entries, and the LIVE recording
itself still shows the inflation (72/81/96/114/138/177 entries, resetting
only on a config push) -- so this is a production bug, not sim-only.
make_job_account() creates one res_account per entry and
sched_update_res_accounts() charges each, so the host limit was charged
(and later released) N-fold.

Fix: hlimits_contains_() pointer-identity check (vres is interned per name
in privdata->instances); a matched vres is stored once per (job, worker).

Test: test_limit_host_dedup.c (sched_limit_host_dedup_test) -- five
plugin_prepare_allocation() attempts for one job on one worker must leave
hlimits[worker]->count == 1 and one staged copy of the vres in
alloc_req.host. RED before the fix (Expected 1 Was 2), GREEN after; full
scheduler suite 86/86 (asan).

Still open (bug-1846 part 2, logged in buglog): 9 live-recorded rounds end
after 11-12 decisions while replay processes the whole candidate list; not
root-caused, needs the live scheduler.log or a fresh capture.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-09-24 04:21:27 +00:00
parent fcf119b4ac
commit 77f047f2e2
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
4 changed files with 319 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

24
src/scheduler/testsuite/meson.build
View file

@ -2207,6 +2207,30 @@ test('scheduler sim contention limit driver parity test',
sched_sim_contention_limit_driver_parity_test, suite: 'scheduler',
env: ['ASAN_OPTIONS=detect_leaks=0'])
# bug-1846: a host-scoped limit must be staged once per (job, worker) no
# matter how many allocation attempts the job makes there --
# match_job_host_limit_() used to append a fresh copy to obj->hlimits on
# every attempt, inflating the DISPATCH host allocation (72 vs 3 entries in
# a real captured replay). Links every scheduler source except main.c, same
# as the driver-parity fixture above.
sched_limit_host_dedup_test_srcs = ['test_limit_host_dedup.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
sched_limit_host_dedup_test_srcs += ['../' + f]
endif
endforeach
sched_limit_host_dedup_test = executable(
'sched_limit_host_dedup_test',
sched_limit_host_dedup_test_srcs,
include_directories:[incdir],
dependencies: [scheduler_test_deps],
install: false)
test('scheduler limit host dedup test',
sched_limit_host_dedup_test, suite: 'scheduler',
env: ['ASAN_OPTIONS=detect_leaks=0'])
# scheduler-sim-live-allocation spec, task 5.2 (Req 2.1, 2.2, 2.3, 2.4): the
# feature's concrete, original acceptance test (bug-1826) -- a real global
# AND host-scoped limit configured in a real SnapshotEntry's Conf4Sched,