lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

test(scheduler-allocation): end-to-end multi-host parallel job proof

Browse source 
Drive a real multi-slot parallel job through the actual production
dispatch path -- real sched_init(), real queue/worker/job ingest via
xce_core_push_events(), a real scheduling session (exercising task 5.2's
cache-clear wiring), the real sched_allocator() thread, cache lookup,
parjob_allocator's real my_alloc(), host_iter, and commit -- and confirm
the resulting job allocation genuinely spans 2 hosts with correct
per-host slot counts (Req 4.1, 4.2, 4.3).

Discovered and logged bug-1770: parse_resreq_primary() unconditionally
hardcodes primary->minTasks/maxTasks to 1 regardless of what a job
actually requests, and nothing in the real ingest pipeline ever copies
the job's own submitted task count (correctly threaded from the
apiserver's num_processors/maxnum_processors all the way to
xce_sched_job_t.mintasks/maxtasks) onto the nested xce_primary_resreq_t
fields the allocator dispatch actually reads. This predates the feature
by ~7 months and is out of this spec's scope to fix, but it means
parjob_allocator has zero live production path until a dedicated
follow-up lands. The test compensates with one clearly-labeled patch of
the real, already-ingested resreq's task-count fields immediately after
real parsing; everything else in the pipeline is unmodified production
code.

Also logged bug-1771 (xce_sched_fcfs's fcfs_del_job() dereferences
job->queue->name with no NULL guard) discovered while building this
task's fixture; worked around by registering the job's queue before
push, mirroring the apiserver's own pre-submission queue validation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
This commit is contained in:
Rongsong Shen 2026-09-05 01:56:54 +00:00
parent e8c2087d5e
commit c009f108b7
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
4 changed files with 694 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

32
src/scheduler/testsuite/meson.build
View file

@ -1896,3 +1896,35 @@ test('scheduler cache dispatch test',
test('scheduler cache dispatch compound test',
sched_cache_dispatch_compound_test, suite: 'scheduler',
env: ['ASAN_OPTIONS=detect_leaks=0'])
# scheduler-allocation spec, task 5.3 (Req 4.1, 4.2, 4.3): the capstone
# end-to-end integration test for this whole feature -- drives a real
# maxTasks>1 parallel job through the full production dispatch path
# (sched_init()'s real, live sched_allocator() thread -> the real, `static`
# sched_make_allocation()/sched_make_primary_alloc_() choke point -> the
# real parjob_allocator.c my_alloc() -> the real host_iter.c host iterator
# -> a real multi-host xce_job_alloc_t commit), asserting the resulting
# job->allocs/decision->u.alloc spans exactly 2 hosts with the correct
# per-host slot counts. Same real-scheduler_srcs link technique as
# sched_snapshot_load_allocator_regression_test/sched_cache_dispatch_test
# above (see this test file's own header comment for the full rationale).
scheduler_parjob_e2e_multislot_test_srcs = [
'test_parjob_e2e_multislot.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
scheduler_parjob_e2e_multislot_test_srcs += ['../' + f]
endif
endforeach
sched_parjob_e2e_multislot_test = executable(
'sched_parjob_e2e_multislot_test',
scheduler_parjob_e2e_multislot_test_srcs,
include_directories:[incdir],
dependencies: [scheduler_deps, unity_dep],
link_args: ['-Wl,--wrap=xcecli_start',
'-Wl,--wrap=xcecli_join'],
install: false)
test('scheduler parjob e2e multislot test',
sched_parjob_e2e_multislot_test, suite: 'scheduler',
env: ['ASAN_OPTIONS=detect_leaks=0'])