lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

build(scheduler): compile scheduler sources once -- shared sched_srcs_lib

Browse source 
The sched_unit_test aggregate and 16 zero-#include standalone tests each
compiled all ~50 scheduler sources as their own TUs (~800 redundant
compiles of identical sources per clean build). Add a shared
sched_srcs_lib static archive (scheduler_srcs minus main.c) in the parent
meson.build; those 17 targets now link it and compile only their own test
TUs.

- Only extern-API consumers convert: these tests call nothing but symbols
  declared in scheduler.h/core.h. Textual includers (white-box statics
  access) keep their own compilation -- their TUs define the included
  files' symbols and could collide with archive members; the 4
  exclusion-list foreach blocks (snapshot_load_allocator_regression,
  repl_trace_wiring, trace_error_paths, trace_bounds) are untouched.
- -Wl,--wrap= interposition is link-time symbol redirection and works
  identically against archive members; the three --wrap tests
  (publish_retry/backpressure/redelivery) are the sensitive probes and
  pass.
- The aggregate keeps -Werror=incompatible-pointer-types and all baked
  path defines; its src foreach is gone (the lib carries the sources).

Verified: fresh asan tree 1135 steps (was 1934), ninja green, meson test
134/134 including all converted targets; disk unchanged at ~1.3G after
the previous commit's capnp dedup.

Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
lei.chen 2026-10-21 14:07:14 +00:00
parent 678163adb8
commit d4957bd290
2 changed files with 66 additions and 146 deletions
Download patch file Download diff file Expand all files Collapse all files

24
src/scheduler/meson.build
View file

@ -193,4 +193,28 @@ sched_sim_isolation_lib = static_library(
dependencies: scheduler_deps,
install: false)
# Build-time dedup: every testsuite target that links the real scheduler
# source list (the sched_unit_test aggregate plus the zero-#include
# standalone tests, seventeen targets in all) used to compile all ~50 of
# these files as its own TUs -- ~800 redundant compiles of the same sources
# in one clean build. This archive compiles them once; those tests link it
# instead. Only extern-API consumers may use it: a test that #includes a
# src/*.c textually (white-box statics access) defines those symbols in its
# own TU and could pull the same archive member in for a duplicate-symbol
# link error, so the textual-includer tests keep their own compilation of
# the full list. -Wl,--wrap= interposition is unaffected (link-time symbol
# redirection works identically against archive members).
sched_srcs_lib_srcs = []
foreach f : scheduler_srcs
if f != 'src/main.c'
sched_srcs_lib_srcs += [f]
endif
endforeach
sched_srcs_lib = static_library(
'sched_srcs',
sched_srcs_lib_srcs,
include_directories: incdir,
dependencies: scheduler_deps,
install: false)
subdir('testsuite')

188
src/scheduler/testsuite/meson.build
View file

@ -42,17 +42,18 @@ scheduler_test = executable('sched_test',
# each statically linking all of scheduler_srcs -- 120+ executables whose
# compile+link dominated the suite wall clock. This suite aggregates them
# into ONE process (same pattern as controller/testsuite/controller_test.c):
# the production scheduler sources are compiled once here as separate TUs,
# each test file keeps only its own suite entry (void test_sched_*(void)
# { UnitySetTestFile(__FILE__); RUN_TEST(...); }), and
# the production scheduler sources come in via sched_srcs_lib (compiled once
# for the whole testsuite; these tests only call extern APIs declared in
# scheduler.h/core.h), each test file keeps only its own suite entry (void
# test_sched_*(void) { UnitySetTestFile(__FILE__); RUN_TEST(...); }), and
# sched_unit_runner.c provides main()/setUp()/tearDown() dispatching through
# its SUITES registry. The per-file transformations were done by hand, one
# file at a time, with every file compiled+run in the aggregate before the
# next was touched.
#
# Only tests with NO textual #include of ../src/*.c qualify: a test that
# #includes a scheduler source textually duplicates its symbols and cannot
# coexist with the aggregate's own real TUs (those tests stay standalone,
# #includes a scheduler source textually defines those symbols in its own TU
# and could collide with the archive's members (those tests stay standalone,
# exactly as before). Tests needing -Wl,--wrap= link interposition,
# #define-shaping of the allocator, or that spawn XCE_SCHEDULER_BIN as a
# subprocess also stay standalone for the same reason.
@ -66,11 +67,6 @@ scheduler_test = executable('sched_test',
# link_with here would duplicate the aggregate's own core/sim TUs --
# instead `depends:` orders their build).
sched_unit_test_srcs = ['sched_unit_runner.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
sched_unit_test_srcs += ['../' + f]
endif
endforeach
foreach f : ['test_sched_election_null_schedobj.c',
'test_picker_iface.c',
'test_picker_pendreason_factory_audit.c',
@ -107,6 +103,7 @@ sched_unit_test = executable(
sched_unit_test_srcs,
include_directories:[incdir],
dependencies: [scheduler_test_deps],
link_with: sched_srcs_lib,
c_args: ['-Werror=incompatible-pointer-types',
'-DXCE_SCHEDULER_ROOT_PATH="'
+ (meson.current_source_dir() / '..')
@ -1464,19 +1461,12 @@ sched_repl_equivalence_test = executable(
# translation unit inside libxce.a with several OTHER xcecli_* entry points
# this test's real scheduler_srcs dependency graph also needs, so simply
# excluding client.c from the link was not viable.
scheduler_decision_publish_retry_test_srcs = [
'test_scheduler_decision_publish_retry.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
scheduler_decision_publish_retry_test_srcs += ['../' + f]
endif
endforeach
sched_scheduler_decision_publish_retry_test = executable(
'sched_scheduler_decision_publish_retry_test',
scheduler_decision_publish_retry_test_srcs,
'test_scheduler_decision_publish_retry.c',
include_directories:[incdir],
dependencies: [scheduler_deps, unity_dep],
link_with: sched_srcs_lib,
c_args: ['-DXCE_MAIN_C_PATH="'
+ (meson.current_source_dir() / '..' / 'src' / 'main.c')
+ '"'],
@ -1496,18 +1486,12 @@ sched_scheduler_decision_publish_retry_test = executable(
# allocator threads) and xcecli_set_pend_reason (blocks on a test-controlled
# condvar, standing in for a stuck outage retry) -- see the test file's own
# header comment for the full design and worst-case derivation.
scheduler_backpressure_test_srcs = ['test_scheduler_backpressure.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
scheduler_backpressure_test_srcs += ['../' + f]
endif
endforeach
sched_scheduler_backpressure_test = executable(
'sched_scheduler_backpressure_test',
scheduler_backpressure_test_srcs,
'test_scheduler_backpressure.c',
include_directories:[incdir],
dependencies: [scheduler_deps, unity_dep],
link_with: sched_srcs_lib,
link_args: ['-Wl,--wrap=xcecli_start',
'-Wl,--wrap=xcecli_join',
'-Wl,--wrap=xcecli_set_pend_reason'],
@ -1526,19 +1510,12 @@ sched_scheduler_backpressure_test = executable(
# driver.c's absolute path (XCE_DRIVER_C_PATH) alongside scheduler.c's
# (XCE_SCHEDULER_C_PATH, same pattern as sched_scheduler_connect_test) for
# its Req 7 AC5 source-text scan.
scheduler_decision_redelivery_test_srcs = [
'test_scheduler_decision_redelivery.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
scheduler_decision_redelivery_test_srcs += ['../' + f]
endif
endforeach
sched_scheduler_decision_redelivery_test = executable(
'sched_scheduler_decision_redelivery_test',
scheduler_decision_redelivery_test_srcs,
'test_scheduler_decision_redelivery.c',
include_directories:[incdir],
dependencies: [scheduler_deps, unity_dep],
link_with: sched_srcs_lib,
c_args: ['-DXCE_SCHEDULER_C_PATH="'
+ (meson.current_source_dir() / '..' / 'src' / 'scheduler.c')
+ '"',
@ -1600,18 +1577,12 @@ sched_snapshot_load_allocator_regression_test = executable(
# the decisive "did the real allocator run" signal a cache HIT and a
# genuine exhausted attempt cannot otherwise be told apart by (both pend,
# and Req 7.5 requires them to report the SAME reason category).
scheduler_cache_dispatch_test_srcs = ['test_scheduler_cache_dispatch.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
scheduler_cache_dispatch_test_srcs += ['../' + f]
endif
endforeach
sched_cache_dispatch_test = executable(
'sched_cache_dispatch_test',
scheduler_cache_dispatch_test_srcs,
'test_scheduler_cache_dispatch.c',
include_directories:[incdir],
dependencies: [scheduler_deps, unity_dep],
link_with: sched_srcs_lib,
link_args: ['-Wl,--wrap=xcecli_start',
'-Wl,--wrap=xcecli_join',
'-Wl,--wrap=sched_meet_host_selection'],
@ -1624,19 +1595,12 @@ sched_cache_dispatch_test = executable(
# a real run with both tests in one binary crashed with a genuine SEGV in
# fcfs_del_job() the first time this was tried -- see this test file's own
# header comment).
scheduler_cache_dispatch_compound_test_srcs = [
'test_scheduler_cache_dispatch_compound.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
scheduler_cache_dispatch_compound_test_srcs += ['../' + f]
endif
endforeach
sched_cache_dispatch_compound_test = executable(
'sched_cache_dispatch_compound_test',
scheduler_cache_dispatch_compound_test_srcs,
'test_scheduler_cache_dispatch_compound.c',
include_directories:[incdir],
dependencies: [scheduler_deps, unity_dep],
link_with: sched_srcs_lib,
link_args: ['-Wl,--wrap=xcecli_start',
'-Wl,--wrap=xcecli_join',
'-Wl,--wrap=sched_meet_host_selection'],
@ -2068,18 +2032,12 @@ test('scheduler repl equivalence test',
# make_alloc() cannot build a record. Real scheduler_srcs (as
# sched_scheduler_connect_test) plus `-Wl,--wrap=strdup` to make a chosen
# strdup() inside make_alloc() fail.
sched_make_allocs_oom_test_srcs = ['test_sched_make_allocs_oom.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
sched_make_allocs_oom_test_srcs += ['../' + f]
endif
endforeach
sched_make_allocs_oom_test = executable(
'sched_make_allocs_oom_test',
sched_make_allocs_oom_test_srcs,
'test_sched_make_allocs_oom.c',
include_directories:[incdir],
dependencies: [scheduler_deps, unity_dep],
link_with: sched_srcs_lib,
link_args: ['-Wl,--wrap=strdup'],
install: false)
@ -2163,19 +2121,12 @@ test('scheduler cache dispatch compound test',
# per-host slot counts. Same real-scheduler_srcs link technique as
# sched_snapshot_load_allocator_regression_test/sched_cache_dispatch_test
# above (see this test file's own header comment for the full rationale).
scheduler_parjob_e2e_multislot_test_srcs = [
'test_parjob_e2e_multislot.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
scheduler_parjob_e2e_multislot_test_srcs += ['../' + f]
endif
endforeach
sched_parjob_e2e_multislot_test = executable(
'sched_parjob_e2e_multislot_test',
scheduler_parjob_e2e_multislot_test_srcs,
'test_parjob_e2e_multislot.c',
include_directories:[incdir],
dependencies: [scheduler_deps, unity_dep],
link_with: sched_srcs_lib,
link_args: ['-Wl,--wrap=xcecli_start',
'-Wl,--wrap=xcecli_join'],
install: false)
@ -2196,19 +2147,12 @@ test('scheduler parjob e2e multislot test',
# for the full rationale -- one link technique for the whole file avoids a
# duplicate-symbol conflict with meson's own separate compilation of
# core.c/core_events.c/snapshot_load.c below).
sched_maxtasks_wiring_ingest_test_srcs = [
'test_maxtasks_wiring_ingest.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
sched_maxtasks_wiring_ingest_test_srcs += ['../' + f]
endif
endforeach
sched_maxtasks_wiring_ingest_test = executable(
'sched_maxtasks_wiring_ingest_test',
sched_maxtasks_wiring_ingest_test_srcs,
'test_maxtasks_wiring_ingest.c',
include_directories:[incdir],
dependencies: [scheduler_deps, unity_dep],
link_with: sched_srcs_lib,
link_args: ['-Wl,--wrap=xcecli_start',
'-Wl,--wrap=xcecli_join'],
install: false)
@ -2343,18 +2287,12 @@ test('scheduler maxtasks wiring sim driver parity test',
# carries the session start in whole seconds as `from`, stamped by
# sched_make_primary_alloc_() from the session core. Same real-
# scheduler_srcs link technique as sched_parjob_e2e_multislot_test.
sched_alloc_from_stamp_test_srcs = ['test_sched_alloc_from_stamp.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
sched_alloc_from_stamp_test_srcs += ['../' + f]
endif
endforeach
sched_alloc_from_stamp_test = executable(
'sched_alloc_from_stamp_test',
sched_alloc_from_stamp_test_srcs,
'test_sched_alloc_from_stamp.c',
include_directories:[incdir],
dependencies: [scheduler_deps, unity_dep],
link_with: sched_srcs_lib,
link_args: ['-Wl,--wrap=xcecli_start',
'-Wl,--wrap=xcecli_join'],
install: false)
@ -2394,18 +2332,12 @@ test('scheduler sim discrete clock test',
# technique as sched_sim_e2e_limit_visibility_test. Set
# XCE_REPL_BASELINE_REGEN=1 to regenerate the fixtures (see the file's
# header comment).
sched_repl_trace_baseline_test_srcs = ['test_repl_trace_baseline.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
sched_repl_trace_baseline_test_srcs += ['../' + f]
endif
endforeach
sched_repl_trace_baseline_test = executable(
'sched_repl_trace_baseline_test',
sched_repl_trace_baseline_test_srcs,
'test_repl_trace_baseline.c',
include_directories:[incdir],
dependencies: [scheduler_test_deps],
link_with: sched_srcs_lib,
c_args: ['-DXCE_SCHEDULER_BIN="' + scheduler.full_path() + '"',
'-DXCE_BASELINE_DIR="'
+ (meson.current_source_dir() / 'fixtures'
@ -2428,19 +2360,13 @@ test('scheduler repl trace baseline test',
# the inline traces, their placement in step/continue mode and at
# breakpoints, `info trace`, and the byte-identical trace file copy. The
# replay writer links the real scheduler_srcs, as the baseline test does.
sched_repl_trace_e2e_test_srcs = ['test_repl_trace_e2e.c',
'test_repl_trace_e2e_fixture.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
sched_repl_trace_e2e_test_srcs += ['../' + f]
endif
endforeach
sched_repl_trace_e2e_test = executable(
'sched_repl_trace_e2e_test',
sched_repl_trace_e2e_test_srcs,
['test_repl_trace_e2e.c',
'test_repl_trace_e2e_fixture.c'],
include_directories:[incdir],
dependencies: [scheduler_test_deps],
link_with: sched_srcs_lib,
c_args: ['-DXCE_SCHEDULER_BIN="' + scheduler.full_path() + '"',
'-DXCE_BASELINE_DIR="'
+ (meson.current_source_dir() / 'fixtures'
@ -2515,18 +2441,12 @@ test('scheduler trace capture test',
# Requirements 11.1-11.3). Real-scheduler_srcs link technique (every source
# its own translation unit), so --wrap=sched_trace_rec_new counts every
# record the allocation path creates.
sched_trace_hooks_gate_test_srcs = ['test_trace_hooks_gate.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
sched_trace_hooks_gate_test_srcs += ['../' + f]
endif
endforeach
sched_trace_hooks_gate_test = executable(
'sched_trace_hooks_gate_test',
sched_trace_hooks_gate_test_srcs,
'test_trace_hooks_gate.c',
include_directories:[incdir],
dependencies: [scheduler_deps, unity_dep],
link_with: sched_srcs_lib,
link_args: ['-Wl,--wrap=xcecli_start',
'-Wl,--wrap=xcecli_join',
'-Wl,--wrap=sched_trace_rec_new'],
@ -2541,18 +2461,12 @@ test('scheduler trace hooks gate test',
# 6.4, 11.2) on the real allocator thread. Real-scheduler_srcs link
# technique, so the decision-constructor wraps see the allocator's calls
# (decision-build failures).
sched_trace_hooks_alloc_loop_test_srcs = ['test_trace_hooks_alloc_loop.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
sched_trace_hooks_alloc_loop_test_srcs += ['../' + f]
endif
endforeach
sched_trace_hooks_alloc_loop_test = executable(
'sched_trace_hooks_alloc_loop_test',
sched_trace_hooks_alloc_loop_test_srcs,
'test_trace_hooks_alloc_loop.c',
include_directories:[incdir],
dependencies: [scheduler_deps, unity_dep],
link_with: sched_srcs_lib,
link_args: ['-Wl,--wrap=xcecli_start',
'-Wl,--wrap=xcecli_join',
'-Wl,--wrap=make_pend_decision',
@ -2568,18 +2482,12 @@ test('scheduler trace hooks alloc loop test',
# rows; Requirements 3.1-3.6) on the real sched_make_allocation() chain over
# a bare simulation core. Real-scheduler_srcs link technique; libxce's
# merge_job_alloc() is wrapped for the merge failure.
sched_trace_hooks_resreq_test_srcs = ['test_trace_hooks_resreq.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
sched_trace_hooks_resreq_test_srcs += ['../' + f]
endif
endforeach
sched_trace_hooks_resreq_test = executable(
'sched_trace_hooks_resreq_test',
sched_trace_hooks_resreq_test_srcs,
'test_trace_hooks_resreq.c',
include_directories:[incdir],
dependencies: [scheduler_deps, unity_dep],
link_with: sched_srcs_lib,
link_args: ['-Wl,--wrap=xcecli_start',
'-Wl,--wrap=xcecli_join',
'-Wl,--wrap=merge_job_alloc'],
@ -2754,18 +2662,12 @@ test('scheduler repl trace wiring test',
# wraps watch verification, recording and the breakpoint check for
# trace-only decisions, the session hook in a non-interactive run, and make
# an allocation fail on demand.
sched_sim_trace_driver_test_srcs = ['test_sim_trace_driver.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
sched_sim_trace_driver_test_srcs += ['../' + f]
endif
endforeach
sched_sim_trace_driver_test = executable(
'sched_sim_trace_driver_test',
sched_sim_trace_driver_test_srcs,
'test_sim_trace_driver.c',
include_directories:[incdir],
dependencies: [scheduler_test_deps],
link_with: sched_srcs_lib,
link_args: ['-Wl,--wrap=sim_decision_eq',
'-Wl,--wrap=sched_record_session_add_decision',
'-Wl,--wrap=xce_repl_on_decision',
@ -2852,20 +2754,14 @@ test('scheduler trace bounds test',
# snapshot, the verify result and the REPL's accounting dumps. Links the
# 6.1 fixture file for write_extra_replay() and the real scheduler_srcs for
# the replay writer and decoders.
sched_trace_equivalence_test_srcs = ['test_trace_equivalence.c',
'test_trace_equivalence_cmp.c',
'test_repl_trace_e2e_fixture.c']
foreach f : scheduler_srcs
if f != 'src/main.c'
sched_trace_equivalence_test_srcs += ['../' + f]
endif
endforeach
sched_trace_equivalence_test = executable(
'sched_trace_equivalence_test',
sched_trace_equivalence_test_srcs,
['test_trace_equivalence.c',
'test_trace_equivalence_cmp.c',
'test_repl_trace_e2e_fixture.c'],
include_directories:[incdir],
dependencies: [scheduler_test_deps],
link_with: sched_srcs_lib,
c_args: ['-DXCE_SCHEDULER_BIN="' + scheduler.full_path() + '"',
'-DXCE_BASELINE_DIR="'
+ (meson.current_source_dir() / 'fixtures'