lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
myxce/src/worker/testsuite/meson.build
lei.chen 30c51ea147 external-license-resources 3.3b: elim load-index collector in xcecd 
Per-host user-script load indices (LSF elim model): a config-declared
script collector (external_resources: [{name, command, interval}]) in
xcecd registers getters into resource_sync's load-index registry, so
user-defined dynamic resources flow the existing resval chain
(collect -> xcecli_update_resvals -> controller -> scheduler) without
any new daemon.

- elim.c: libcyaml config parse (fail-loud at load), posix_spawn script
  runner with kill-watchdog timeout (mirrors esd_script.c, no fork in
  xcecd), first-line output parse (bare number or "<name> <value>"),
  per-source interval cache (getter serves last value until next_run),
  fail-open keep-last-value on script failure
- resource_sync.c: fix two init-order traps exposed by external
  registration -- init_builtin_load_indices no longer re-callocs a
  populated g_load_indices array (clobbered entries -> strcmp garbage
  -> segfault), and register self-allocates when called before the
  first collect (capacity-0 trap); new public
  resource_sync_ensure_builtin_load_indices() so elim_register_sources
  can register before lazy builtin init and builtin-name shadowing is
  a clean -EEXIST at startup
- main.c: xcecd -c/--config option (elim config file); ownership
  main.c -> xcecd_init -> resource_sync registration, with rollback on
  init failure and cleanup in xcecd_cleanup
- tests: 18-case white-box elim_test (config/parse/getter-cache/
  register/unregister/script-runner incl. timeout no-zombie), plus
  stubs for the startup-order test
- t1/t2/t3 E2E: per-host dynres1 values visible in CLI (11/22/33 from
  each node's own script); job with rusage[dynres1=15] RUNs on t2
  (22 >= 15, t1=11 correctly skipped), RUN->DONE; over-ask
  rusage[dynres1=50] stays PEND; second saturating job overflows to
  t3; kill frees the reservation and a new job re-fills t2

Note: rusage clauses use square-bracket syntax rusage[...]; the
LSF-style paren form rusage(...) is parsed as a select expression and
rejected by the select grammar (observed live, pre-existing).

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-28 11:17:36 +00:00

309 lines
15 KiB
Meson
Raw Blame History

# Worker unit test suite (component-failover-adoption, task 2.1). White-box
# tests #include the production .c file(s) under test directly (same pattern
# as scheduler/testsuite's test_handlers.c etc.), so they link the same
# dependency set the real xcecd/xcecd_worker executables use rather than a
# subset of xcecd_srcs -- see test_controller_connector.c's header comment
# for why controller_connector.c is not also listed in xcecd_srcs here (it
# would double-define every symbol in that file).
# <unity.h> and <unity/unity.h> both resolve through unity_dep itself: the
# wrap-path override (or the nix pc under the legacy build) carries the
# include roots for both layouts, so no extra parent-includedir hack here.
worker_test_incdir = include_directories('../include', '../../include', '.',
'../src', '../../lib')
worker_test_deps = [xcecd_worker_deps, unity_dep]
controller_connector_test = executable('controller_connector_test',
['test_controller_connector.c',
'../src/pending_signal.c'],
c_args: ['-DDBUS_MANAGER',
'-DXCE_CONTROLLER_CONNECTOR_C_PATH="'
+ (meson.current_source_dir() / '..' / 'src' / 'controller_connector.c')
+ '"'],
include_directories: [worker_test_incdir],
dependencies: worker_test_deps,
install: false)
test('worker controller_connector test', controller_connector_test,
suite: 'worker',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# bug-1569: collect_load_indices() (resource_sync.c) must tag every periodic
# resource report with xcecd_ctx.virtual_name (the worker's own join
# identity), not gethostname()'s raw OS/container hostname -- see
# test_resource_sync.c's own header comment for the full mechanism trace.
resource_sync_test = executable('resource_sync_test',
'test_resource_sync.c',
include_directories: [worker_test_incdir],
dependencies: worker_test_deps,
install: false)
test('worker resource_sync test', resource_sync_test,
suite: 'worker',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# external-license-resources design §3.3b: the elim collector -- a
# config-driven script runner (posix_spawn + watchdog) that registers
# user-defined per-host load indices into resource_sync's registry at
# worker startup. White-box #includes elim.c AND resource_sync.c (the
# registry capacity-0-before-init trap is only observable against the real
# registry); see test_elim.c's header comment.
elim_test = executable('elim_test',
'test_elim.c',
include_directories: [worker_test_incdir],
dependencies: worker_test_deps,
install: false)
test('worker elim test', elim_test,
suite: 'worker',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# bug-1554: real maxswap (sysinfo() totalswap)/maxtmp (statvfs("/tmp"))
# detection -- white-box #include of hardware_monitor.c, same convention as
# resource_sync_test above.
hardware_monitor_test = executable('hardware_monitor_test',
'test_hardware_monitor.c',
include_directories: [worker_test_incdir],
dependencies: worker_test_deps,
install: false)
test('worker hardware_monitor test', hardware_monitor_test,
suite: 'worker',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# component-failover-adoption task 2.2: worker startup-ordering white-box
# test. #include's xcecd.c directly (same technique as controller_connector_
# test above, but a separate binary/file -- xcecd.c and controller_connector.c
# cannot both be #include-d into one TU, see test_xcecd_startup_order.c's
# header comment). Same -DDBUS_MANAGER + -DVIRTUAL_CLUSTER flag pair the real
# `xcecd_virtual` executable already uses (see xcecd_virtual target above).
xcecd_startup_order_test = executable('xcecd_startup_order_test',
'test_xcecd_startup_order.c',
c_args: ['-DDBUS_MANAGER', '-DVIRTUAL_CLUSTER'],
include_directories: [worker_test_incdir],
dependencies: worker_test_deps,
install: false)
test('worker xcecd_startup_order test', xcecd_startup_order_test,
suite: 'worker',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# bug-1589: dbus_workers_update() (xcecd_dbus_manager.c) worker-list
# traversal regression test -- see test_dbus_workers_update.c's header
# comment for the full root-cause trace. Same -DDBUS_MANAGER
# -DVIRTUAL_CLUSTER flag pair the real `xcecd_virtual` executable uses
# (worker/meson.build); DBUS_MANAGER has no effect on xcecd_dbus_manager.c
# itself (no #ifdef DBUS_MANAGER branches there) but is included to match
# production's exact build flags.
dbus_workers_update_test = executable('dbus_workers_update_test',
'test_dbus_workers_update.c',
c_args: ['-DDBUS_MANAGER', '-DVIRTUAL_CLUSTER'],
include_directories: [worker_test_incdir],
dependencies: worker_test_deps,
install: false)
test('worker dbus_workers_update test', dbus_workers_update_test,
suite: 'worker',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# User-reported log-severity bug: a deliberately-killed job (SIGTERM/SIGKILL
# via worker_signal_callback(), xcecd_worker.c) was logged at ERROR by
# ipc_worker_send_status() (uv_ipc.c) even though process_exit_callback()
# two lines earlier already judges the same event INFO-worthy. See
# test_uv_ipc_job_exit_severity.c's own header comment.
uv_ipc_job_exit_severity_test = executable('uv_ipc_job_exit_severity_test',
'test_uv_ipc_job_exit_severity.c',
include_directories: [worker_test_incdir],
dependencies: worker_test_deps,
install: false)
test('worker uv_ipc job_exit_severity test', uv_ipc_job_exit_severity_test,
suite: 'worker',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# worker-rejoin-resync task 4.1 (Requirements 3.4, 3.5, 3.7): bounded
# finished-job retention -- white-box #include of xcecd_worker_utils.c,
# same convention as resource_sync_test above.
finished_job_retention_test = executable('finished_job_retention_test',
['test_finished_job_retention.c', '../src/pending_signal.c'],
include_directories: [worker_test_incdir],
dependencies: worker_test_deps,
install: false)
test('worker finished_job_retention test', finished_job_retention_test,
suite: 'worker',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# worker-rejoin-resync task 4.2 (Requirements 3.1, 3.2, 3.3, 3.6, 8.2, 8.3):
# answer the controller's job-state query (running list + task 4.1's
# finished-job retention), including extras -- white-box #include of both
# ../src/xcecd_worker_utils.c and ../src/xcecd_resync.c, same convention
# as finished_job_retention_test above.
xcecd_resync_test = executable('xcecd_resync_test',
['test_xcecd_resync.c', '../src/pending_signal.c'],
include_directories: [worker_test_incdir],
dependencies: worker_test_deps,
install: false)
test('worker xcecd_resync test', xcecd_resync_test,
suite: 'worker',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# resreq-maxtasks-wiring task 2.3 (Requirement 4.4): closes a pre-existing,
# zero-coverage gap in the worker's launch-type dispatch decision --
# job_start_remote_task()'s MPI-vs-default launch-type gate and
# create_mpi_cmdline()'s command-line construction -- against unmodified
# production code (pure test addition, no production change). Same
# white-box #include of controller_connector.c and -DDBUS_MANAGER flag as
# controller_connector_test above; see test_worker_launch_type_dispatch.c's
# own header comment for why xcecli_runtask()/xcecli_unblock_tasks() cannot
# be stubbed here the way the other job-handling externs above are.
launch_type_dispatch_test = executable('launch_type_dispatch_test',
['test_worker_launch_type_dispatch.c', '../src/pending_signal.c'],
c_args: ['-DDBUS_MANAGER',
'-DXCE_CONTROLLER_CONNECTOR_C_PATH="'
+ (meson.current_source_dir() / '..' / 'src' / 'controller_connector.c')
+ '"'],
include_directories: [worker_test_incdir],
dependencies: worker_test_deps,
install: false)
test('worker launch_type_dispatch test', launch_type_dispatch_test,
suite: 'worker',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# Live-reported bug: a replay session's `info worker` showed
# "resource mem: ... used=334251534843904.000000 (304.00 TB)". Root-caused
# to collect_job_rusages() (xcecd_cgroups.c) trusting a systemd
# UINT64_MAX "resource counter not yet available" sentinel as a real
# reading. White-box #include of xcecd_cgroups.c, same convention as
# hardware_monitor_test/resource_sync_test above -- see
# test_xcecd_cgroups_rusage.c's own header comment for the exact
# reproduced arithmetic and the D-Bus/systemd sandbox limitation.
xcecd_cgroups_rusage_test = executable('xcecd_cgroups_rusage_test',
'test_xcecd_cgroups_rusage.c',
include_directories: [worker_test_incdir],
dependencies: worker_test_deps,
install: false)
test('worker xcecd_cgroups rusage test', xcecd_cgroups_rusage_test,
suite: 'worker',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# bug-1868: the root worker's per-task output file creation must not follow
# a symlink/hardlink planted at its predictable /tmp name (job_output_path.h).
job_output_path_test = executable('job_output_path_test',
'test_job_output_path.c',
include_directories: [worker_test_incdir],
dependencies: worker_test_deps,
install: false)
test('worker job_output_path test', job_output_path_test,
suite: 'worker',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# bug-1869: xcecd's own fixed-name /tmp files (xcecd_worker.log, the
# restart flag) must not follow links planted at those names.
xcecd_tmpfile_test = executable('xcecd_tmpfile_test',
'test_xcecd_tmpfile.c',
include_directories: [worker_test_incdir],
dependencies: worker_test_deps,
install: false)
test('worker xcecd_tmpfile test', xcecd_tmpfile_test,
suite: 'worker',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# bug-1872: which /proc/diskstats lines are physical disks (diskstats.h);
# NVMe and mmc disks were never counted.
diskstats_test = executable('diskstats_test',
'test_diskstats.c',
include_directories: [worker_test_incdir],
dependencies: worker_test_deps,
install: false)
test('worker diskstats test', diskstats_test,
suite: 'worker',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# job-array-one-based-index task 3.2 (Requirements 6.1-6.5, 7.8): XCE_JOBINDEX
# in a running job's environment -- white-box #include of xcecd_exdata.c.
exdata_jobindex_test = executable('exdata_jobindex_test',
'test_exdata_jobindex.c',
include_directories: [worker_test_incdir],
dependencies: worker_test_deps,
install: false)
test('worker exdata_jobindex test', exdata_jobindex_test,
suite: 'worker',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# bug-1937: parse_exdata()/parse_limits() on the exdata JSON the apiserver
# builds ("transfer_files" key) -- white-box #include of xcecd_exdata.c.
exdata_parse_test = executable('exdata_parse_test',
'test_exdata_parse.c',
include_directories: [worker_test_incdir],
dependencies: worker_test_deps,
install: false)
test('worker exdata_parse test', exdata_parse_test,
suite: 'worker',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# bug-1936: the job runs in its submission's cwd with HOME = exec home --
# job_assignment_t carries both (uv_ipc.c), uv_proc_enter_job_dirs()
# (uv_proc.c) applies them, and a spawned job inherits them. White-box
# #include of uv_ipc.c and uv_proc.c.
job_dirs_test = executable('job_dirs_test',
'test_job_dirs.c',
include_directories: [worker_test_incdir],
dependencies: worker_test_deps,
install: false)
test('worker job_dirs test', job_dirs_test,
suite: 'worker',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# bug-1908: pending-start signal table (pending_signal.c).
pending_signal_test = executable('pending_signal_test',
['test_pending_signal.c',
'../src/pending_signal.c'],
include_directories: [worker_test_incdir],
dependencies: worker_test_deps,
install: false)
test('worker pending_signal test', pending_signal_test,
suite: 'worker',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# bug-1954: job_signal_to_deliver() (job_signal.c), the signal a job-control
# request delivers to the job.
job_signal_test = executable('job_signal_test',
['test_job_signal.c', '../src/job_signal.c'],
include_directories: [worker_test_incdir],
dependencies: worker_test_deps,
install: false)
test('worker job_signal test', job_signal_test,
suite: 'worker',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
Reference in a new issue View git blame Copy permalink