lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(controller): sched_worker_change() use-after-free on worker disconnect (bug-1578)

Browse source 
sched_worker_change() aliased w->host.resvals_to_sched to a local xlist_t's
backing array, pushed it to the scheduler, then freed that array via
xlist_fini() without resetting the alias - leaving a dangling pointer for
the rest of the worker's life. On disconnect (handle_worker_leave ->
mark_worker_unavail -> persist_worker_hook_fun -> capnp_encode_worker ->
encode_Worker -> encode_NameValue_list), this dereferenced freed memory and
crashed the controller - but only once a scheduler had ever connected
(on_scheduler_join gates the hook registration), which made it easy to miss
in isolated worker-only testing.

The correct reset already existed in the function but was dead-code'd
behind #if 0 /*FIXME*/ - apparently from a prior fix attempt that got
wrapped instead of landed. Fix mirrors the proven-correct pattern already
used by the sibling apiserver_worker_change(): null the alias before
xlist_fini() frees the backing array, rather than double-freeing via
free_name_val_list() first.

Adds test_sched_worker_change.c, a regression test driving the real
production hook-dispatch path (xce_run_hooks -> sched_worker_hook_fun ->
sched_worker_dead -> sched_worker_change) and the real encode_Worker()
encoder the original crash traversed - confirmed to reproduce the exact
ASAN heap-use-after-free when the fix is reverted, and pass clean when
restored, closing the gap that let this regress silently once already.

Found while live-testing the resource-value-verification feature (deploying
a scheduler + injecting worker resource values via xce_test, then
disconnecting) - unrelated to that feature's own changes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-08-18 08:15:37 +00:00
parent 23c3fd6b6a
commit b406dd762a
Signed by: rshen
SSH key fingerprint: SHA256:8Vf2pZ3h6rJa7EhGK+WaAQ9Mta/apZvteW6g9U/WKKk
3 changed files with 392 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

30
src/controller/meson.build
View file

@ -255,6 +255,36 @@ test('sched push test', sched_push_unit_test,
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# bug-1578 regression test: sched_worker_change()'s dangling-pointer fix
# (w->host.resvals_to_sched/nresvals_to_sched left aliased onto a freed
# xlist_t backing array -- see sched.c's sched_worker_change() for the full
# history). Same real-vs-fake link-collision reason as sched_push_unit_test
# just above requires its own standalone binary rather than folding into
# controller_unit_test_src: sched_worker_change() is only reachable by
# linking the REAL sched.c/dataex.c/egress.c, which collides with other
# controller_unit_test_src files' own fake on_scheduler_join()/notify()
# definitions. Mirrors sched_push_unit_test_src's exact source-set strategy
# (including opget.c's check_worker_() being stubbed locally rather than
# linking all of opget.c).
sched_worker_change_unit_test_src = [
'testsuite/test_sched_worker_change.c',
'sched.c',
'dataex.c',
'egress.c',
]
sched_worker_change_unit_test = executable('sched_worker_change_unit_test',
sched_worker_change_unit_test_src,
include_directories: [incdir],
dependencies: controller_unit_test_deps,
c_args: ['-DXCE_TESTING'],
install: false)
test('sched worker change test', sched_worker_change_unit_test,
suite: 'controller',
timeout: 60,
env: ['ASAN_OPTIONS=detect_leaks=0'])
# job-array-single-wal-record task 5.2: apiserver.c's push logic
# (apiserver_job_new_fun / apiserver_push_jobarray) faces the identical
# real-vs-fake link collision task 5.1 hit for sched.c (test_commit_barrier_