lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(scheduler): invalidate stale builtin_fcfs picker cursors across calls (bug-1686)

Browse source 
User-reported live crash: xce_scheduler SIGSEGV in fcfs_cursor_enter_queue
(builtin_fcfs.c), called from builtin_fcfs_next_job <- sched_job_picker
(picker thread).

priv->cursor/priv->job_cursor (fcfs_plugin_data_t) are raw ordq_item_t*
pointers that persist across SEPARATE next_job() calls, each with its own
independent priv->lock acquisition/release. The existing mutual-exclusion
fix (task 2.4) correctly protects the cursor walk from corruption DURING
one locked call, but does nothing to stop a LATER call from dereferencing
a node a properly-locked, INTERVENING call already freed.
on_queue_update()'s re-sort (remove old ordq_item_t node, insert a fresh
one via ordq_put) is exactly such a free -- with only one queue in the
deployment, the picker's cursor is essentially always parked on it
between calls (sched_queue_is_ready/sched_job_is_ready are unconditional
true stubs today, so the cursor never advances away while jobs remain),
making a queue config update land in that window trivially reachable.

Fixed by invalidating (NULLing) whichever cursor points at a node right
before that node is freed, at all three sites that can free a node the
cursors might reference: on_queue_update's reuse-and-resort path
(priv->cursor + priv->job_cursor in lockstep), builtin_fcfs_on_job_state_change's
!JOB_IS_PEND removal, and fcfs_remove_job_everywhere's cross-queue removal
loop (both job_cursor only, via an ordq_peek-then-invalidate-then-delete
pattern). Always safe: next_job() treats a NULL cursor as "no more
candidates this round", and begin_session() resets both cursors fresh
every round regardless -- worst case is a bounded one-round delay, never
a crash or a permanently lost job.

New test_builtin_fcfs_cursor_uaf.c reproduces the bug fully single-threaded
(next_job() releases the lock between calls, so no real concurrency is
needed to hit the window). RED-phase independently confirmed a real ASAN
heap-use-after-free with the exact same call chain as the live crash
(freed by on_queue_update, read by next_job()); GREEN confirms 0 failures
and no job permanently lost across the invalidation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AgYRU3zDTqNGV5Jk8aiKLg
This commit is contained in:
Rongsong Shen 2026-08-25 08:49:43 +00:00
parent b9f13355df
commit bc3f76b42e
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
4 changed files with 324 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

22
src/scheduler/testsuite/meson.build
View file

@ -1139,6 +1139,25 @@ sched_limit_plugin_test = executable('sched_limit_plugin_test',
dependencies: [scheduler_test_deps],
install: false)
# White-box regression test for bug-1686 (live SIGSEGV in
# fcfs_cursor_enter_queue() <- builtin_fcfs_next_job(), a heap-use-after-free
# on priv->cursor/job_cursor -- raw ordq_item_t* values that persist across
# separate next_job() calls and can be freed in between by a properly-locked
# on_queue_update() re-sort of the same queue). It #includes
# ../src/plugins/builtin_fcfs.c directly (same white-box pattern as
# sched_picker_fcfs_session_test) so it exercises the real cursor logic and
# the real private fcfs_plugin_data_t layout -- including direct field
# access to priv->cursor/priv->job_cursor, which only a white-box #include
# gives a test -- not a re-implementation of any of it. Supplies its own
# always-ready sched_queue_is_ready()/sched_job_is_ready() test doubles and a
# plugin_register() stub, same convention as sched_picker_fcfs_session_test.
sched_builtin_fcfs_cursor_uaf_test = executable(
'sched_builtin_fcfs_cursor_uaf_test',
['test_builtin_fcfs_cursor_uaf.c'],
include_directories:[incdir],
dependencies: [scheduler_test_deps],
install: false)
# Register as meson tests so `meson test --suite scheduler` runs them all.
# detect_leaks=0 matches the suite convention (fixtures are not torn down).
test('scheduler test', scheduler_test, suite: 'scheduler',
@ -1288,3 +1307,6 @@ test('scheduler dispatch commit test',
env: ['ASAN_OPTIONS=detect_leaks=0'])
test('scheduler limit plugin test', sched_limit_plugin_test, suite: 'scheduler',
env: ['ASAN_OPTIONS=detect_leaks=0'])
test('scheduler builtin fcfs cursor uaf test',
sched_builtin_fcfs_cursor_uaf_test, suite: 'scheduler',
env: ['ASAN_OPTIONS=detect_leaks=0'])