Rongsong Shen
6d510d8cf4
When the (still-default-OFF) peer-control dispatch flag is ON, raft_cb_become_follower runs as a vtask pool worker and frees every peer vote-ep while other pool workers may still hold those eps' data — a UAF (bug-1197). This makes that path quiescence-safe so the flag can later be flipped ON (deferred to the 4.x prod-flip gate; production stays OFF here). Mechanism (no libxce-wide endpoint refcount; reuses the existing per-ep FIFO + busy serialization): - Deferred teardown-as-vtask: the loser loop, when dispatch is ON and the pool is wired, does per vote-ep endpoint_mark_tearing_down -> iomgr_unmonitor -> enqueue a VTASK_IO/VTASK_LIVENESS teardown vtask bound to that ep. Per-ep busy FIFO makes it run strictly after any in-flight/queued same-ep peer-control vtask; the body NULLs iod->ep before endpoint_fini+free(ep) to disarm run_task's post-body endpoint_unblock_tasks (which would touch the freed ep->lock). OFF / pool-unwired path is byte-for-byte the legacy endpoint_fini. - RCU-like drain closes the dispatch/teardown TOCTOU window: new _Atomic int endpoint_t.dispatch_inflight (+ tearing_down:1 bit). The ON dispatch brackets its flag-check+enqueue with fetch_add/fetch_sub (seq_cst); endpoint_mark_tearing_down sets the flag under ep->lock, releases, then spins until dispatch_inflight drains to 0. So every dispatch that read the flag false finishes its enqueue before mark returns (FIFO-before the teardown vtask); any later dispatch reads the flag set and drops the packet. ep->lock is deliberately NOT held across the enqueue (it inverts with the taskq lock in vtask_eligible); the counter is the lock-free handshake. Drain is bounded: the held window is just a calloc + non-blocking enqueue on the single io_worker. Still open before the production flag flip (4.x): the borrowed-ep race in snapshot_lane_fn/configpush_lane_fn is distinct (slowlane thread vs io_worker RDHUP teardown, not the vtask-pool FIFO) and needs its own per-standby-ep discipline. Tests: test_ha_vep_teardown.c — 5 ASan tests (ordering, late-event drop, run_task unblock disarm, OFF-path unchanged, deterministic mark-drains-inflight-dispatch). xcelib_test 282/0; controller_unit_test EXIT=0; ASan-clean. Adversarial cpp-review: no CRITICAL/HIGH; handshake formally sound, flag-ON path race-free. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S6fDCtXkkEgupu7rUah1y2
193 lines
5.1 KiB
Meson
193 lines
5.1 KiB
Meson
controller_src = [
|
|
'cluster.c',
|
|
'commit.c',
|
|
'config_repl.c',
|
|
'ctrlstatus.c',
|
|
'dataex.c',
|
|
'egress.c',
|
|
'ha.c',
|
|
'ha_convergence.c',
|
|
'job.c',
|
|
'join.c',
|
|
'main.c',
|
|
'misc.c',
|
|
'myapp.c',
|
|
'opget.c',
|
|
'opupd.c',
|
|
'queue.c',
|
|
'reply.c',
|
|
'resources.c',
|
|
'sched.c',
|
|
'startup_validation.c',
|
|
'task.c',
|
|
'worker.c',
|
|
'apiserver.c',
|
|
'myconf.c',
|
|
'persist.c',
|
|
'repl.c',
|
|
'ctrl_metrics.c',
|
|
'slowlane.c',
|
|
]
|
|
|
|
incdir = include_directories('.', '../include', '../lib',
|
|
'include')
|
|
|
|
# cc = meson.get_compiler('c')
|
|
# math_dep = cc.find_library('m', required : false)
|
|
|
|
zlib_dep = dependency('zlib')
|
|
|
|
controller_deps = [
|
|
libxce_dep,
|
|
librtti_dep,
|
|
c_capnproto_dep,
|
|
math_dep,
|
|
tinylog_dep,
|
|
thread_dep,
|
|
zlib_dep,
|
|
]
|
|
|
|
controller = executable('xce_controller',
|
|
controller_src,
|
|
include_directories: incdir,
|
|
dependencies: controller_deps,
|
|
link_args: wrap_link_args,
|
|
install: true)
|
|
|
|
xce_test_src = [ 'testsuite/test.c', 'testsuite/waldump.c', 'testsuite/walcompare.c']
|
|
|
|
readline_dep = dependency('readline')
|
|
ncurses_dep = dependency('ncurses')
|
|
history_dep = dependency('history')
|
|
|
|
xce_test_deps = [
|
|
libxce_dep,
|
|
librtti_dep,
|
|
struct2json_dep,
|
|
readline_dep,
|
|
history_dep,
|
|
ncurses_dep,
|
|
tinylog_dep,
|
|
zlib_dep, # waldump: crc32 WAL validation
|
|
c_capnproto_dep, # waldump: decode JOB_SUBMIT jobid
|
|
]
|
|
xce_test = executable('xce_test',
|
|
xce_test_src,
|
|
include_directories: [incdir],
|
|
dependencies: xce_test_deps,
|
|
install: true)
|
|
|
|
# Controller Unity test suite (persist types and future controller unit tests)
|
|
controller_unit_test_src = [
|
|
'testsuite/controller_test.c',
|
|
'testsuite/test_persist_types.c',
|
|
'testsuite/test_persist_wal.c',
|
|
'testsuite/test_persist_schema.c',
|
|
'testsuite/test_persist_checkpoint.c',
|
|
'testsuite/test_persist_recovery.c',
|
|
'testsuite/test_hooks.c',
|
|
'testsuite/test_persist_hooks.c',
|
|
'testsuite/test_gc_retention.c',
|
|
'testsuite/test_cluster_persist.c',
|
|
'testsuite/test_e2e_wal.c',
|
|
'testsuite/test_e2e_checkpoint.c',
|
|
'testsuite/test_e2e_hooks.c',
|
|
'testsuite/test_e2e_gc.c',
|
|
'testsuite/test_e2e_capnp.c',
|
|
'testsuite/test_ha_role.c',
|
|
'testsuite/test_ha_role_lifecycle.c',
|
|
'testsuite/test_ha_cluster.c',
|
|
'testsuite/test_ha_redirect.c',
|
|
'testsuite/test_join_redirect.c',
|
|
'testsuite/test_p1_integration.c',
|
|
'testsuite/test_repl_unit.c',
|
|
'testsuite/test_repl_apply.c',
|
|
'testsuite/test_persist_repl.c',
|
|
'testsuite/test_repl_catchup.c',
|
|
'testsuite/test_p2_integration.c',
|
|
'testsuite/test_p3_hardening.c',
|
|
'testsuite/test_p3_manual_failover.c',
|
|
'testsuite/test_ctrl_metrics.c',
|
|
'testsuite/test_config_repl.c',
|
|
'testsuite/test_worker_join.c',
|
|
'testsuite/test_ha_startup.c',
|
|
'testsuite/test_ha_election.c',
|
|
'testsuite/test_ha_bootstrap.c',
|
|
'testsuite/test_ha_bootstrap_support.c',
|
|
'testsuite/test_ha_durability.c',
|
|
'testsuite/test_repl_majority.c',
|
|
'testsuite/test_persist_truncate.c',
|
|
'testsuite/test_ha_ctrlhello.c',
|
|
'testsuite/test_ha_failover.c',
|
|
'testsuite/test_ha_vote_send.c',
|
|
'testsuite/test_ha_start_election_async.c',
|
|
'testsuite/test_ha_vote_gate.c',
|
|
'testsuite/test_ha_repl_push.c',
|
|
'testsuite/test_ha_connection_reuse.c',
|
|
'testsuite/test_ha_rdhup_integration.c',
|
|
'testsuite/test_ha_liveness.c',
|
|
'testsuite/test_ha_prevote.c',
|
|
'testsuite/test_ha_failover_prevote.c',
|
|
'testsuite/test_commit_engine.c',
|
|
'testsuite/test_commit_barrier_rollout.c',
|
|
'testsuite/test_commit_barrier_reply.c',
|
|
'testsuite/test_egress_filter.c',
|
|
'testsuite/test_demotion_convergence.c',
|
|
'testsuite/test_convergence_verification.c',
|
|
'testsuite/test_coldstart_lsn0.c',
|
|
'testsuite/test_commit_durability.c',
|
|
'testsuite/test_ctrlstatus.c',
|
|
'testsuite/test_ha_heartbeat_committed.c',
|
|
'testsuite/test_ha_vote_floor_heartbeat.c',
|
|
'testsuite/test_term_safety_adapter.c',
|
|
'testsuite/test_drive_async_config.c',
|
|
'testsuite/test_slowlane.c',
|
|
'testsuite/test_commit_admission_pool.c',
|
|
'testsuite/test_ha_vep_teardown.c',
|
|
'testsuite/test_waldump.c',
|
|
'testsuite/waldump.c',
|
|
'testsuite/test_walcompare.c',
|
|
'testsuite/walcompare.c',
|
|
'ctrlstatus.c',
|
|
'commit.c',
|
|
'job.c',
|
|
'egress.c',
|
|
'ha_convergence.c',
|
|
'queue.c',
|
|
'join.c',
|
|
'opupd.c',
|
|
'startup_validation.c',
|
|
'worker.c',
|
|
'persist.c',
|
|
'ha.c',
|
|
'repl.c',
|
|
'ctrl_metrics.c',
|
|
'config_repl.c',
|
|
'slowlane.c',
|
|
]
|
|
|
|
ssl_dep = dependency('openssl')
|
|
|
|
controller_unit_test_deps = [
|
|
libxce_dep,
|
|
librtti_dep,
|
|
unity_dep,
|
|
tinylog_dep,
|
|
thread_dep,
|
|
zlib_dep,
|
|
c_capnproto_dep,
|
|
ssl_dep,
|
|
]
|
|
|
|
controller_unit_test = executable('controller_unit_test',
|
|
controller_unit_test_src,
|
|
include_directories: [incdir],
|
|
dependencies: controller_unit_test_deps,
|
|
c_args: ['-DXCE_TESTING'],
|
|
install: false)
|
|
|
|
test('controller test', controller_unit_test,
|
|
suite: 'controller',
|
|
timeout: 300,
|
|
env: ['ASAN_OPTIONS=detect_leaks=0'])
|
|
|