lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(raft-drive-async): peer-control vtask dispatch shim, dormant (task 3.1)

Browse source 
Lands the PeerCtrlDispatch machinery behind a DEFAULT-OFF flag
(g_drive_async_peerctrl_dispatch_enabled); production behavior is
UNCHANGED. peer_ctrl_msg_handler becomes a thin shim that takes
ownership of ep->fsm_in.pkt and either runs the routing body inline
(flag OFF, byte-for-byte the prior path) or dispatches it to the vtask
pool via vtask_enq_classified (flag ON). The 3 routers now take pkt as
an explicit param (no cross-thread ep->fsm_in.pkt re-read). Per-endpoint
arrival order is enforced by bind_vtask's endpoint_block_tasks.

Packet ownership: shim clears ep->fsm_in.pkt at dispatch (sole-owner
transfer); peer_ctrl_vtask_entry NULLs iod->pkt before the body frees
it once. New vtask_t.owns_io_pkt (default 0) lets taskq_item_killer free
the carried packet for an un-executed queued task at vtaskmgr_fini WITHOUT
double-freeing the libxce endpoint-owned path (bug-1199).

Adversarial review found two FLAG-ON hazards, documented with loud
comments (flag is OFF so production is unaffected), to be resolved before
task 3.4 flips the flag ON:
  - bug-1197 (CRITICAL): raft_cb_become_follower endpoint_fini()s peer
    vote-eps from a pool worker, racing the io_worker's read of ep->data;
    needs iomgr_unmonitor before endpoint_fini in every pool-reachable
    teardown.
  - bug-1198 (HIGH): peer_ctrl_classify tags all work VTASK_LIVENESS, but
    the CTRL_PRIMARY repl-ack path can run a blocking snapshot checkpoint
    inline; resolved by task 3.2 (offload snapshot to SlowLane).

Per spec, 3.1 is not 'done' until the task 3.4 deadlock gate is green.
Tests: 3 new peerctrl tests; controller 196/196, lib 282/0; 0 net leaks.
This commit is contained in:
Rongsong Shen 2026-06-26 11:12:04 +00:00
parent 21e26a17b6
commit e17a6a613c
7 changed files with 1678 additions and 31 deletions
Download patch file Download diff file Expand all files Collapse all files

5
src/controller/cluster.c
View file

@ -1362,6 +1362,11 @@ int cluster_init_phase2(cluster_t* c)
* until peer_ctrl dispatches classified tasks (task 3.1). */
vtaskmgr_set_liveness_reserved(&c->vtaskmgr, liveness_reserved);
commit_set_pool(&c->commit, &c->vtaskmgr);
/* raft-drive-async task 3.1: give ha the pool handle so
* peer_ctrl_msg_handler can dispatch peer-control messages onto it when
* the dispatch flag is ON. Dormant until the flag flips (default OFF);
* stored as void* (ha.h does not pull vtaskmgr.h). */
c->ha.pool = &c->vtaskmgr;
if (c->vtaskmgr.n_vcpus < c->ha.n_peers) {
/* Under-provisioned: the n_vcpus ceiling clamped the reservation
* below the ideal n_peers-1. Still correct (deadlock-free via fast