lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(raft-drive-async): peer-control vtask dispatch shim, dormant (task 3.1)

Browse source 
Lands the PeerCtrlDispatch machinery behind a DEFAULT-OFF flag
(g_drive_async_peerctrl_dispatch_enabled); production behavior is
UNCHANGED. peer_ctrl_msg_handler becomes a thin shim that takes
ownership of ep->fsm_in.pkt and either runs the routing body inline
(flag OFF, byte-for-byte the prior path) or dispatches it to the vtask
pool via vtask_enq_classified (flag ON). The 3 routers now take pkt as
an explicit param (no cross-thread ep->fsm_in.pkt re-read). Per-endpoint
arrival order is enforced by bind_vtask's endpoint_block_tasks.

Packet ownership: shim clears ep->fsm_in.pkt at dispatch (sole-owner
transfer); peer_ctrl_vtask_entry NULLs iod->pkt before the body frees
it once. New vtask_t.owns_io_pkt (default 0) lets taskq_item_killer free
the carried packet for an un-executed queued task at vtaskmgr_fini WITHOUT
double-freeing the libxce endpoint-owned path (bug-1199).

Adversarial review found two FLAG-ON hazards, documented with loud
comments (flag is OFF so production is unaffected), to be resolved before
task 3.4 flips the flag ON:
  - bug-1197 (CRITICAL): raft_cb_become_follower endpoint_fini()s peer
    vote-eps from a pool worker, racing the io_worker's read of ep->data;
    needs iomgr_unmonitor before endpoint_fini in every pool-reachable
    teardown.
  - bug-1198 (HIGH): peer_ctrl_classify tags all work VTASK_LIVENESS, but
    the CTRL_PRIMARY repl-ack path can run a blocking snapshot checkpoint
    inline; resolved by task 3.2 (offload snapshot to SlowLane).

Per spec, 3.1 is not 'done' until the task 3.4 deadlock gate is green.
Tests: 3 new peerctrl tests; controller 196/196, lib 282/0; 0 net leaks.
This commit is contained in:
Rongsong Shen 2026-06-26 11:12:04 +00:00
parent 21e26a17b6
commit e17a6a613c
7 changed files with 1678 additions and 31 deletions
Download patch file Download diff file Expand all files Collapse all files

21
src/lib/vtaskmgr.c
View file

@ -27,6 +27,7 @@ int vtask_init(vtask_t* task, vtask_func_t entry, void* data)
task->data = data;
task->owner = NULL;
task->cls = VTASK_LIVENESS; /* never-gated by default; legacy enq paths */
task->owns_io_pkt = 0; /* legacy: endpoint owns the packet (task 3.1) */
return 0;
}
@ -166,6 +167,26 @@ static void taskq_item_killer(data_item_t* it)
{
if (it != NULL) {
vtask_t* task = ITPTR_TO_VTASK(it);
/* raft-drive-async task 3.1: a task destroyed while still queued (never
* executed) at vtaskmgr_fini would otherwise leak its carried packet —
* the ownership transfer (iod->pkt = NULL) only happens inside the
* executed entry (peer_ctrl_vtask_entry). Free the packet here ONLY
* when the producer marked the task as the packet's sole owner
* (owns_io_pkt); the legacy endpoint-owned model (owns_io_pkt == 0,
* e.g. the libxce client path) is left untouched so we never double-free
* a packet whose lifetime is the endpoint's. This runs only on
* un-executed tasks, so it cannot race the entry's free. vtask_fini
* frees only task->data (the wrapper struct), never the packet. */
if (task->owns_io_pkt && task->data != NULL &&
task->data->vtype == VTASK_IO &&
task->data->u.iodata.pkt != NULL) {
io_packet_t* pkt = task->data->u.iodata.pkt;
free_packet(pkt);
free(pkt);
task->data->u.iodata.pkt = NULL;
}
vtask_fini(task);
free(task);
}