feat(raft-drive-async): offload requester-less config-push to slow lane (task 3.3)
push_config_transport_only spawns one thread per standby then joins them with a CONF_PUSH_TIMEOUT_MS bounded wait — blocking the caller. On the requester-less drive-thread paths (promotion + heartbeat convergence) that wait stalls the io_worker. Route those through ha_config_push_offloadable, gated by g_drive_async_configpush_offload (default OFF → inline, byte-for-byte legacy; ON → slow lane, non-blocking). Flag ON deep-copies the config bytes + sha into a lane ctx (caller keeps owning and freeing its own buffer); a single cluster-wide single-flight guard (ha_state_t.configpush_inflight scalar CAS + lane key=1) bounds it to one config-push fan-out in flight; the in-flight flag is cleared on every terminal path (two alloc failures, -EEXIST, hard submit error, lane completion). Send body factored into ha_config_push_do_send. Three requester-less drive-thread call sites converted: raft_cb_become_leader (promotion), the HA_REPL_PUSH_CONFIG heartbeat case, and the direct-CTRL_PRIMARY FSM promotion branch. The admin-upgrade barrier (push_config_to_standbys, has a waiting requester) and join.c are untouched. Lands DORMANT like tasks 3.1/3.2: the lane reads c->ha.standby_endpoints and writes those borrowed standby endpoints, racing io_worker RDHUP teardown if run off-thread (bug-1197 family). Flag OFF keeps push+teardown serialized on the io_worker. Task 3.4 flips this ON together with the snapshot and peerctrl flags after establishing endpoint-lifetime discipline. Tests (controller suite EXIT=0): test_configpush_offload_runs_on_lane, test_configpush_inline_when_flag_off, test_configpush_offload_dedups. Adversarial cpp-review: no CRITICAL/HIGH/MEDIUM findings. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S6fDCtXkkEgupu7rUah1y2
This commit is contained in:
parent
6c0523b69b
commit
bd02064ebc
5 changed files with 515 additions and 13 deletions
|
|
@ -96,7 +96,7 @@ Tasks mirror the design's 4-phase migration. The slow lane is foundational (ever
|
|||
- _Boundary: PeerCtrlDispatch, SlowLane_
|
||||
- _Depends: 3.1_
|
||||
|
||||
- [ ] 3.3 (P) Offload the transport-only config-push to the slow lane
|
||||
- [~] 3.3 (P) Offload the transport-only config-push to the slow lane <!-- landed DORMANT (flag g_drive_async_configpush_offload default OFF) pending 3.4 flip — see Implementation Notes -->
|
||||
- Move the requester-less iomgr/promotion config-push transport (and its wait) to the lane; replace the drive-thread join with a lane-internal wait and non-blocking return; leave the admin upgrade path (with a waiting requester) on its existing vtask-blocking path unchanged.
|
||||
- Observable: a promotion/heartbeat-driven config push no longer blocks iomgr; the admin upgrade reply path is unchanged.
|
||||
- _Requirements: 1.2_
|
||||
|
|
@ -148,3 +148,5 @@ Tasks mirror the design's 4-phase migration. The slow lane is foundational (ever
|
|||
- **Part B (production-live, default-ON, no flag)**: `trim_wal_segments` (`persist.c`) now bounds deletion by `floor = repl_trim_lsn(&p->repl, active_seg_lsn, snap_interval_s)` = `min(active_seg_lsn, every active standby acked_lsn, every valid disconnected-slot last_disconnected_lsn)`, after a `repl_expire_disconnect_slots` call to age out stale slot guards. `floor ≤ active_seg_lsn` structurally (repl_trim_lsn caps at its `checkpoint_lsn` arg), so the new `seg_start_lsn < floor` check subsumes the old `< active_seg_lsn` guard and can only PRESERVE more segments — never delete one a live standby still needs (closes the bug-795/828 trim-vs-cursor race for connected/recently-disconnected standbys; does NOT fix empty/cursor=0 join — needs entity-bearing snapshots, separate spec). Lock-safe: `trim_wal_segments` runs with `p->lock` released; `repl_trim_lsn`/`repl_expire_disconnect_slots` take only `r->lock` — no inversion. Tests: `test_trim_guard_preserves_standby_segments`, `test_trim_without_standbys_proceeds_normally` (test_e2e_checkpoint.c).
|
||||
- **Part A (DORMANT, `g_drive_async_snapshot_offload` DEFAULT 0/OFF)**: `ha_snapshot_send_dispatch` routes `HA_REPL_SEND_SNAPSHOT` inline (flag OFF — byte-for-byte legacy, send on io_worker) or to `slowlane_submit`+`snapshot_lane_fn` (flag ON). Send body factored into `ha_snapshot_do_send` (shared by both paths, holds the XCE_TESTING hook). Per-peer single-flight via `ha_state_t.snapshot_inflight[MAX_HA_CANDIDATES]` CAS + slowlane key `peer_idx+1`; flag cleared on all 4 terminal paths (alloc-fail, -EEXIST, hard-error, lane-fn completion) exactly like reconnect (2.1). Tests: `test_snapshot_offload_runs_on_lane`, `test_snapshot_inline_when_flag_off`, `test_snapshot_offload_dedups_per_peer` (test_ha_liveness.c).
|
||||
- **WHY DORMANT (flag-ON gate, owned by 3.4)**: `snapshot_lane_fn` captures a BORROWED standby endpoint (unlike reconnect, which owns its ep). With the flag ON the send runs on a lane thread while the io_worker may RDHUP-`endpoint_fini` that ep → cross-thread UAF (bug-1197 family). Flag OFF keeps send+teardown serialized on the io_worker. **3.4 is the single gate** that flips BOTH `g_drive_async_snapshot_offload` AND `g_drive_async_peerctrl_dispatch` ON, after establishing the endpoint-lifetime discipline (refcount or lane-serialized teardown) and proving liveness under pool saturation. Caught + fixed pre-commit: the offload flag was initially authored DEFAULT-ON (would have shipped the UAF) → corrected to OFF in both ha.c and the ha.h accessor doc (bug-1210).
|
||||
|
||||
- **3.3 (landed DORMANT — checkbox `[~]` until 3.4 gate)**: requester-less config-push transport offload, flag `g_drive_async_configpush_offload` **DEFAULT 0/OFF** (toggle `ha_set_configpush_offload_for_test`). `ha_config_push_offloadable(c, bytes, blen, sha256, new_tag)` routes inline (flag OFF — verbatim `push_config_transport_only`, caller still owns+frees `bytes`) or to `slowlane_submit`+`configpush_lane_fn` (flag ON — DEEP-COPIES `bytes`+sha into a ctx the lane frees). Single cluster-wide single-flight via scalar `ha_state_t.configpush_inflight` CAS + lane key=1 (config push is a cluster-wide fan-out, not per-peer); flag cleared on all terminal paths (alloc-fail×2, -EEXIST, hard-error, lane completion). Send body factored into `ha_config_push_do_send` (production → `push_config_transport_only`; XCE_TESTING → `ha_configpush_probe` stub since cluster.c/transport isn't linked in test). THREE requester-less drive-thread call sites converted: `raft_cb_become_leader` (promotion, both bytes/NULL branches), the `HA_REPL_PUSH_CONFIG` heartbeat case, and the direct-CTRL_PRIMARY FSM promotion branch (~ha.c:4476). UNCHANGED: `push_config_to_standbys` admin-upgrade barrier (has a waiting requester) and `join.c`. Tests: `test_configpush_offload_runs_on_lane`, `test_configpush_inline_when_flag_off`, `test_configpush_offload_dedups` (test_ha_liveness.c). Same BORROWED-ENDPOINT/bug-1197 dormancy rationale as 3.2 (lane reads `c->ha.standby_endpoints`); **3.4 flips this ON together with snapshot + peerctrl flags**. cpp-review: no CRITICAL/HIGH/MEDIUM; controller suite EXIT=0.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue