lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(controller): offload demotion convergence and catch-up streaming by default (bug-1450, bug-1451)

Browse source 
f88e4ef4 moved ha_demotion_convergence_fn's WAL truncate + self-restart
and ha_catchup_continuation_fn's WAL streaming off the delaymgr thread
(which drives heartbeats and election timeouts cluster-wide) onto the
slow lane, but left both flags OFF: flipping them ON hung pre-existing
unit fixtures that demote a primary or trigger catch-up without ever
starting c->slowlane -- the work was queued in a lane with no workers and
never ran.

- slowlane_is_started(): true while the lane has live workers.
- Both dispatchers run inline when the flag is OFF *or* the lane was never
  started. Production always starts the lane, so it always offloads; the
  lightweight fixtures keep their old inline behaviour, and neither
  one-shot action can be silently dropped.
- g_drive_async_demotion_offload / g_drive_async_catchup_offload now
  default to 1. The offload tests save and restore the previous value
  instead of forcing 0.

New test_catchup_offload_without_started_lane_runs_inline (RED before: the
catch-up body never ran, the stream gate was never reached). Controller
suite 1239/0 with both offloads ON by default.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-09-27 05:46:09 +00:00
parent 88c4310e11
commit 7aac018c00
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
6 changed files with 72 additions and 17 deletions
Download patch file Download diff file Expand all files Collapse all files

19
src/controller/ha_convergence.c
View file

@ -203,7 +203,7 @@ typedef struct {
* delaymgr thread that also drives heartbeats/election timeouts cluster-wide
* never blocks on the truncation I/O or the tlog_exit()+execv() inside
* ha_self_restart) or runs it INLINE on the calling (delaymgr) thread (OFF,
* the PRODUCTION DEFAULT for now — see below). This is the SAME established
* formerly the production default -- see below). This is the SAME established
* slow-lane pattern already used for snapshot send
* (ha_snapshot_send_dispatch/snapshot_lane_fn), config-push
* (ha_config_push_offloadable/configpush_lane_fn), and peer connect
@ -232,8 +232,13 @@ typedef struct {
* the cluster-wide default to ON is the same kind of follow-up audit, out of
* scope for this fix (bug-1450, buglog.json). Compiled in ALL builds; the
* setter is the test seam to flip RED<->GREEN.
*
* bug-1450 follow-up (2026-09-27): now ON by default. The fixture hang above
* is avoided by running inline whenever the slow lane was never started
* (slowlane_is_started), which is exactly those lightweight fixtures;
* production always starts the lane.
*/
static _Atomic int g_drive_async_demotion_offload = 0;
static _Atomic int g_drive_async_demotion_offload = 1;
void ha_set_demotion_offload_for_test(int on)
{
@ -335,10 +340,12 @@ int ha_demotion_convergence_fn(void *arg)
}
cluster_t *c = ctx->c;
if (!ha_demotion_offload_enabled()) {
/* Legacy inline path (flag OFF) — blocks the calling (delaymgr)
* thread. Retained solely so the RED test can show the pre-change
* stall; production runs with the flag ON. */
if (!ha_demotion_offload_enabled() || !slowlane_is_started(&c->slowlane)) {
/* Inline path -- blocks the calling (delaymgr) thread. Taken with
* the flag OFF (kept so the RED test can show the pre-change stall)
* or when the slow lane was never started (lightweight fixtures):
* work submitted there would never run, and this one-shot safety
* action must never be dropped (bug-1450). */
int rc = ha_demotion_convergence_do_run(c);
free(ctx);
return rc;