lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(config-upgrade-snapshot): remove restart/replay fallback, marker-trigger materializes alone

Browse source 
do_upgrade_config_with_bytes drops step 9 (wait-for-checkpoint,
restart_requested=1, cbuf_put stop signal) -- materialization now
happens entirely via task 5's apply_committed_entry WAL_CONFIG_MARKER
case on every member's own apply, no restart required. Confirmed
do_upgrade_config_with_bytes was the sole production setter of
restart_requested; cluster.c's read of it is now permanently
false-for-this-trigger, which is the intended behavior.

Adds xce_upgrade_marker_committed_hook (#ifdef XCE_TESTING, mirrors
xce_snapshot_chunk_hook's nullable-function-pointer shape), firing
right after config_commit_change succeeds -- a deterministic seam for
later leadership-change fault injection (task 8.7) at exactly the
marker-commit-to-every-member-installed window. NULL by default, true
no-op in production.

Task 7 of 19.

Requirements: 1.2, 1.3, 1.4, 2.2, 5.4, 6.3
This commit is contained in:
Rongsong Shen 2026-07-25 05:37:27 +00:00
parent 89b85e1680
commit 41af05402d
3 changed files with 135 additions and 46 deletions
Download patch file Download diff file Expand all files Collapse all files

63
src/controller/config_repl.c
View file

@ -1570,6 +1570,21 @@ int push_config_to_standbys(cluster_t *c, const uint8_t *bytes, size_t blen,
/* ── handle_upgrade_config ──────────────────────── */
/* Test-only seam (config-upgrade-snapshot task 7): fires in
* do_upgrade_config_with_bytes right after config_commit_change() succeeds
* -- the configuration-change marker has just reached the committed
* prefix, but materialization on every member (apply_committed_entry's
* WAL_CONFIG_MARKER case, persist.c) has not necessarily completed
* anywhere yet. This is exactly the window a later regression test needs a
* deterministic way to land a forced leadership change in (Requirements
* 4.3, 5.4, 6.3). NULL in production -- no behaviour change. Mirrors
* xce_snapshot_chunk_hook's shape. Declared extern in the test file (same
* pattern as xce_snapshot_chunk_hook). */
#ifdef XCE_TESTING
int (*xce_upgrade_marker_committed_hook)(cluster_t *c,
uint64_t version_tag) = NULL;
#endif
/*
* do_upgrade_config_with_bytes: core upgrade logic called after the raw
* capnp bytes have been extracted from the wire via
@ -1586,7 +1601,12 @@ int push_config_to_standbys(cluster_t *c, const uint8_t *bytes, size_t blen,
* 7. Push to standbys and wait for the majority-ACK barrier; on failure
* reply with send_upgrade_error and return (no success ACK, no restart)
* 8. ACK admin tool (success ⇔ majority-acked config)
* 9. Wait for in-flight checkpoint, then signal restart
*
* The function's job ends at step 8. Materialization of the new
* configuration on every member -- including this one -- happens
* asynchronously and independently via each member's own apply of the
* committed WAL_CONFIG_MARKER (apply_committed_entry, persist.c,
* config-upgrade-snapshot task 5). No restart is requested or required.
*/
static int do_upgrade_config_with_bytes(cluster_t *c, endpoint_t *ep,
xce_request_t *req,
@ -1720,6 +1740,11 @@ static int do_upgrade_config_with_bytes(cluster_t *c, endpoint_t *ep,
return rc;
}
#ifdef XCE_TESTING
if (xce_upgrade_marker_committed_hook)
(void)xce_upgrade_marker_committed_hook(c, c->myconf->config_version_tag);
#endif
/* 8. ACK the admin tool (success ⇔ majority-acked config) */
upgrade_config_reply_t urep;
memset(&urep, 0, sizeof(urep));
@ -1734,41 +1759,13 @@ static int do_upgrade_config_with_bytes(cluster_t *c, endpoint_t *ep,
reply.data.upgradeconfigreplymsg = &urep;
xce_reply(ep, c->myhost, &reply);
/* NOTICE (bug-857, spec `.kiro/specs/config-upgrade-snapshot/`) — config
* upgrade MUST snapshot current state, not re-derive it per node.
* Today the upgrade commits the config-change marker, then each member
* RESTARTS and re-materializes its post-upgrade state by replaying its own
* pre-upgrade WAL under the NEW config (steps 9 + restart recovery). Replay
* is config-dependent (`lost_and_found` re-homes entities whose config
* entity vanished), so members can DIVERGE in materialized state after an
* upgrade. The fix: at the config-change marker's (term,lsn) boundary the
* controller must materialize a full ENTITY-BEARING state snapshot, embed
* the new config in it, and install that SAME snapshot on every member via
* the snapshot_transfer_s transport (install-before-activate, hash-verified)
* so post-upgrade state is identical by construction and no member
* re-derives it. HARD dependency: `entity-bearing-snapshots` (snapshots are
* a 16-byte LSN sentinel today; the entity payload does not exist yet).
* Until that lands, the per-node-replay path below is the interim behavior. */
/* 9. Wait for in-flight checkpoint (up to 30 s), then signal restart */
struct timespec ckpt_start, ckpt_now;
clock_gettime(CLOCK_MONOTONIC, &ckpt_start);
while (atomic_load(&c->persist.checkpoint_in_progress)) {
usleep(10000);
clock_gettime(CLOCK_MONOTONIC, &ckpt_now);
if (ckpt_now.tv_sec - ckpt_start.tv_sec > 30) {
ERROR("Checkpoint did not complete within 30 s"
" -- restarting anyway");
break;
}
}
/* Materialization now happens via apply_committed_entry's
* WAL_CONFIG_MARKER case (persist.c, config-upgrade-snapshot task 5) on
* every member's own apply of this marker -- no restart needed. */
INFO("Config upgraded to tag=%016" PRIx64 " gen=%" PRIu64
" (acked=%d standbys) -- requesting restart",
" (acked=%d standbys)",
c->myconf->config_version_tag, new_gen, acked);
atomic_store(&c->restart_requested, 1);
int stop_val = 1;
cbuf_put(&c->wbuf, &stop_val);
return 0;
}