From 50b976aedca31f02e9c3c97aaec02bcfc816519f Mon Sep 17 00:00:00 2001 From: Rongsong Shen Date: Wed, 24 Jun 2026 09:45:13 +0000 Subject: [PATCH] =?UTF-8?q?docs(controller):=20NOTICE=20=E2=80=94=20config?= =?UTF-8?q?=20upgrade=20must=20snapshot=20state,=20not=20re-derive?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add an in-situ NOTICE at the config-upgrade restart site (do_upgrade_config_with_bytes, step 9). Today an upgrade commits the config-change marker, then each member restarts and re-materializes its post-upgrade state by replaying its own pre-upgrade WAL under the NEW config; replay is config-dependent (lost_and_found re-homes orphaned entities), so members can diverge in materialized state (bug-857). The notice records the intended fix — materialize a full entity-bearing snapshot at the marker's (term,lsn) boundary, embed the new config, and install the SAME snapshot on every member via snapshot_transfer_s — and its hard dependency on entity-bearing-snapshots. Comment-only; anchors to .kiro/specs/config-upgrade-snapshot/. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01MSxwb2Xt5F84ATgiAZnH8V --- src/controller/config_repl.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/src/controller/config_repl.c b/src/controller/config_repl.c index 9338260..88d18e3 100644 --- a/src/controller/config_repl.c +++ b/src/controller/config_repl.c @@ -1638,6 +1638,22 @@ static int do_upgrade_config_with_bytes(cluster_t *c, endpoint_t *ep, reply.data.upgradeconfigreplymsg = &urep; xce_reply(ep, c->myhost, &reply); + /* NOTICE (bug-857, spec `.kiro/specs/config-upgrade-snapshot/`) — config + * upgrade MUST snapshot current state, not re-derive it per node. + * Today the upgrade commits the config-change marker, then each member + * RESTARTS and re-materializes its post-upgrade state by replaying its own + * pre-upgrade WAL under the NEW config (steps 9 + restart recovery). Replay + * is config-dependent (`lost_and_found` re-homes entities whose config + * entity vanished), so members can DIVERGE in materialized state after an + * upgrade. The fix: at the config-change marker's (term,lsn) boundary the + * controller must materialize a full ENTITY-BEARING state snapshot, embed + * the new config in it, and install that SAME snapshot on every member via + * the snapshot_transfer_s transport (install-before-activate, hash-verified) + * so post-upgrade state is identical by construction and no member + * re-derives it. HARD dependency: `entity-bearing-snapshots` (snapshots are + * a 16-byte LSN sentinel today; the entity payload does not exist yet). + * Until that lands, the per-node-replay path below is the interim behavior. */ + /* 9. Wait for in-flight checkpoint (up to 30 s), then signal restart */ struct timespec ckpt_start, ckpt_now; clock_gettime(CLOCK_MONOTONIC, &ckpt_start);