feat(config-upgrade-snapshot): live config-upgrade soak infra + fix bug-1468/bug-1469 (task 8.6)
Build the live 3-node chaotic-multi-kill config-upgrade soak scenario (run-ha-soak.sh --config-upgrade) required by Requirements 4.1, 4.3, 6.1, 6.4. This needed genuinely new client-side infrastructure: a hand-rolled capnp send path for OP_upgradeConfig (the auto-generated encoder is a no-op for its Data fields, worked around the same way xce_replconfigpush_req_raw already does), plus cmd_hostctrl/cmd_upgradeconfig CLI commands. Building and running this live surfaced two real production bugs, both fixed here: - bug-1468: an unrelated prior commit inverted OPCode_upgradeConfig's primary/standby dispatch guard in cluster.c, making config upgrades unreachable from any network client. - bug-1469: apply_committed_entry's WAL_CONFIG_MARKER case never called config_activate_committed(), so the node that originates a config upgrade never refreshed its own in-memory state as part of committing it. Killing that node before some later, unrelated event happened to refresh it left its checkpoint permanently stale -- the first live, empirical reproduction of bug-857's originally hypothesized failure mode. Fixed by activating before the marker-triggered checkpoint write, since checkpoint content is built from live runtime tables that only reflect the reload after activation runs. A separate, narrower checkpoint divergence (bug-1471, not config-related) remains open and still blocks a clean soak PASS -- task 8.6 stays in-progress until it's resolved.
This commit is contained in:
parent
2bd44e8e4d
commit
959b0cdacb
13 changed files with 1022 additions and 10 deletions
|
|
@ -669,10 +669,30 @@ static int myreq_handler(vtask_t* task)
|
|||
break;
|
||||
case OPCode_upgradeConfig:
|
||||
#if CONF_ENABLE_UPGRADE_CONFIG
|
||||
/* OPCode_upgradeConfig is sent BY the primary TO standbys to push config.
|
||||
* Standbys must apply it; only a primary sending this to itself is invalid. */
|
||||
if (ha_is_primary(&c->ha)) {
|
||||
/* Primary should not receive its own config push — ignore */
|
||||
/* OPCode_upgradeConfig is sent BY an admin client (or xceadmin's
|
||||
* config-push tool) TO the primary controller, which decodes the
|
||||
* blob, verifies it, and drives it through config_commit_change()
|
||||
* (the (term,lsn) marker consensus path) — see
|
||||
* do_upgrade_config_with_bytes's doc comment, config_repl.c. Only the
|
||||
* primary can commit a WAL entry, so a standby cannot service this
|
||||
* request; ha_dispatch_gate already denies OPCode_upgradeConfig for
|
||||
* every non-primary role (it is absent from the CTRL_STANDBY /
|
||||
* CTRL_INIT / CTRL_ELECTING allow-lists, ha.c), so this check is a
|
||||
* defensive belt-and-suspenders guard, not the primary gate.
|
||||
*
|
||||
* bug-fix (2026-07-25): this check was inverted from
|
||||
* commit fd93997f ("election-iomgr-refactor", 2026-05-31) onward —
|
||||
* it rejected the request WHEN primary, unconditionally reachable
|
||||
* only on primary (per the dispatch-gate note above), making
|
||||
* handle_upgrade_config() permanently unreachable from any network
|
||||
* caller (only unit tests calling it directly ever exercised it).
|
||||
* Independently diagnosed and documented as a blocking risk by the
|
||||
* admin-config-tool spec (design.md "Boundary Commitments" /
|
||||
* "Risks") before config-upgrade-snapshot task 8.6 needed a live
|
||||
* OP_upgradeConfig sender and hit the same wall. Restored to the
|
||||
* pre-regression (correct) polarity. */
|
||||
if (!ha_is_primary(&c->ha)) {
|
||||
/* Standby cannot commit a config change — reject loud. */
|
||||
xce_reply_t ureply;
|
||||
init_reply_by_req(&ureply, &request);
|
||||
ureply.status = -EAGAIN;
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue