wdgzyjx/myxce
1
0
Fork 0
forked from lei.chen/myxce
Code Pull requests Activity

feat(config-upgrade-snapshot): live config-upgrade soak infra + fix bug-1468/bug-1469 (task 8.6)

Browse source 
Build the live 3-node chaotic-multi-kill config-upgrade soak scenario
(run-ha-soak.sh --config-upgrade) required by Requirements 4.1, 4.3, 6.1,
6.4. This needed genuinely new client-side infrastructure: a hand-rolled
capnp send path for OP_upgradeConfig (the auto-generated encoder is a
no-op for its Data fields, worked around the same way
xce_replconfigpush_req_raw already does), plus cmd_hostctrl/cmd_upgradeconfig
CLI commands.

Building and running this live surfaced two real production bugs, both
fixed here:

- bug-1468: an unrelated prior commit inverted OPCode_upgradeConfig's
  primary/standby dispatch guard in cluster.c, making config upgrades
  unreachable from any network client.
- bug-1469: apply_committed_entry's WAL_CONFIG_MARKER case never called
  config_activate_committed(), so the node that originates a config
  upgrade never refreshed its own in-memory state as part of committing
  it. Killing that node before some later, unrelated event happened to
  refresh it left its checkpoint permanently stale -- the first live,
  empirical reproduction of bug-857's originally hypothesized failure
  mode. Fixed by activating before the marker-triggered checkpoint write,
  since checkpoint content is built from live runtime tables that only
  reflect the reload after activation runs.

A separate, narrower checkpoint divergence (bug-1471, not config-related)
remains open and still blocks a clean soak PASS -- task 8.6 stays
in-progress until it's resolved.
This commit is contained in:
Rongsong Shen 2026-07-25 15:48:20 +00:00
parent 2bd44e8e4d
commit 959b0cdacb
13 changed files with 1022 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

8
src/include/client.h
View file

@ -95,6 +95,14 @@ extern int xcecli_jobctrl(xcecli_ctx_t, xce_jobctrl_t*);
extern int xcecli_notify_jobchkpt(xcecli_ctx_t, uint64_t, char*);
extern int xcecli_queuectrl(xcecli_ctx_t, xce_queuectrl_t*);
extern int xcecli_hostctrl(xcecli_ctx_t, xce_hostctrl_t*);
/* xcecli_upgradeconfig: submit a configuration upgrade (sha256 + serialised
* ClusterConfig blob) to the primary via OP_upgradeConfig, transparently
* surviving a primary change mid-call via XCECLI_FAILOVER_CALL (same
* write-op failover contract as xcecli_hostctrl/xcecli_queuectrl). Returns
* the server's reply status (0 on success). */
extern int xcecli_upgradeconfig(xcecli_ctx_t, const uint8_t *sha256,
size_t sha256_len, const uint8_t *bytes,
size_t bytes_len);
extern int xcecli_runtask(xcecli_ctx_t, xce_taskrun_t*);
extern int xcecli_notify_taskstart(xcecli_ctx_t, xce_task_pidinfo_t*);
extern int xcecli_notify_taskfail(xcecli_ctx_t, xce_task_failinfo_t*);

13
src/include/request.h
View file

@ -185,6 +185,19 @@ extern int xce_upgrade_config_req_extract_raw(const uint8_t *pkt_data,
size_t *out_len,
uint8_t **out_sha256);
/*
* xce_upgradeconfig_req_raw: send OP_upgradeConfig carrying raw byte arrays
* (sha256 + the serialised ClusterConfig blob). Bypasses the generated
* encode_UpgradeConfigRequest path (a no-op for the sha256/bytes Data fields
* — same class of limitation as xce_replconfigpush_req_raw) by constructing
* the capnp message directly via capn_new_list8 + capn_setv8 + write_Request.
* Ordinary CLIENT request: seq/from are populated the same way xce_request()
* populates them for every other client op.
*/
extern int xce_upgradeconfig_req_raw(endpoint_t *ep,
const uint8_t *sha256, size_t sha256_len,
const uint8_t *bytes, size_t bytes_len);
/* ── Client-Side Transparent Retry tunables (Req 6.1, 6.2) ──────────────────
*
* xce_retry_config_t: configurable retry parameters with safe production

11
src/include/xceapi.h
View file

@ -28,6 +28,17 @@ extern int xce_update_resvals(xce_ctx_t*, int, xce_resval_t**);
extern int xce_jobctrl(xce_ctx_t*, xce_jobctrl_t*);
extern int xce_queuectrl(xce_ctx_t*, xce_queuectrl_t*);
extern int xce_hostctrl(xce_ctx_t*, xce_hostctrl_t*);
/* xce_upgradeconfig: send OP_upgradeConfig (sha256 + serialised ClusterConfig
* blob) via xce_upgradeconfig_req_raw and wait for the reply. Returns the
* server's reply status (0 on success; a negative errno/E_XCE_* code on
* failure — the server logs the human-readable reason, see
* do_upgrade_config_with_bytes/send_upgrade_error, config_repl.c). Mirrors
* xce_hostctrl/xce_queuectrl's shape; the richer UpgradeConfigReply fields
* (version_tag/acked_count/error text) are not surfaced to the caller — no
* consumer of this API currently needs them beyond success/failure. */
extern int xce_upgradeconfig(xce_ctx_t*, const uint8_t *sha256,
size_t sha256_len, const uint8_t *bytes,
size_t bytes_len);
extern int xce_runtask(xce_ctx_t*, xce_taskrun_t*);
extern int xce_update_jusage(xce_ctx_t*, uint64_t, int, int*, int, int*, int,
xce_rusage_t**);