forked from lei.chen/myxce
fix(ha-commit-engine): raise commit-barrier default 50->500ms + add --sync-timeout-ms
Live symptom: jobsub intermittently failed with 'Connection timed out' even though the cluster had quorum. Root cause: the synchronous commit barrier waits only DEFAULT_SYNC_TIMEOUT_MS=50ms for durable majority replication, and that was not configurable. 50ms is too tight for durable sync majority replication — especially since standbys now fdatasync each replicated entry before ACK — so a multi-entry jobsub's later WAL entry intermittently misses the window: commit: WAIT deadline expired without commit (lsn=3 committed_lsn=2 timeout_ms=50) -> persist hook returns -ETIMEDOUT -> handle_jobsubmit rejects -> client ETIMEDOUT. Fix: - Raise DEFAULT_SYNC_TIMEOUT_MS 50 -> 500 (config.h) — realistic for durable sync majority replication, still bounded for failure detection. - Add a --sync-timeout-ms controller flag threaded arguments -> myconf -> persist pcfg (override applied before persist_init); 0 = use default, positive overrides — so operators can tune per environment. TDD: literal-pinned persist_config_defaults().sync_timeout_ms == 500. All timeout-sensitive tests already pass an explicit timeout_ms (none rely on the default), so none are slowed; full controller suite green under ASAN. No commit-engine/barrier/replication/wire change beyond the default value + plumbing. bug-561.
This commit is contained in:
parent
c4fd3e1c8f
commit
ca1d9ae42d
5 changed files with 24 additions and 1 deletions
|
|
@ -1139,6 +1139,10 @@ int cluster_init_phase2(cluster_t* c)
|
|||
: "controller";
|
||||
persist_config_t pcfg = persist_config_defaults();
|
||||
|
||||
/* --sync-timeout-ms override (0 = keep the persist default). */
|
||||
if (c->myconf->sync_timeout_ms > 0)
|
||||
pcfg.sync_timeout_ms = c->myconf->sync_timeout_ms;
|
||||
|
||||
persist_set_workdir(&c->persist, workdir, name);
|
||||
rc = persist_init(&c->persist, &pcfg, c);
|
||||
if (rc != 0) {
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue