forked from lei.chen/myxce
checkpoint: bug-851 root cause + raft TLA+ conformance spec family
Investigation & fixes (failover committed-job loss): - root-cause bug-851 on an owned 3-node cluster: chaotic failover causes silent committed-prefix divergence (WAL hdr.term always 0; two leaders per term; no (term,lsn) reconcile). Evidence in repro-851/ + INVESTIGATION. - live-validated fixes 848/849/850 (recovery-on-promotion, advertise real applied_lsn, persist-layer contiguity guard); add waldump WAL inspector. Specs (strict raft.tla conformance — library owns the log): - TLA-gap-analysis.md: impl vs raft/raft.tla, gaps G1-G6. - raft-log-replication, raft-commit-safety, raft-sim-conformance: extend src/lib/raft to own log/commit/verification via callbacks (specs generated, cross-spec review passed). - ha-term-safety: controller-side adapter + config marker (R1-R7); design done. - config-upgrade-snapshot: R8 descoped here (deps entity-bearing-snapshots, bug-857). - roadmap + buglog (851/854/857) + .wolf knowledge updated.
This commit is contained in:
parent
7d70820aac
commit
20e59d0b42
85 changed files with 10525 additions and 1993 deletions
42
src/controller/testsuite/waldump.h
Normal file
42
src/controller/testsuite/waldump.h
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
#pragma once
|
||||
/*
|
||||
* waldump.h — read-only WAL segment inspector for failover/durability triage.
|
||||
*
|
||||
* Decodes the on-disk WAL (`wal_entry_hdr_t` + payload, see wal_types.h),
|
||||
* lists every physical entry, decodes JOB_SUBMIT jobids via the Cap'n Proto
|
||||
* Job schema, and — the decisive signal for the committed-job-loss bug —
|
||||
* flags LSN gaps (a missing prefix, or any non-contiguous hole) and reports
|
||||
* the CRC-chain "recovered tip" a real recovery would stop at.
|
||||
*
|
||||
* Strictly read-only: opens segments O_RDONLY, applies no entries, mutates no
|
||||
* persist_t state. Safe to run against a live node's data dir.
|
||||
*/
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
|
||||
/* Aggregate result of scanning one path (a seg file, a wal/ dir, or a node
|
||||
* data dir). All counts are over the physically-present, CRC-valid entries. */
|
||||
typedef struct {
|
||||
uint64_t first_lsn; /* lowest LSN seen (0 = no entries) */
|
||||
uint64_t last_lsn; /* highest LSN seen */
|
||||
uint64_t recovered_tip; /* highest LSN reachable via an intact CRC chain */
|
||||
uint32_t n_entries; /* total entries scanned */
|
||||
uint32_t n_submits; /* WAL_JOB_SUBMIT entries */
|
||||
uint32_t n_gaps; /* LSN discontinuities (missing prefix counts as 1) */
|
||||
uint32_t n_segments; /* segment files scanned */
|
||||
uint32_t n_dup_submits; /* JOB_SUBMITs with same jobid as the immediately-
|
||||
* preceding entry at an adjacent LSN (bug-845) */
|
||||
uint32_t n_dup_identical; /* of n_dup_submits, those whose payload is
|
||||
* byte-identical to the prior entry */
|
||||
int crc_broken; /* nonzero if a CRC/chain break truncated a segment */
|
||||
} waldump_stats_t;
|
||||
|
||||
/* Dump every WAL entry under `path` to `out` (human-readable) and fill
|
||||
* `*stats` when non-NULL. `path` may be:
|
||||
* - a single seg-*.wal file,
|
||||
* - a directory holding seg-*.wal files (a wal/ dir), or
|
||||
* - a node/data dir whose wal/ subdir (data/wal or wal) holds the segments.
|
||||
* Returns 0 on success, or a negative errno on a hard I/O error (e.g. the
|
||||
* path does not exist / no segments found). Gaps and CRC breaks are reported
|
||||
* through `out`/`stats`, NOT via the return code. */
|
||||
int waldump_path(const char* path, FILE* out, waldump_stats_t* stats);
|
||||
Loading…
Add table
Add a link
Reference in a new issue