forked from lei.chen/myxce
Rongsong Shen
11244dd654
Root cause (confirmed via targeted DIAG1419 logging on a live chaotic soak): a client's commit-timeout retry during a genuine quorum-loss window appends a second WAL_JOB_SUBMIT for the same jobid; the timed-out entry is never retracted, so if it later achieves majority commit anyway both entries end up committed. This is a legitimate, harmless duplicate — restore_job()'s own -EEXIST correctly no-ops the second one — but replay_wal() tallied every raw WAL_JOB_SUBMIT occurrence instead of genuine creations, so the duplicate made new_job_submits disagree with jobmgr's real growth, firing a false APPLY_DIVERGENT and deadlocking promotion. Reverts the layer-3 claim/release machinery (job_wal_commit_claim/release, claim_committed_job_submits, and their wiring): three attempts at preventing the duplicate at submit/dispatch/commit time all failed to close the gap, because the duplicate isn't a bug to prevent — a WAL entry can't be retracted once appended, so any submit-time guard is structurally the wrong layer. Real fix: apply_committed_entry()'s WAL_JOB_SUBMIT case now returns 1 for a genuine creation vs 0 for an idempotent no-op (restore_job() -EEXIST), and replay_wal() only tallies new_job_submits_out on rc==1. Updates the affected apply-seam/reapply-idempotency tests for the new return contract, and reverses two apply-correctness tests whose premise (duplicate-jobid replay must diverge) is now understood to be the actual bug rather than the expected behavior — they now force divergence directly via the same production fencing seam instead of relying on that trigger. Verified: two consecutive 10-round chaotic soaks, zero divergence, zero job loss, no election stall (previously reproduced within ~5 rounds). |
||
|---|---|---|
| .. | ||
| autoscaler | ||
| controller | ||
| docs | ||
| frontend | ||
| include | ||
| lib | ||
| nixSupport | ||
| scheduler | ||
| schema | ||
| tools | ||
| worker | ||
| flake.lock | ||
| flake.nix | ||
| meson.build | ||
| meson.options | ||