wdgzyjx/myxce
1
0
Fork 0
forked from lei.chen/myxce
Code Pull requests Activity

feat(ha-term-safety): retire term-0 repl_broadcast live send; carry leader term on replication (task 3.4)

Browse source 
Closes bug-990: the live replicator repl_broadcast wire-encoded term=0
(entry.term was never set), so standbys persisted hdr.term=0 and took the
legacy direct-apply path instead of the library (term,lsn) intake. Now the
live broadcast and catch-up streamer carry the leader's real term + AE prev
tuple; ha_apply_wal_catchup_entry routes every term-carrying entry through
raft_on_append_entries_request -> on_persist_entry (leader term stamped),
retiring the term-0 direct-apply. Standbys now persist the leader's real term
and reconcile by (term,lsn); the live commit chain is end-to-end.

848/849/850 + persist-before-ACK + commit-barrier + bug-960 preserved.
Proven at integration level; the definitive live STRICT_TERM failover run is
env-blocked here (bug-924 overlayfs) and recommended on a real fs.
This commit is contained in:
Rongsong Shen 2026-06-22 15:45:14 +00:00
parent 87242aa8e9
commit efe621c124
11 changed files with 307 additions and 45 deletions
Download patch file Download diff file Expand all files Collapse all files

82
src/controller/ha.c
View file

@ -2073,6 +2073,54 @@ static void ha_send_wal_ack_reply(ha_state_t *ha, struct endpoint_t *ep,
xce_reply(ep, c->myhost, &reply);
}
/* ha_apply_term_entry_via_library: route ONE term-carrying replicated entry
* (a live broadcast or a catch-up sub-entry) through the raft library
* (raft_on_append_entries_request via ha_consume_append_entries) so the LIBRARY
* does logOk + (term,lsn) reconcile and on_persist_entry stamps the LEADER's
* term — never the local term-0 direct-apply path (ha-term-safety task 3.4).
*
* `e` is the single entry (its term/prev_log_index/prev_log_term carry the AE
* metadata, set by the leader's broadcast/stream path). On accept (the entry's
* term is now recorded at its LSN) the caller ACKs and advances applied. Returns
* 1 if the entry was accepted (recorded), 0 otherwise (reject / not yet
* applicable — e.g. a logOk-failing forward gap that catch-up will backfill). */
static int ha_apply_term_entry_via_library(ha_state_t *ha, persist_t *p,
cluster_t *c, struct endpoint_t *ep,
wal_catchup_entry_t *e)
{
/* Wrap the single entry in an AE container whose TOP-LEVEL fields carry the
* prev tuple + term that ha_consume_append_entries reads, with entries[0]
* pointing at the entry itself (the same shape repl_send_append_entries /
* repl_broadcast produce on the wire). */
wal_catchup_entry_t *sub_arr[1] = { e };
wal_catchup_entry_t ae;
memset(&ae, 0, sizeof(ae));
ae.n_entries = 1;
ae.entries = sub_arr;
ae.term = e->term;
ae.prev_log_index = e->prev_log_index;
ae.prev_log_term = e->prev_log_term;
ae.leader_commit = e->leader_commit;
uint64_t before_term = persist_entry_term(p, e->lsn);
int rc = ha_consume_append_entries(c, &ae, /*leader_idx*/ -1);
if (rc != 0)
return 0;
uint64_t after_term = persist_entry_term(p, e->lsn);
if (after_term == e->term) {
/* Accepted (appended / conflict-truncate-then-persist) or already-have
* (idempotent re-stream): ACK + advance applied on that signal. */
(void)before_term;
ha_advance_applied_lsn(ha, e->lsn + 1);
ha_send_wal_ack_reply(ha, ep, c, e->lsn);
return 1;
}
/* logOk reject (e.g. forward gap before the prefix arrives): no ACK; the
* ACK-driven catch-up continuation re-streams from current_lsn in order. */
return 0;
}
int ha_apply_wal_catchup_entry(ha_state_t *ha, persist_t *p,
wal_catchup_entry_t *entry,
struct endpoint_t *ep, cluster_t *c)
@ -2094,23 +2142,7 @@ int ha_apply_wal_catchup_entry(ha_state_t *ha, persist_t *p,
* computes the target index from the prev tuple, then ACK on accept. */
if (c && c->ha.raft_c == c && entry->n_entries == 1 &&
entry->entries && entry->entries[0] && entry->entries[0]->term != 0) {
wal_catchup_entry_t *e = entry->entries[0];
uint64_t before_term = persist_entry_term(p, e->lsn);
int rc = ha_consume_append_entries(c, entry, /*leader_idx*/-1);
if (rc == 0) {
/* Accept (append or conflict-truncate-then-persist) leaves the entry
* term recorded at its LSN — ACK + advance applied on that signal.
* A reject (logOk fail) leaves the prior term unchanged → no ACK. */
uint64_t after_term = persist_entry_term(p, e->lsn);
if (after_term == e->term && after_term != before_term) {
ha_advance_applied_lsn(ha, e->lsn + 1);
ha_send_wal_ack_reply(ha, ep, c, e->lsn);
} else if (after_term == e->term) {
/* already-have (idempotent re-stream): ACK without a rewrite. */
ha_advance_applied_lsn(ha, e->lsn + 1);
ha_send_wal_ack_reply(ha, ep, c, e->lsn);
}
}
ha_apply_term_entry_via_library(ha, p, c, ep, entry->entries[0]);
return 0;
}
@ -2131,6 +2163,22 @@ int ha_apply_wal_catchup_entry(ha_state_t *ha, persist_t *p,
continue;
}
/* ha-term-safety task 3.4: a term-carrying catch-up sub-entry routes
* through the LIBRARY (logOk + (term,lsn) reconcile + on_persist_entry
* stamps the LEADER's term). Each sub-entry carries its own prev tuple
* (wal_entry_to_wire encodes prevLog{Index,Term} per element). This
* retires the term-0 direct-apply path for live/catch-up replication.
* Entries stream in ascending LSN order, so the library's contiguity
* (prev tuple) is satisfied in sequence; a not-yet-applicable entry
* (logOk reject) is left for the catch-up continuation. */
if (c && c->ha.raft_c == c && sub->term != 0) {
if (ha_apply_term_entry_via_library(ha, p, c, ep, sub)) {
last_good_lsn = sub->lsn;
had_good = 1;
}
continue;
}
int rc = ha_apply_single_entry(p, sub);
if (rc != 0) {
/* -EAGAIN = contiguity gap (bug-850): entry ahead of our tail,