feat(election-iomgr-refactor): COMMITTING path in ha_elect_evaluate_and_signal + committing/election timeout watchdogs — tasks 4.1, 4.3, 4.4

This commit is contained in:
Rongsong Shen 2026-05-30 11:37:12 +00:00
parent 6de4b3ff0e
commit f4a6726f1b
6 changed files with 646 additions and 65 deletions

View file

@ -58,14 +58,14 @@
- _Depends: 1.1, 1.2, 2.2, 2.3_
- [ ] 3. Core: FSM gate extensions and role-based vote denial
- [ ] 3.1 (P) Extend ha_dispatch_gate for CTRL_PRIMARY_COMMITTING state
- [x] 3.1 (P) Extend ha_dispatch_gate for CTRL_PRIMARY_COMMITTING state
- Add `case CTRL_PRIMARY_COMMITTING:` to `ha_dispatch_gate` in ha.c: allow `OPCode_heartbeat`, `OPCode_replConfig`, `OPCode_replConfigPush`, `OPCode_upgradeConfigAck`, `OPCode_hello`, `OPCode_walAck`, `OPCode_replEntry`, `OPCode_replAck`, `OPCode_replCatchUpDone`; deny all others with `-XCE_ERR_NOT_READY`
- Observable: unit test sends `OPCode_jobsubmit` in COMMITTING state → returns `XCE_ERR_NOT_READY`; `OPCode_replCatchUpDone` → returns 0
- _Requirements: 5.1, 5.2, 8.1_
- _Boundary: ha.c (dispatch gate)_
- _Depends: 2.1_
- [ ] 3.2 (P) Extend handle_request_vote with role-based deny gate
- [x] 3.2 (P) Extend handle_request_vote with role-based deny gate
- Before existing term/voted_for/freshness checks in `handle_request_vote`: load `ha->role`; return 0 (deny) if `CTRL_PRIMARY` or `CTRL_PRIMARY_COMMITTING`; return 0 (deny) if `CTRL_STANDBY && ha_heartbeat_is_fresh(ha)`
- Implement `ha_heartbeat_is_fresh(ha)`: compare `clock_gettime(CLOCK_MONOTONIC)` against `ha->last_heartbeat`; return 1 if elapsed < `heartbeat_timeout_ms`
- Observable: unit test with mocked roles: PRIMARY→deny, COMMITTING→deny, STANDBY+fresh→deny, STANDBY+stale→evaluate, ELECTING→evaluate
@ -74,7 +74,7 @@
- _Depends: 2.1_
- [ ] 4. Core: committing phase — replication push and availability gate
- [ ] 4.1 Extend ha_elect_evaluate_and_signal with COMMITTING path
- [x] 4.1 Extend ha_elect_evaluate_and_signal with COMMITTING path
- Extend `ha_elect_evaluate_and_signal`: after determining role=PRIMARY, check `!ha->phase2_done`; if first boot call `cluster_init_phase2(c)` then set `ha->phase2_done=1`; set `ha->role = CTRL_PRIMARY_COMMITTING`; reset `repl_ack_count=0`; schedule committing timeout watchdog via `delay_task_ms`
- On STANDBY result: if first boot call `cluster_init_phase2(c)` + set `phase2_done=1`; set role=CTRL_STANDBY; election timeout callback sets `cancelled=1`
- Observable: unit test verifies `cluster_init_phase2` called exactly once on first election; second election (RDHUP) skips it; role enters COMMITTING not PRIMARY after winning
@ -90,7 +90,7 @@
- _Boundary: ha.c + cluster.c (replication push)_
- _Depends: 4.1_
- [ ] 4.3 Implement ha_repl_ack_received and majority-ack → PRIMARY transition
- [x] 4.3 Implement ha_repl_ack_received and majority-ack → PRIMARY transition
- Implement `ha_repl_ack_received(ha, c)`: atomically increment `repl_ack_count`; compute quorum `= (ha->n_peers + 2) / 2 - 1`; if `repl_ack_count >= quorum`: set committing timeout `cancelled=1`, call `ha_promote_to_primary(c)` (sets role=CTRL_PRIMARY, enables hooks)
- Wire `ha_repl_ack_received` into `OPCode_replCatchUpDone` handler in cluster.c
- Observable: unit test: 2 of 3 acks arrive → PRIMARY; 1 of 3 acks → stays COMMITTING
@ -98,7 +98,7 @@
- _Boundary: ha.c + cluster.c (ack counting)_
- _Depends: 4.1_
- [ ] 4.4 Implement committing timeout watchdog (degraded-mode fallback)
- [x] 4.4 Implement committing timeout watchdog (degraded-mode fallback)
- Define `committing_timeout_ctx_t`: `_Atomic int cancelled`, `ha_state_t *ha`, `cluster_t *c`
- Implement `committing_timeout_fn`: if `!cancelled` and `ha->role == CTRL_PRIMARY_COMMITTING`, log WARN "committing timeout — opening in degraded mode", call `ha_promote_to_primary(c)`
- Schedule via `delay_task_ms(delaymgr, committing_timeout_ms, 1, committing_timeout_fn, ctx)` in the COMMITTING entry path