forked from lei.chen/myxce
Rongsong Shen
b9ad01878c
Root-caused via raft-ha-core-safety 5.10a investigation: schedule_checkpoint() (persist.c) fires correctly once wal_record_count crosses its threshold, but its delay_task()-scheduled checkpoint_task_fn never executed in a live multi-node deployment -- confirmed live via temporary diagnostic logging (wal_record_count=100 max=100 logged, but the task itself never ran). Root cause: delay_task_t carried TWO incomparable time representations -- `nextrun` (wall-clock time_t seconds, used by delay_task()) and `nextrun_ts` (CLOCK_MONOTONIC timespec, used by delay_task_ms()). task_cmp()'s mixed-type fallback compared them directly: an ms-task's nextrun is always 0, a seconds-task's nextrun is a real epoch value (e.g. 1785600123) -- so 0 always sorted "less than" any seconds-task, regardless of actual due time. ordq (src/lib/ordq.c) keeps its queue as an ascending sorted linked list by this comparator, and delay_worker() only ever inspects the list HEAD (ordq_peek with no matchfn returns q->head immediately, no scanning). Since cluster_t.delaymgr is shared with ha.c's ~10 recurring delay_task_ms() call sites (heartbeats, election timers, RDHUP polling), the head was structurally almost-always occupied by an ms-task -- any delay_task() (seconds) caller sharing that delaymgr, including schedule_checkpoint's checkpoint_task_fn, could be starved indefinitely. Fix: eliminate the dual representation entirely. Every task -- whether scheduled via delay_task() (seconds) or delay_task_ms() (milliseconds) -- now stores one flat uint64_t CLOCK_MONOTONIC millisecond deadline (nextrun_ms). delay_task() becomes a thin seconds->ms wrapper around delay_task_ms(). task_cmp() and task_due() collapse to a single integer comparison; delay_worker()'s reschedule is a flat integer add (no more tv_sec/tv_nsec carry arithmetic, no more is_ms_task branching). Net -42 lines. Live-verified: after this fix, a forced checkpoint round produces a real installed snapshot file (previously: schedule_checkpoint fired but the task silently never ran). controller_unit_test: full 58-group sweep, 0 Failures, 0 ASAN reports (before and after this change) -- no regression, including every delaymgr-touching group (test_demotion_convergence, test_ha_start_election_async, test_ha_core_safety, test_ha_startup, etc).
46 lines
1.5 KiB
C
46 lines
1.5 KiB
C
#if !defined(_DELAY_H_)
|
|
|
|
#define _DELAY_H_ 1
|
|
|
|
#include <pthread.h>
|
|
#include <stdint.h>
|
|
#include <sys/types.h>
|
|
#include <time.h>
|
|
|
|
#include "ordq.h"
|
|
|
|
typedef int (*delay_task_entry_fn_t)(void*);
|
|
typedef int (*task_run_fn_t)(delay_task_entry_fn_t, void*);
|
|
|
|
/* Single normalized time domain: every task (whether scheduled via
|
|
* delay_task() seconds or delay_task_ms() milliseconds) is stored as one
|
|
* flat CLOCK_MONOTONIC millisecond deadline. There is no separate
|
|
* wall-clock/ms task "flavor" to distinguish -- delay_task() is a thin
|
|
* seconds->ms wrapper around delay_task_ms() (see delay.c), so every entry
|
|
* in the queue is always comparable on the same scale with a single
|
|
* integer comparison (no tv_sec/tv_nsec pair, no carry arithmetic). */
|
|
typedef struct {
|
|
uint64_t nextrun_ms; /* CLOCK_MONOTONIC deadline, in ms */
|
|
int run_period_ms;
|
|
int runcount;
|
|
delay_task_entry_fn_t entry;
|
|
void* args;
|
|
} delay_task_t;
|
|
|
|
typedef struct {
|
|
pthread_t worker;
|
|
int running;
|
|
int precision_ms;
|
|
task_run_fn_t runner_fn;
|
|
timer_t timerid;
|
|
ordq_t tasks;
|
|
} delaymgr_t;
|
|
|
|
extern int delaymgr_init(delaymgr_t*, int, task_run_fn_t);
|
|
extern void delaymgr_stop(delaymgr_t*);
|
|
extern void delaymgr_fini(delaymgr_t*);
|
|
extern int delaymgr_start(delaymgr_t*);
|
|
extern int delay_task(delaymgr_t*, int, int, delay_task_entry_fn_t, void*);
|
|
extern int delay_task_ms(delaymgr_t*, int, int, delay_task_entry_fn_t, void*);
|
|
|
|
#endif
|