fix(scheduler): close post-failover rejoin detection gap (bug-1760)
An idle scheduler (nothing queued to publish) never noticed a dead controller connection: sched_needs_rejoin was only ever set as a side effect of an active client-library call, and an idle round makes none. After any controller failover with nothing already in flight, jobs submitted to the new primary stayed PEND indefinitely. Adds xcecli_is_connection_broken() (libxce) -- a pure, socket-free read of the transport's already-async-maintained dead/alive flag, set for free by the existing RDHUP disconnect handler with zero application calls. sched_check_pending_rejoin() now retries xcecli_join() on either the existing reactive flag or this new passive signal, so a healthy idle scheduler still makes zero extra network calls while a genuinely dead connection self-heals within one ~1s round -- live-verified against a 3-node HA cluster (rejoin fired ~300ms after new-primary election with nothing submitted in between). Also closes out bug-1735 (job memory/CPU usage reporting): confirmed already fixed on this branch since 2026-08-30, just never marked in the bug log. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
This commit is contained in:
parent
48587368ca
commit
88b0ac9937
9 changed files with 441 additions and 28 deletions
|
|
@ -1313,11 +1313,15 @@ sched_scheduler_decision_publish_retry_test = executable(
|
|||
scheduler_decision_publish_retry_test_srcs,
|
||||
include_directories:[incdir],
|
||||
dependencies: [scheduler_deps, unity_dep],
|
||||
c_args: ['-DXCE_MAIN_C_PATH="'
|
||||
+ (meson.current_source_dir() / '..' / 'src' / 'main.c')
|
||||
+ '"'],
|
||||
link_args: ['-Wl,--wrap=xcecli_jobdispatch',
|
||||
'-Wl,--wrap=xcecli_set_pend_reason',
|
||||
'-Wl,--wrap=xcecli_join',
|
||||
'-Wl,--wrap=xcecli_start',
|
||||
'-Wl,--wrap=xcecli_set_reconnect_handler'],
|
||||
'-Wl,--wrap=xcecli_set_reconnect_handler',
|
||||
'-Wl,--wrap=xcecli_is_connection_broken'],
|
||||
install: false)
|
||||
|
||||
# Worst-case decision/allocation backpressure test (component-failover-
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue