lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(scheduler): close post-failover rejoin detection gap (bug-1760)

Browse source 
An idle scheduler (nothing queued to publish) never noticed a dead
controller connection: sched_needs_rejoin was only ever set as a side
effect of an active client-library call, and an idle round makes none.
After any controller failover with nothing already in flight, jobs
submitted to the new primary stayed PEND indefinitely.

Adds xcecli_is_connection_broken() (libxce) -- a pure, socket-free read
of the transport's already-async-maintained dead/alive flag, set for
free by the existing RDHUP disconnect handler with zero application
calls. sched_check_pending_rejoin() now retries xcecli_join() on either
the existing reactive flag or this new passive signal, so a healthy
idle scheduler still makes zero extra network calls while a genuinely
dead connection self-heals within one ~1s round -- live-verified
against a 3-node HA cluster (rejoin fired ~300ms after new-primary
election with nothing submitted in between).

Also closes out bug-1735 (job memory/CPU usage reporting): confirmed
already fixed on this branch since 2026-08-30, just never marked in
the bug log.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
This commit is contained in:
Rongsong Shen 2026-09-04 04:37:11 +00:00
parent 48587368ca
commit 88b0ac9937
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
9 changed files with 441 additions and 28 deletions
Download patch file Download diff file Expand all files Collapse all files

14
src/include/client.h
View file

@ -51,6 +51,20 @@ extern int xcecli_wakeup(xcecli_ctx_t, int);
extern int xcecli_hello(xcecli_ctx_t, char*);
extern int xcecli_nop(xcecli_ctx_t);
extern int xcecli_heartbeat(xcecli_ctx_t, int);
/*
* xcecli_is_connection_broken (bug-1760) - PURE, socket-free, side-effect-free
* query: has the transport been detected dead since the last successful
* (re)connect? Mirrors the private xcecli_ff_needs_reconnect() predicate
* (client_internal.h) exactly, exposed publicly so a consumer's own idle
* loop can passively check for a dead connection -- with ZERO network I/O
* and zero blocking risk -- instead of periodically probing a possibly-
* healthy one. The underlying signal (ctx->state == XCE_CTX_ERR) is set
* asynchronously by the transport's own RDHUP/EPOLLERR handler
* (handle_disconnect(), xcectx.c) the instant the connection drops, with no
* application call involved -- this function only ever reads that
* already-computed flag. Returns false for a NULL ctx.
*/
extern bool xcecli_is_connection_broken(xcecli_ctx_t);
extern int xcecli_shutdown(xcecli_ctx_t);
extern int xcecli_reslist(xcecli_ctx_t, xce_resource_t***);
extern int xcecli_join(xcecli_ctx_t, int, hostattrs_t*);