--- id: "bug-609" status: "fixed" status_updated: "2026-09-24" status_note: "" component: "lib" severity: "unknown" timestamp: "2026-06-16T10:52:01.800Z" last_seen: "2026-09-19T09:00:00Z" occurrences: 2 files: ["src/lib/client_failover.c"] tags: ["md", "libxce", "failover", "EINVAL", "XCE_CTX_ERR", "reconnect", "E2E"] related: ["bug-608"] legacy: true --- # bug-609: After primary killed, libxce client xce_joblist/jobsubmit returns 'Invalid argument' (-EINVAL) on… ## Description After primary killed, libxce client xce_joblist/jobsubmit returns 'Invalid argument' (-EINVAL) on every fresh call — transparent failover did not engage ## Environment _Not recorded (migrated from buglog.json)._ ## Reproduction _Not recorded (migrated from buglog.json)._ ## Root cause handle_disconnect leaves xce_ctx_t.state=XCE_CTX_ERR after the primary's connection drops; the per-op xceapi guard (xceapi.c:14) returns -EINVAL when state!=XCE_CTX_RUN, BEFORE the op reaches a primary. xcecli_retry_decide classified -EINVAL as TERMINAL (non-retriable), so the failover loop bailed without rotating/recovering. The ctx->state ERR->RUN recovery only ran AFTER a rotate, which a terminal classification never triggered. Surfaced by the live 3-node E2E (task 6.2); the unit tests fed status codes directly to decide() and bypassed the real guard, so they missed it. ## Fix In xcecli_retry_decide, before classifying *rc, detect ac->ctx.state==XCE_CTX_ERR (known-dead connection) and treat it as a prompt transport-style failover cause (XCE_FO_RETRY_NOW, no backoff) regardless of *rc, so the loop rotates+recovers. On budget exhaustion surface -E_XCE_NOT_READY (cluster-not-ready), never -EINVAL. Regression: test_decide_known_dead_ctx_retries_not_terminal. ## Verification _Not recorded (migrated from buglog.json)._ ## References - Related: bug-608