chore(wolf): checkpoint buglog.json (1054 bugs)

.wolf/buglog.json

@@ -19457,6 +19457,35 @@
       ],
       "occurrences": 1,
       "last_seen": "2026-08-26T09:32:18Z"
+    },
+    {
+      "id": "bug-1713",
+      "timestamp": "2026-08-26T17:15:00Z",
+      "error_message": "User-reported, live 3-node cluster, right after bug-1700/bug-1701's redirect fix landed and was live-tested: `xceadmin scheduler record status --controller <standby>` now correctly redirects ('redirected: localhost:9903 -> localhost:9901'), but the SAME command against the SAME redirect target intermittently fails with `Error: controller returned status -10013` (sometimes succeeds with real recording status, sometimes fails on an immediate repeat) instead of always succeeding.",
+      "file": "src/xceadmin/src/xceadmin/scheduler.py (_send_admin_request), possibly xceadmin/discovery.py (follow_redirects)",
+      "root_cause": "FULLY ROOT-CAUSED. -10013 is E_XCE_NOT_READY (errnox.h: XCE_ERRNO_BASE+12, 'Controller in CTRL_INIT or CTRL_ELECTING state'). The redirect from bug-1701's fix correctly points at whichever node was primary as of its last ctrlhello, but that node can be transiently mid-(re-)election by the time the redirected request actually arrives -- expected/normal on a freshly-booted cluster (election churn during the first several seconds), or any time a real failover is in flight. ha_dispatch_gate() returns -XCE_ERR_NOT_READY for exactly this transient window, with NO redirect populated (by design -- a CTRL_INIT/CTRL_ELECTING node genuinely has no settled primary to point to yet). The sibling code E_XCE_NOMASTER (errnox.h:22) carries an explicit design-intent comment: 'Standby has no known primary -- client should retry' -- but `_send_admin_request()` (scheduler.py:119-161) treats EVERY nonzero reply.status, including both E_XCE_NOT_READY and E_XCE_NOMASTER, as an immediate hard SchedAdminError with no retry at all. This is a documented client-side gap, not a server bug: the server-side behavior (refuse with a specific, distinguishable transient-state code) is correct and intentional; the client simply never implements the retry the errno's own comment already promises.",
+      "fix": "PENDING. Add bounded retry-with-backoff in xceadmin's scheduler-admin client path (_send_admin_request, scheduler.py, and verify whether push.py's config-push client has/needs the identical treatment for the same two codes) specifically for E_XCE_NOMASTER (10011) and E_XCE_NOT_READY (10013) -- both represent 'no settled primary right now, try again shortly', NOT a real failure. Do NOT add retry for E_XCE_STANDBY (10014, now redirect-handled by bug-1701) or any other status code. Suggested shape: a small fixed number of retries (e.g. 3-5) with a short fixed or lightly-backed-off delay (e.g. 1-2s), re-running the SAME follow_redirects() call from the top each attempt (a fresh attempt may get redirected differently once the election settles) -- mirror this project's existing bounded-retry conventions (e.g. verify_classify's retry loop in the HA soak driver, or follow_redirects' own _MAX_REDIRECT_HOPS=3 pattern) rather than inventing a new one. Needs a unit test proving: (a) a mocked server returning E_XCE_NOMASTER/E_XCE_NOT_READY N times then succeeding results in eventual success, not an immediate raise; (b) exceeding the retry budget still raises a clear, distinguishable error (not a silent hang); (c) E_XCE_STANDBY is NOT retried this way (still handled purely by redirect). Live-verify against a freshly-booted real cluster (the exact scenario that surfaced this) showing zero intermittent -10013 failures across N repeated invocations immediately after start-all.sh.",
+      "tags": [
+        "xceadmin",
+        "scheduler-admin",
+        "E_XCE_NOT_READY",
+        "E_XCE_NOMASTER",
+        "retry",
+        "transient-state",
+        "primary-discovery",
+        "bug-1700-adjacent",
+        "bug-1701-adjacent",
+        "found-live",
+        "root-caused",
+        "fix-scoped",
+        "unfixed"
+      ],
+      "related_bugs": [
+        "bug-1700",
+        "bug-1701"
+      ],
+      "occurrences": 1,
+      "last_seen": "2026-08-26T17:15:00Z"
     }
   ]
-}
+}