feat(component-failover-adoption): reconcile N_MAX_MASTERS with MAX_HA_CANDIDATES

N_MAX_MASTERS (client.h) now derives from config.h's MAX_HA_CANDIDATES
instead of an independently-maintained literal (3 vs 4 previously).
Fixes worker's and scheduler's hardcoded `> 3` startup checks to
reference the constant symbolically, and updates the client-retry
test fixture so it doesn't exercise an uninitialized 4th slot.

Task 1.1 of component-failover-adoption (Requirement 1 AC7).

.kiro/specs/component-failover-adoption/tasks.md

@@ -1,126 +1,163 @@
 # Implementation Plan
 
-- [ ] 1. Foundation: shared candidate-list conversion helper
-- [ ] 1.1 Implement the candidate-list-to-client-config conversion helper in libxce
-  - Add `xce_cluster_config_to_client_config()` to `src/lib/cluster.c`, declared in `src/include/xce/cluster.h`, generalizing the clamp-to-`N_MAX_MASTERS`-with-WARN logic `xce_test` already applies inline
-  - Copies candidates in order; leaves `from`/`key` untouched for the caller to set
-  - Observable completion: given a validated `xce_cluster_config_t` with 1, 3, and 4 candidates, the function returns a correctly populated `xce_client_config_t` in each case, truncating and logging a WARN only for the 4-candidate case
-  - _Requirements: 1.5_
+- [ ] 1. Foundation: candidate-count constant reconciliation and shared conversion helper
+- [x] 1.1 Reconcile the candidate-count limit constant across libxce
+  - Derive `client.h`'s `N_MAX_MASTERS` from `config.h`'s `MAX_HA_CANDIDATES` instead of an independent literal, so the two constants can never silently diverge again
+  - Fix the two hardcoded `> 3` candidate-count checks (worker's and scheduler's startup argument validation) to reference the constant symbolically instead of a magic number, so they track the raised value automatically
+  - Update the client-retry test fixture that currently hardcodes exactly 3 literal host/port entries so it doesn't silently exercise an uninitialized 4th slot once the constant is raised
+  - Observable completion: the project builds clean with the candidate-master array now sized to 4; the two updated startup checks and the updated test fixture all reference the same symbolic constant, not a literal
+  - _Requirements: 1.7_
 
-- [ ] 1.2 Unit test the conversion helper
-  - Cover: candidate order preserved, truncation at `N_MAX_MASTERS` with WARN when given more, `NULL`/empty-candidate-list rejection with `-EINVAL`
-  - Add cases to `src/lib/testsuite/test_cluster.c`
-  - Observable completion: new test cases pass under the existing libxce unit test run
-  - _Requirements: 1.5_
-
-- [ ] 2. Worker: adopt full-candidate-list connect and transparent reconnect
-- [ ] 2.1 (P) Replace worker's per-candidate join loop with a single library-driven connect/join
-  - In `src/worker/src/controller_connector.c`: delete `xcecd_join_cluster`'s manual per-candidate iteration and `init_client_config`'s hardcoded `nmasters=1`; build the client config via the Task 1 helper from `cc_ctx->cluster_cfg` (or the existing single-host fallback when no `--peers` given) and call `xcecli_ctx_init`/`xcecli_start`/`xcecli_join` once
-  - Register a rejoin callback via `xcecli_set_reconnect_handler` (declared in `src/worker/include/controller_connector.h`) that ONLY sets a `needs_rejoin` flag and returns `0` — it must NOT call `xcecli_join` or any other `xcecli_*` function itself, because the library invokes it while `ac->failover.lock` is still held (a non-recursive mutex) and re-entering that lock via a wrapped call self-deadlocks
-  - In `checking_connection()`, immediately after each `xcecli_hello` call returns (lock now released), check and clear `needs_rejoin`; if set, call `xcecli_join(ctx, HostRole_worker, &host_attrs)` as a separate, subsequent call
-  - Preserve existing WARN/INFO log statements for standby-redirect and per-candidate outcomes around the new single-call path
-  - Reconcile `checking_connection()`'s bounded `MAX_RETRY_COUNT`/`RETRY_INTERVAL`-then-`handle_disconnect()` loop with the library's own indefinite retry: the library now owns connection recovery (Requirement 2.3 — keep retrying, never permanently give up), so `checking_connection()`'s outer bound must no longer tear the worker down after 5 failed hello probes; it becomes a liveness-probe cadence only
-  - Observable completion: starting the worker against a 3-candidate config where candidate[0] is a live standby results in the worker joined as expected, with log lines showing the standby redirect and the successful candidate; a simulated outage longer than the old 5-retry window no longer triggers `handle_disconnect()`; invoking the registered rejoin callback while the ctx's failover lock is held returns immediately without blocking
-  - _Requirements: 1.1, 1.2, 1.3, 1.5, 1.6, 2.1, 2.2, 2.3, 5.1, 5.2_
-  - _Boundary: WorkerControllerConnector_
+- [ ] 1.2 Implement the candidate-list-to-client-config conversion helper in libxce
+  - Add a pure, socket-free helper that converts a validated candidate list into a fully-populated client config, copying candidates in order and leaving identity/auth fields for the caller to set
+  - A one-candidate input is handled through the identical code path as any other count — no special-cased branch for "single controller"
+  - The truncate-with-warning safety net for an over-count input stays in place as a defensive guard, even though it can no longer be reached via a properly validated input now that the two limits match
+  - Observable completion: given a validated candidate list with 1, 3, and 4 candidates, the helper returns a correctly populated client config in each case, exercising the defensive truncation path only when called with an artificially invalid over-count input
+  - _Requirements: 1.1, 1.2, 1.3, 1.5, 1.6, 1.7_
   - _Depends: 1.1_
 
-- [ ] 2.2 Fix worker's candidate-list startup ordering
-  - In `src/worker/src/main.c`: move the `cc_ctx->cluster_cfg = cluster_cfg` assignment to before `xcecd_start()` is called, so the very first connection attempt (inside `xcecd_start`/`controller_connector_start`) already sees the full `--peers` candidate list instead of only the single-host fallback
-  - Observable completion: starting the worker with `--peers` where the FIRST listed candidate is unreachable succeeds on the first connect attempt (no restart needed), verified by a log trace showing candidate rotation during the initial join, not only during a later reconnect
-  - _Requirements: 1.4_
+- [ ] 1.3 Unit test the conversion helper and the constant reconciliation
+  - Cover: candidate order preserved; one-candidate input produces a one-master output with no branch taken; truncation-with-warning exercised directly against the defensive path; `NULL`/empty-candidate-list rejection
+  - Add a compile-time or unit-level assertion that the two candidate-count constants stay equal, guarding against reintroducing the divergence
+  - Observable completion: new test cases pass under the existing libxce unit test run, including the constant-equality assertion
+  - _Requirements: 1.5, 1.7_
+  - _Depends: 1.2_
+
+- [ ] 2. Worker: unified candidate-list connect and transparent reconnect
+- [ ] 2.1 (P) Replace worker's dual connect mechanisms with the single unified pipeline
+  - Delete the standalone client-config construction used for worker's default connection and the separate single-host branch used when no candidate list is configured — both bypass the shared candidate-list validation and conversion helper entirely today
+  - Build exactly one validated candidate list before connecting, whether it came from an operator-supplied peer list or was synthesized as a single candidate from the existing default when none was given, then run it through the shared helper and connect once
+  - Register a rejoin callback that only sets a flag and returns immediately — it must not call back into the client library itself, since the library invokes it while still holding its own lock
+  - Reconcile the periodic liveness probe's own bounded give-up behavior (today it tears the worker's connection down for good after a handful of failed probes) with the library's own indefinite retry, so an outage longer than that old bound no longer causes the worker to give up permanently — the library now owns connection recovery
+  - Observable completion: starting the worker with no candidate list configured and starting it with a single-entry candidate list produce the identical connect log sequence; a three-candidate config where the first is a live standby joins correctly, showing the standby redirect and the successful candidate; a simulated outage longer than the old give-up bound no longer tears the connection down for good; invoking the registered rejoin callback while the client's lock is held returns immediately without blocking and performs no I/O
+  - _Requirements: 1.1, 1.2, 1.3, 1.5, 2.1, 2.2, 2.3, 5.1, 5.2_
+  - _Boundary: WorkerControllerConnector_
+  - _Depends: 1.2_
+
+- [ ] 2.2 Fix worker's candidate-list startup ordering and add rejoin retry-on-failure
+  - Move the candidate-list assignment to before the connector starts, so the very first connection attempt already sees the full configured list instead of only a synthesized single candidate
+  - After each periodic liveness probe returns, check whether a rejoin is pending; if so, issue the rejoin call as a separate step and clear the pending indicator only if that call succeeds — on its own failure, leave the indicator set so the next liveness probe retries the rejoin rather than leaving the worker connected but never actually re-registered
+  - Observable completion: starting the worker with a peer list where the first listed candidate is unreachable succeeds on the very first connect attempt, not only on a later reconnect; a rejoin call that fails once before succeeding is retried on the next liveness-probe cycle rather than abandoned
+  - _Requirements: 1.4, 2.5_
+  - _Depends: 2.1_
 
 - [ ] 2.3 Validate worker thread isolation and add worker candidate/reconnect test coverage
-  - Confirm (via code read) that `hardware_monitor_update`/`system_monitor_update`/`process_manager_update` do not execute on the connector thread that now carries the library's blocking retry/backoff — fix isolation if this is not already true
-  - Add `src/worker/testsuite/test_e2e_multimaster_join.c`: worker starts and joins across a multi-candidate config; killing the joined primary results in reconnect + successful re-join to a surviving candidate without a worker restart
-  - Observable completion: the new test passes, and worker's hardware/process monitoring continues updating (observable via existing monitor state) while a connect/reconnect retry is in progress
-  - _Requirements: 1.6, 3.1, 3.2_
+  - Confirm the connector thread is isolated from the worker's other recurring liveness duties, and that a blocked connector thread during retry/backoff does not stall them
+  - Add end-to-end coverage: worker starts and joins across a multi-candidate config; killing the joined primary results in reconnect and re-join to a surviving candidate without a worker restart
+  - Add coverage for the case where every configured candidate is unreachable or rejects the join at startup: assert the worker reports a clear failure identifying every candidate it attempted, not a generic error
+  - Confirm the worker's periodic liveness probe to a connected controller keeps its regular cadence and keeps succeeding even while a separate reconnect is in progress elsewhere, so a stalled reconnect attempt cannot itself look like a missed heartbeat and trigger an unwanted failover
+  - Observable completion: the new test passes, and the worker's other liveness duties continue observably updating while a connect/reconnect retry is in progress on the connector thread; the all-candidates-unreachable case produces a failure message naming each attempted candidate; the liveness probe's cadence to the connected controller is unaffected by a concurrent reconnect elsewhere
+  - _Requirements: 1.6, 3.1, 3.3_
   - _Depends: 2.1, 2.2_
 
-- [ ] 3. Scheduler: adopt full-candidate-list connect and add runtime reconnect
-- [ ] 3.1 (P) Replace scheduler's inline per-candidate loop with a single library-driven connect/join
-  - In `src/scheduler/src/scheduler.c`: delete `sched_init`'s inline per-candidate loop; build the client config via the Task 1 helper from `sched->cluster_cfg` (or the existing single-host fallback) and call `xcecli_ctx_init`/`xcecli_start`/`xcecli_join` once
-  - Register a rejoin callback via `xcecli_set_reconnect_handler` that ONLY sets a `needs_rejoin` flag and returns `0` — same constraint as worker's Task 2.1: the library invokes it while `ac->failover.lock` is held, so it must not call `xcecli_join` itself or it self-deadlocks
-  - After each `xcecli_jobdispatch` call returns (lock now released), check and clear `needs_rejoin`; if set, call `xcecli_join(ctx, HostRole_scheduler, NULL)` as a separate, subsequent call before continuing
-  - Preserve/add per-candidate outcome logging equivalent to what the deleted loop produced
-  - Observable completion: starting the scheduler against a 3-candidate config where candidate[0] is a live standby results in the scheduler joined as expected, with log lines showing the standby redirect and the successful candidate — and killing the joined primary afterward results in automatic reconnect and re-join to a surviving candidate, which the scheduler has never done before this task; invoking the registered rejoin callback while the ctx's failover lock is held returns immediately without blocking
-  - _Requirements: 1.1, 1.2, 1.3, 1.5, 1.6, 2.1, 2.2, 2.3, 5.1, 5.2_
+- [ ] 3. Scheduler: unified candidate-list connect, decision-publish redelivery, and runtime reconnect
+- [ ] 3.1 (P) Replace scheduler's dual connect mechanisms with the single unified pipeline
+  - Delete the scheduler's inline per-candidate loop and its separate single-host branch used when no candidate list is configured — both bypass the shared candidate-list validation and conversion helper entirely today
+  - Build exactly one validated candidate list before connecting, whether from an operator-supplied peer list or synthesized as a single candidate from the existing default when none was given, then run it through the shared helper and connect once
+  - Register a rejoin callback that only sets a flag and returns immediately, for the same self-deadlock reason as worker's
+  - Observable completion: starting the scheduler with no candidate list and with a single-entry candidate list produce the identical connect log sequence; a three-candidate config where the first is a live standby joins correctly; invoking the registered rejoin callback while the client's lock is held returns immediately without blocking and performs no I/O
+  - _Requirements: 1.1, 1.2, 1.3, 1.5, 1.7, 2.1, 2.2, 2.3, 5.1, 5.2_
   - _Boundary: SchedulerControllerClient_
-  - _Depends: 1.1_
+  - _Depends: 1.2_
 
-- [ ] 3.2 Add scheduler candidate/reconnect test coverage
-  - Add `src/scheduler/testsuite/test_e2e_multimaster_join.c`: scheduler starts and joins across a multi-candidate config; killing the joined primary results in reconnect and re-join without a scheduler restart
-  - Confirm `sched_job_picker` continues running on its own thread, unaffected, while `sched_allocator`'s `xcecli_jobdispatch` call is blocked in reconnect/backoff (existing thread split, not new code)
-  - Observable completion: the new test passes, and the job-picker thread's activity (e.g. its own state/counters) continues advancing during a simulated reconnect delay on the allocator thread
-  - _Requirements: 3.1, 3.2_
+- [ ] 3.2 Add decision-publish retry-until-success-or-shutdown, composed with rejoin retry-on-failure
+  - Wrap the scheduler's per-decision publish call so that a retry-budget-exhausted outcome is retried rather than treated as final and the decision discarded — checking whether the scheduler is shutting down before each attempt, not only after, so decisions still queued behind the one actively retrying do not each burn a full retry cycle just to discover shutdown was requested
+  - After a publish finally succeeds, check whether a rejoin is pending; if so, issue it as a separate step and clear the pending indicator only on that call's success
+  - Log an extended-retry success distinctly from an ordinary first-attempt success
+  - Observable completion: a publish call that fails repeatedly before succeeding results in the same decision retried — never dropped, never skipped to the next queued one; setting the shutdown flag mid-retry breaks the loop within one retry cycle
+  - _Requirements: 7.1, 7.2, 7.3, 7.4, 2.5_
   - _Depends: 3.1_
 
-- [ ] 4. apiserver: adopt full-candidate-list connect via CGO, config parity, rejoin, and fail-fast
+- [ ] 3.3 Validate scheduler allocation/decision backpressure behavior and add connect/reconnect test coverage
+  - Confirm the bounded, occupancy-dependent buffering behavior: the scheduler's allocation/decision processing continues briefly past the point a retry begins, then blocks until connectivity is restored and resumes automatically — assert the worst-case (least-slack) occupancy scenario explicitly, not only the most-generous one, since that is the guarantee actually being made
+  - Add end-to-end coverage: scheduler starts and joins across a multi-candidate config; killing the joined primary results in reconnect and re-join without a scheduler restart
+  - Add coverage for the case where every configured candidate is unreachable or rejects the join at startup: assert the scheduler reports a clear failure identifying every candidate it attempted
+  - Confirm by code inspection that the scheduler makes no periodic liveness/heartbeat call of its own to a connected controller — the reconnect-retries-don't-delay-heartbeat-responses guarantee is satisfied by construction here, not by a dedicated runtime assertion, since there is no such response to delay
+  - Observable completion: the new test passes, showing allocation/decision processing continuing briefly during a simulated reconnect delay, then stalling for the remainder of the outage, then resuming automatically once the connection recovers; the all-candidates-unreachable case produces a failure message naming each attempted candidate
+  - _Requirements: 1.6, 3.2, 3.3_
+  - _Depends: 3.1_
+
+- [ ] 3.4 Add scheduler decision-redelivery test coverage
+  - Assert decisions already queued behind a stuck one publish afterward in their original order, not out of order and not dropped
+  - With several decisions queued when scheduler shutdown fires mid-retry on the first, assert total shutdown latency stays bounded to roughly one retry cycle rather than accumulating one cycle per queued decision
+  - Confirm this fix is scoped to the scheduler's own decision-publishing direction only — it does not touch worker job-state resync and adds no persistence for queued-but-unpublished decisions, so a scheduler process restart mid-outage still loses whatever was queued, same as before this work
+  - Observable completion: all decision-redelivery acceptance criteria are exercised by passing tests, including the bounded-shutdown-latency case and a restart-loses-in-flight-state confirmation
+  - _Requirements: 7.1, 7.2, 7.3, 7.4, 7.5, 7.6, 2.5_
+  - _Depends: 3.2_
+
+- [ ] 4. apiserver: unified candidate-list connect via CGO, config parity, rejoin, and fail-fast
 - [ ] 4.1 (P) Route apiserver's candidate-list parsing through the real C parser
-  - In `src/frontend/apiserver/backend/clib.go`: call `xce_parse_peers`/`xce_cluster_config_validate` via CGO to parse `config.Peers` into a `xce_cluster_config_t`, replacing the hand-written Go `parsePeer` function and its call sites
-  - Map C validation errors (`-EINVAL` cases) to clear Go error messages naming the malformed entry
-  - Observable completion: the same malformed-candidate-string test cases that scheduler/worker already reject (from Task 1.2's neighboring validation coverage) are rejected identically when passed through apiserver's config path
+  - Replace the hand-written Go candidate-list parser with the shared C parser used by worker and scheduler, retiring the Go parser and its call sites entirely
+  - Map validation errors to clear Go error messages naming the malformed entry
+  - Observable completion: the same malformed-candidate-string cases worker/scheduler already reject are rejected identically when passed through apiserver's config path
   - _Requirements: 6.1, 6.2_
   - _Boundary: ApiserverBackendBridge_
-  - _Depends: 1.1_
+  - _Depends: 1.2_
 
-- [ ] 4.2 Replace apiserver's per-peer connect loop with a single library-driven connect
-  - In `src/frontend/apiserver/backend/clib.go`: replace `RegisterToBackend`'s `for _, peer := range config.Peers { tryConnect(...) }` loop with one call that builds the client config via the Task 1 helper (through CGO) from the full parsed candidate list (or the single `Address`/`Port` fallback) and calls `xce_backend_init` once
-  - Preserve/extend existing connect-outcome logging (`zap`) to report each candidate attempted
-  - Observable completion: starting apiserver with 3 configured peers where the first is a live standby results in apiserver registered against the actual primary in a single `RegisterToBackend` call, with log lines showing the standby redirect
-  - _Requirements: 1.1, 1.2, 1.3, 1.5, 1.6, 5.1, 5.2_
+- [ ] 4.2 Replace apiserver's dual connect mechanisms with the single unified pipeline
+  - Delete apiserver's per-peer Go connect loop and its separate direct-construction path used when no peer list is configured — both bypass the shared candidate-list validation and conversion helper entirely today
+  - Build exactly one validated candidate list before connecting, whether from the configured peer list (via Task 4.1's parser) or synthesized as a single candidate from the legacy address/port fields when no peer list was given, then run it through the shared helper (via CGO) and connect once
+  - Observable completion: starting apiserver with no peer list and with a single-entry peer list produce the identical connect log sequence; a three-peer config where the first is a live standby registers against the actual primary in one call
+  - _Requirements: 1.1, 1.2, 1.3, 1.5, 1.7, 5.1, 5.2_
   - _Depends: 4.1_
 
-- [ ] 4.3 Add apiserver's rejoin logic with a correctly-placed cgo registration
-  - New `src/frontend/apiserver/backend/reconnect.go`: two plain Go functions — (1) a `//export`-annotated flag-setter that ONLY sets a `needs_rejoin` atomic flag and returns `0` (no I/O — the library invokes it while `ac->failover.lock` is held, so calling `xcecli_join` directly here self-deadlocks); (2) a plain function that performs the actual rejoin — calls `xcecli_join(gCtx.ctx, HostRole_apiserver, NULL)` and clears backend-health degraded state on success
-  - In `src/frontend/apiserver/backend/clib.go`'s cgo preamble: add a static C shim (mirroring the existing `xce_backend_handler`/`go_push_handler` pattern) with its own `extern` declaration for the flag-setter export, and pass the shim to `xcecli_set_reconnect_handler` — the registration must live in `clib.go`, not in `reconnect.go`, because cgo symbol resolution is per-file (same reason `go_push_handler` needs `xce_backend_handler`, documented in `wire_dispatch_support.go`)
-  - Add one shared internal helper in `clib.go` that wraps a `C.xcecli_*` call: after the call returns (lock now released), check and clear `needs_rejoin`; if set, call `reconnect.go`'s actual-rejoin function as a separate step. Route all four backend-call wrappers (job submit, job control, queue control, host control) through this ONE helper instead of duplicating the check in each (Task 4.5 extends this same helper with the pre-call health check, rather than adding a second one)
-  - Observable completion: after killing and restoring connectivity to apiserver's joined controller while a write request is in flight (job submit retried through the full failover path), apiserver resumes receiving backend push notifications (existing `go_push_handler` path) without an apiserver process restart; invoking the registered flag-setter while the ctx's failover lock is held returns immediately without blocking
-  - _Requirements: 2.1, 2.2, 2.3, 2.4_
+- [ ] 4.3 Add apiserver's rejoin logic with a correctly-placed cgo registration and retry-on-failure
+  - Add a flag-setter registered as the library's rejoin callback that does no I/O, only sets a pending-rejoin flag and returns immediately, and a separate function that performs the actual rejoin call
+  - Register the flag-setter's C shim in the same file as its `extern` declaration (cgo symbol resolution is per-file)
+  - Add one shared helper wrapping every backend call: after the call returns, check the pending-rejoin flag; if set, issue the rejoin as a separate step, clearing the flag only if that call succeeds — on its own failure, leave it set so the next backend call's return retries the rejoin
+  - Observable completion: after killing and restoring connectivity while a write request is in flight, apiserver resumes receiving backend push notifications without a process restart; a rejoin call that fails once before succeeding is retried on the next backend call rather than abandoned; invoking the registered flag-setter while the client's lock is held returns immediately without blocking and performs no I/O
+  - _Requirements: 2.1, 2.2, 2.3, 2.4, 2.5_
   - _Depends: 4.2_
 
 - [ ] 4.4 Trigger apiserver's rejoin from the idle-period heartbeat path
-  - In `src/frontend/apiserver/backend/heartbeat.go`'s tick callback: track the previous tick's `xcecli_heartbeat` outcome; when a tick transitions from failure to success (the fire-and-forget call's own best-effort transport reconnect just succeeded), explicitly call Task 4.3's actual-rejoin function (function (2), not the flag-setter) directly — safe here since `XCECLI_FF_CALL` has already released the lock by the time the heartbeat call returns. `XCECLI_FF_CALL` (backing `xcecli_heartbeat`) never invokes the library's own rejoin hook, unlike the full-retry path write calls use, so without this the idle-period case never re-joins
-  - Observable completion: killing the primary while apiserver has no in-flight write request (only the heartbeat ticking) still results in apiserver rejoining and resuming push notifications, verified by the same push-notification check as Task 4.3 but with no write traffic present during the outage
-  - _Requirements: 2.4_
+  - On a transport failure-to-success transition detected by the heartbeat prober, set the same pending-rejoin flag the write path uses (the heartbeat's own probe mechanism never sets it itself)
+  - On every subsequent heartbeat tick where the probe succeeds and a rejoin is still pending, retry the rejoin — checked every tick, not only on the transition edge, so a rejoin attempt that itself fails still gets retried on a later tick
+  - Do not gate this retry on the separate backend-degraded circuit-breaker state, since that state clears itself on its own timer independent of whether a rejoin actually succeeded
+  - Observable completion: killing the primary while apiserver has no in-flight write request still results in apiserver rejoining and resuming push notifications; a rejoin that fails on the first post-recovery tick before succeeding on a later tick confirms the retry, not just the single-attempt case
+  - _Requirements: 2.4, 2.5_
   - _Depends: 4.3_
 
 - [ ] 4.5 Add apiserver's backend-health circuit breaker
-  - New `src/frontend/apiserver/backend/health.go`: atomic degraded-until state with `IsDegraded()`, `MarkDegraded(ttl)`, `MarkHealthy()`
-  - Extend Task 4.3's shared internal helper (not a second, separate hook) to also check `IsDegraded()` before the `C.xcecli_*` call and return a clear "backend unavailable" error immediately instead of invoking it when degraded; wire `MarkDegraded` into the same helper's post-call path when a call's result indicates retry-budget exhaustion; wire `MarkHealthy` into both Task 4.3's rejoin function (write-path recovery) and Task 4.4's heartbeat-triggered rejoin (idle-path recovery)
-  - Observable completion: with the flag forced degraded in a unit test, a wrapped backend call returns immediately with the unavailable error instead of invoking the C call
+  - Add a degraded-until state that lets request handlers fail fast instead of blocking on the full retry budget when the backend is known-degraded, clearing on either a successful rejoin or its own timeout
+  - Wire it into the shared backend-call helper: check before every call, mark degraded when a call's outcome indicates retry-budget exhaustion, mark healthy on a successful rejoin from either the write path or the idle-period path
+  - Keep this state's own timeout independent from the pending-rejoin flag — the circuit breaker may reset optimistically on its timer, but that must never suppress the pending-rejoin flag's own retry-until-actually-joined behavior
+  - Observable completion: with the state forced degraded, a wrapped backend call returns immediately with a clear unavailable error instead of invoking the backend call
   - _Requirements: 4.1, 4.2_
   - _Depends: 4.4_
 
 - [ ] 4.6 apiserver unit test coverage
-  - `src/frontend/apiserver/backend/health_test.go`: degraded→healthy via `MarkHealthy()` and via TTL expiry; concurrent-goroutine safety
-  - Extend `src/frontend/apiserver/backend/clib_test.go` with candidate-list parity cases from Task 4.1
-  - Observable completion: `make test` in `src/frontend/` passes with the new cases included
-  - _Requirements: 4.1, 4.2, 6.1, 6.2_
+  - Cover: degraded-to-healthy transitions via both an explicit successful rejoin and timeout expiry; concurrent-request safety; candidate-list parity cases from Task 4.1
+  - Observable completion: the full apiserver test suite passes with the new cases included
+  - _Requirements: 1.5, 4.1, 4.2, 6.1, 6.2_
   - _Depends: 4.5_
 
 - [ ] 5. Integration and cross-component validation
 - [ ] 5.1 E2E: full-cluster reconnect after primary failover
-  - Extend the project's existing `/xce-deploy-ha` + `/xce-test-ha` pattern to a scenario with worker, scheduler, and apiserver all configured with the full 3-candidate list; kill the primary mid-session with a write request in flight
+  - Extend the project's existing HA soak pattern to a scenario with worker, scheduler, and apiserver all configured with the full candidate list; kill the primary mid-session with a write request in flight
   - Assert: worker reconnects and resumes job-state reporting; scheduler reconnects and resumes dispatch; apiserver reconnects, resumes serving API requests, and resumes receiving push notifications — none of the three restarted
   - Observable completion: the soak scenario passes with all three components showing a successful re-join to the new primary in their logs and no manual intervention
   - _Requirements: 2.1, 2.2, 2.3, 2.4_
-  - _Depends: 2.3, 3.2, 4.6_
+  - _Depends: 2.3, 3.3, 3.4, 4.6_
 
-- [ ] 5.2 E2E: apiserver reconnects during an idle period (no in-flight write traffic)
-  - Kill the primary while apiserver has only its heartbeat prober active (no pending write requests); assert apiserver still rejoins and resumes push notifications via the Task 4.4 heartbeat-triggered path
+- [ ] 5.2 E2E: apiserver reconnects during an idle period
+  - Kill the primary while apiserver has only its heartbeat prober active, no pending write requests; assert apiserver still rejoins and resumes push notifications via the heartbeat-triggered path
   - Observable completion: the scenario passes without requiring any client-triggered write request during the outage window
-  - _Requirements: 2.4_
+  - _Requirements: 2.4, 2.5_
   - _Depends: 4.6_
 
-- [ ] 5.3 E2E: apiserver fails fast while backend is degraded
-  - With apiserver's backend connection killed and not yet reconnected, issue an API request that requires the backend and assert it returns a clear 503-style error within a bounded, sub-second time — not after the full retry budget
-  - Observable completion: the timed assertion passes against a live apiserver instance
+- [ ] 5.3 E2E: apiserver fails fast while backend is degraded, and keeps serving requests that don't need it
+  - With apiserver's backend connection killed and not yet reconnected, issue an API request that requires the backend and assert it returns a clear service-unavailable error within a bounded, sub-second time — not after the full retry budget
+  - In the same degraded window, issue a request that does not require the backend connection and assert it still succeeds normally, confirming degradation is scoped to backend-dependent requests only
+  - Observable completion: both timed assertions pass against the same live apiserver instance while it remains degraded
   - _Requirements: 4.1, 4.2_
   - _Depends: 4.6_
 
-- [ ] 5.4 Regression: confirm existing single-controller mode is unaffected
-  - Run the full worker, scheduler, and apiserver test suites (ASAN where applicable) plus the existing non-HA (no `--peers`/no `backend.peers`) startup path for each component
-  - Observable completion: all existing tests pass unchanged, and each component still starts and operates normally when no candidate list is configured (Requirement 1.5's single/no-candidate fallback)
+- [ ] 5.4 Regression: unified single-candidate path is observably correct across all three components
+  - Run the full worker, scheduler, and apiserver test suites, plus the startup path for each component with no candidate list configured
+  - Confirm each component still starts and operates normally with no candidate list configured, and that this now takes the identical code path as a validated single-entry candidate list rather than a separately-maintained fallback
+  - Observable completion: all existing tests pass unchanged, and each component's no-candidate-list startup log sequence matches its single-entry-candidate-list startup log sequence
   - _Requirements: 1.5_
   - _Depends: 5.1, 5.2, 5.3_
+
+## Implementation Notes
+- Task 1.1: `src/lib/cluster.c:99-102`'s `xce_cluster_config_validate()` still has a third hardcoded `>3` WARN check (message says "<=3 only") of the same magic-number class Req 1.7 targets, and `src/include/xce/cluster.h:31`'s doc comment repeats the stale "3" figure. Task 1.1's text named only "the two" worker/scheduler checks, so this was left untouched there — task 1.2 already touches `cluster.c` to add the new conversion helper, so fold this fix in there rather than letting it silently persist as the one remaining un-reconciled magic number.