feat(scheduler-dispatch-commit): controller resolves retryable dispatch failures with bounded escalation (task 3.3)

The four retryable-classified failure sites in dispatch_a_job_() (worker
lookup, worker null, worker not ready, RPC failure) now resolve instead
of just returning an error code: lock(j, RTTI_LOCK_WRITE) -> genuinely
classify via xce_classify_dispatch_fail() -> increment dispatch_fail_count
-> within XCE_DISPATCH_FAIL_RETRY_BOUND (3) resolves retryable (nothing
further under the lock), past it resolves terminal via mark_job_fail()
exactly like task 3.2's sites -> unlock(j) -> dereference(j) (closing
bug-1627's 4-site reference leak) -> outside the lock, notify_dispatch()
fires only on the retryable outcome. Each site's natural error code is
captured into a stable local before any intervening call, since errno
isn't guaranteed stable once lock()/ERROR()/free_launchinfo() sit between
the original failure and the eventual return.

handle_jobdispatch() is intentionally untouched -- it still unconditionally
calls notify_dispatch() itself for every outcome, so the retryable path
now temporarily double-notifies until task 4.1 removes that caller-side
call. Expected, not a defect.

New test_dispatch_fail_retry.c (4 tests, including a bounded-escalation
test driving 4 consecutive RPC failures on the same job: 3rd retries,
4th resolves EXIT) plus a fixed stale assertion in test_dispatch_commit.c
and additive, read-only notify-capture accessors in
test_commit_barrier_reply.c (the binary's sole notify() stub host).

Independently reviewed and approved, including explicit scrutiny of the
cross-file test accessor addition and the stale-assertion fix -- both
confirmed legitimate, not scope creep or regression-hiding.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cxa1PmqAB5PyXfJaNvj8TN
This commit is contained in:
Rongsong Shen 2026-08-23 17:37:07 +00:00
parent f756c4b09f
commit 05c45ef3c8
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
7 changed files with 880 additions and 12 deletions

View file

@ -133,6 +133,43 @@ static void cbrp_notify_capture_reset(void)
sizeof(g_notify_reply_already_enqueued));
}
/* scheduler-dispatch-commit task 3.3: minimal read-only accessors exposing
* this file's notify() capture (above) to other test files sharing this
* same controller_unit_test binary. This file owns the sole notify()
* definition linked into that binary (dataex.c's real one is excluded --
* see the "Link stubs" comment at the top of this file), so it is the only
* place any test in the binary can observe a notify_dispatch() (job.c)
* call. Read-only: does not alter this file's own capture/reset behavior
* or any of its own tests. Consumed by test_dispatch_fail_retry.c to
* confirm notify_dispatch() fires on the new retryable-resolution path. */
int cbrp_notify_capture_count_ext(void)
{
return g_notify_count;
}
int cbrp_notify_capture_event_ext(int idx)
{
return (idx >= 0 && idx < g_notify_count && idx < CBRP_MAX_NOTIFY)
? g_notify_events[idx] : -1;
}
uint64_t cbrp_notify_capture_jobid_ext(int idx)
{
return (idx >= 0 && idx < g_notify_count && idx < CBRP_MAX_NOTIFY)
? g_notify_jobids[idx] : 0;
}
int cbrp_notify_capture_code_ext(int idx)
{
return (idx >= 0 && idx < g_notify_count && idx < CBRP_MAX_NOTIFY)
? g_notify_codes[idx] : 0;
}
void cbrp_notify_capture_reset_ext(void)
{
cbrp_notify_capture_reset();
}
static worker_t *g_stub_ep_worker; /* NULL by default (3.1 behavior) */
/* clang-format off */