feat(scheduler-dispatch-commit): controller resolves retryable dispatch failures with bounded escalation (task 3.3)

The four retryable-classified failure sites in dispatch_a_job_() (worker
lookup, worker null, worker not ready, RPC failure) now resolve instead
of just returning an error code: lock(j, RTTI_LOCK_WRITE) -> genuinely
classify via xce_classify_dispatch_fail() -> increment dispatch_fail_count
-> within XCE_DISPATCH_FAIL_RETRY_BOUND (3) resolves retryable (nothing
further under the lock), past it resolves terminal via mark_job_fail()
exactly like task 3.2's sites -> unlock(j) -> dereference(j) (closing
bug-1627's 4-site reference leak) -> outside the lock, notify_dispatch()
fires only on the retryable outcome. Each site's natural error code is
captured into a stable local before any intervening call, since errno
isn't guaranteed stable once lock()/ERROR()/free_launchinfo() sit between
the original failure and the eventual return.

handle_jobdispatch() is intentionally untouched -- it still unconditionally
calls notify_dispatch() itself for every outcome, so the retryable path
now temporarily double-notifies until task 4.1 removes that caller-side
call. Expected, not a defect.

New test_dispatch_fail_retry.c (4 tests, including a bounded-escalation
test driving 4 consecutive RPC failures on the same job: 3rd retries,
4th resolves EXIT) plus a fixed stale assertion in test_dispatch_commit.c
and additive, read-only notify-capture accessors in
test_commit_barrier_reply.c (the binary's sole notify() stub host).

Independently reviewed and approved, including explicit scrutiny of the
cross-file test accessor addition and the stale-assertion fix -- both
confirmed legitimate, not scope creep or regression-hiding.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cxa1PmqAB5PyXfJaNvj8TN
This commit is contained in:
Rongsong Shen 2026-08-23 17:37:07 +00:00
parent f756c4b09f
commit 05c45ef3c8
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
7 changed files with 880 additions and 12 deletions

View file

@ -181,6 +181,7 @@ controller_unit_test_src = [
'testsuite/test_job_dispatch.c',
'testsuite/test_dispatch_commit.c',
'testsuite/test_dispatch_fail_terminal.c',
'testsuite/test_dispatch_fail_retry.c',
'testsuite/test_upgrade_snapshot_restore.c',
'testsuite/test_upgrade_snapshot_apply.c',
'testsuite/test_bug857_regression.c',