fix(controller): persist a failed job's exit code through WAL_JOB_FAIL (bug-1754)
A worker reports every nonzero job exit as a FAIL (xcecd_job_finished ->
xcecd_notify_job_fail(jobid, stage, exit_code)), not a FINISH. The deferred
WAL_JOB_FAIL fact carried only {jobid, STAGE_FLAG|stage} and no apply seam
touched job.exit_info, so exit_info stayed NULL for every failed job, live
and after replay, and HOOK_DETAIL_JOB_FAIL subscribers saw code 0.
- persist_submit_job_fail_fact(): header + int32_t code trailer, used by
record_job_fail_fact(). Old entries (no trailer) decode as before.
- apply_job_fail_exit_code(): records the code in exit_info for pre-exit
stages only; a POST_EXEC code is the script's and leaves exit_info alone.
Kept out of apply_job_fail_transition() so mark_job_fail()'s internal
-errno failures (not replayed with a code) stay live/replay consistent.
- Both apply seams adopt the code. apply_wal_entry()'s WAL_JOB_FAIL case
also stops assigning the wire state 0 for STAGE_FLAG entries and runs the
real transition, as apply_committed_entry() already does.
New test_wal_job_fail_fact.c (5 tests, 3 RED before the fix); the two
test_worker_resync.c assertions that pinned exit_info == NULL now assert the
worker-reported code. Controller suite: 1228 tests, 0 failures.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
1ba643b8be
commit
6876e4f6f2
10 changed files with 543 additions and 43 deletions
|
|
@ -3013,31 +3013,15 @@ static void test_full_rejoin_reconciliation_round_trip(void)
|
|||
TEST_ASSERT_EQUAL_UINT32_MESSAGE(JOB_STAT_EXIT, j2_finfail->job.state,
|
||||
"finished-failure: the DEFERRED on_job_fail/record_job_fail_"
|
||||
"fact fact must apply to EXIT on replay (Req 4.1)");
|
||||
/* Req 4.2 is DOCUMENTED-unsatisfiable for the finished-FAILURE
|
||||
* finish_time specifically (bug-1747, pre-existing, systemic,
|
||||
* explicitly deferred, unrelated to this task's own boundary) --
|
||||
* NOT re-asserted here as if it were correct. This round-trip test
|
||||
* additionally confirms, and CONCERNS reports, a related, more
|
||||
* complete gap bug-1747's own text does not mention: the worker-
|
||||
* reported EXIT CODE (entry->exit_code, plumbed all the way into
|
||||
* job_failinfo_t.code by resync_apply_finished_entry, resync.c) is
|
||||
* not merely mistimed but is NEVER durably recorded or applied to
|
||||
* job.job.exit_info AT ALL for a FAIL outcome, by any existing path
|
||||
* in this codebase -- record_job_fail_fact's own WAL_JOB_FAIL
|
||||
* payload (persist_submit_job_fact_ex(..., WAL_JOB_FAIL, jobid, 0,
|
||||
* WAL_JOB_FAIL_STAGE_FLAG|stage), job.c) carries no code slot at
|
||||
* all, and apply_job_fail_transition() (job.c) never touches job.
|
||||
* job.exit_info. This is asserted honestly below as NULL -- the
|
||||
* TRUE current behavior -- rather than asserting a "correct" value
|
||||
* this codebase cannot actually produce (this file's own task 3.2
|
||||
* Implementation Notes: "a green suite is not proof of correctness
|
||||
* when the test itself encodes the wrong expectation"). */
|
||||
TEST_ASSERT_NULL_MESSAGE(j2_finfail->job.exit_info,
|
||||
"pre-existing, systemic gap (see comment above, reported in "
|
||||
"this task's own CONCERNS, not fixed here): a worker-reported "
|
||||
"exit code for a FAIL outcome is never durably recorded by any "
|
||||
"existing path in this codebase -- exit_info stays NULL, not "
|
||||
"the worker-reported code");
|
||||
/* Req 4.2 remains DOCUMENTED-unsatisfiable for the finished-
|
||||
* FAILURE finish_time specifically (bug-1747) -- not asserted here.
|
||||
* The worker-reported EXIT CODE, however, is durably recorded via
|
||||
* WAL_JOB_FAIL's code trailer since bug-1754. */
|
||||
TEST_ASSERT_NOT_NULL_MESSAGE(j2_finfail->job.exit_info,
|
||||
"bug-1754: a finished-failure's worker-reported exit code must "
|
||||
"survive restart replay in job.exit_info");
|
||||
TEST_ASSERT_EQUAL_INT(worker_reported_exit_code,
|
||||
j2_finfail->job.exit_info->exitcode);
|
||||
|
||||
TEST_ASSERT_EQUAL_UINT32_MESSAGE(JOB_STAT_PEND, j2_unrec->job.state,
|
||||
"unrecognized: the EAGER record_job_reconcile_pending_fact's "
|
||||
|
|
@ -4080,12 +4064,11 @@ static void test_burst_recovery_unbounded_by_job_count(void)
|
|||
"EXIT, not DONE, distinctly from its 5 finished-success "
|
||||
"siblings in the SAME round");
|
||||
/* Req 4.2 remains documented-unsatisfiable for FAIL's own
|
||||
* finish_time specifically (bug-1747, pre-existing, systemic, out
|
||||
* of this task's boundary -- see tasks 7.1/7.2's own identical
|
||||
* finding); exit_info stays NULL for the same pre-existing reason,
|
||||
* asserted honestly rather than as a wrong "correct" value this
|
||||
* codebase cannot actually produce. */
|
||||
TEST_ASSERT_NULL(j2_lost[TWR_BURST_N_LOST - 1]->job.exit_info);
|
||||
* finish_time specifically (bug-1747). Its exit code is recorded
|
||||
* since bug-1754. */
|
||||
TEST_ASSERT_NOT_NULL(j2_lost[TWR_BURST_N_LOST - 1]->job.exit_info);
|
||||
TEST_ASSERT_EQUAL_INT(lost_fail_exit_code,
|
||||
j2_lost[TWR_BURST_N_LOST - 1]->job.exit_info->exitcode);
|
||||
|
||||
/* The 'none duplicated' completion criterion, confirmed to
|
||||
* survive a restart replay too: the already-DONE job's ORIGINAL
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue