lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(controller): persist a failed job's exit code through WAL_JOB_FAIL (bug-1754)

Browse source 
A worker reports every nonzero job exit as a FAIL (xcecd_job_finished ->
xcecd_notify_job_fail(jobid, stage, exit_code)), not a FINISH. The deferred
WAL_JOB_FAIL fact carried only {jobid, STAGE_FLAG|stage} and no apply seam
touched job.exit_info, so exit_info stayed NULL for every failed job, live
and after replay, and HOOK_DETAIL_JOB_FAIL subscribers saw code 0.

- persist_submit_job_fail_fact(): header + int32_t code trailer, used by
  record_job_fail_fact(). Old entries (no trailer) decode as before.
- apply_job_fail_exit_code(): records the code in exit_info for pre-exit
  stages only; a POST_EXEC code is the script's and leaves exit_info alone.
  Kept out of apply_job_fail_transition() so mark_job_fail()'s internal
  -errno failures (not replayed with a code) stay live/replay consistent.
- Both apply seams adopt the code. apply_wal_entry()'s WAL_JOB_FAIL case
  also stops assigning the wire state 0 for STAGE_FLAG entries and runs the
  real transition, as apply_committed_entry() already does.

New test_wal_job_fail_fact.c (5 tests, 3 RED before the fix); the two
test_worker_resync.c assertions that pinned exit_info == NULL now assert the
worker-reported code. Controller suite: 1228 tests, 0 failures.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-09-26 03:32:54 +00:00
parent 1ba643b8be
commit 6876e4f6f2
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
10 changed files with 543 additions and 43 deletions
Download patch file Download diff file Expand all files Collapse all files

5
src/controller/resync.c
View file

@ -642,9 +642,8 @@ static void resync_apply_running_entry(cluster_t* c, worker_t* w,
* finishedFail: still routed through the existing, UNMODIFIED on_job_fail()
* /record_job_fail_fact() -- Requirement 4.2 remains unsatisfiable for this
* outcome via any existing path in this codebase, not a reconciliation-
* specific gap: WAL_JOB_FAIL's payload (persist_submit_job_fact_ex(...,
* WAL_JOB_FAIL, jobid, 0, WAL_JOB_FAIL_STAGE_FLAG|stage)) carries no
* finish-time slot at any layer, and apply_job_fail_transition() (job.c)
* specific gap: WAL_JOB_FAIL's payload (persist_submit_job_fail_fact():
* stage plus, since bug-1754, the exit code) carries no finish-time slot, and apply_job_fail_transition() (job.c)
* unconditionally sets job->job.finish_time = time(NULL) at APPLY time
* (not even receipt time) for EVERY fail-reporting caller in this system,
* confirmed by direct inspection -- a pre-existing, systemic gap that