feat(worker-rejoin-resync): reconcile unrecognized job to pending (task 2.1)
Add record_job_reconcile_pending_fact(cluster_t*, job_t*) to job.c: eagerly applies apply_job_update_transition(job, JOB_STAT_PEND) (real, live mutation, not a deferred peek -- matching dispatch_a_job_'s established pattern, not record_job_finish_fact's), releases job->job.allocs on an applied (non-no-op) transition, then durably appends via the generic persist_submit_job_fact(WAL_JOB_UPDATE). Caller holds job's lock throughout and is responsible for calling persist_notify_job_updated_already_persisted() after unlocking -- this function deliberately does not call it itself. Closes bug-1631 at this new call site: a RUN job with a real allocation reconciled to PEND (Requirement 4.6, worker reports it unrecognized) would otherwise carry a stale allocation into its next dispatch attempt, exactly the gap bug-1631 documented as dormant until something re-admits an already-dispatched job to the scheduler's PEND pool -- this feature is that path. dispatch_a_job_ itself is unchanged (still correct for a job's first-ever dispatch, the only case reachable there). 3 tests: allocs release + eager-mutation-before-unlock proof, no-op on already-pending, and the Requirement 4.4 terminal-state-preservation proof (rejects overwriting JOB_STAT_DONE). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y4M5MJM2zK6nU43da267Ku
This commit is contained in:
parent
eda2eb1bf0
commit
cfe66dde16
7 changed files with 433 additions and 2 deletions
|
|
@ -2866,6 +2866,71 @@ static int mark_job_update(cluster_t* c, job_t* job, int state)
|
|||
return 0;
|
||||
}
|
||||
|
||||
/* worker-rejoin-resync task 2.1 (Req 4.1-4.6, 5.4, 7.1-7.3): returns a job
|
||||
* the worker reports as unrecognized back to PEND, releasing any resource
|
||||
* allocation it was carrying so a subsequent redispatch attempt never
|
||||
* inherits a stale allocation from a prior dispatch (bug-1631, confirmed
|
||||
* live and deliberately left unfixed until this feature provided the first
|
||||
* real re-admission path -- see design.md's Error Handling section).
|
||||
*
|
||||
* Follows dispatch_a_job_()'s own established pattern (job.c, its `if (rc
|
||||
* >= 0) { ... }` success block): eager, synchronous mutation under the
|
||||
* write lock the caller already holds, immediately followed by a durable
|
||||
* WAL append, immediately followed (by the CALLER, after unlocking) by an
|
||||
* explicit persist_notify_job_updated_already_persisted() call -- NOT
|
||||
* record_job_finish_fact's/record_job_fail_fact's purely-deferred pattern
|
||||
* (mutation withheld until apply_committed_entry replays the committed
|
||||
* fact). The fact recorded here is something the worker already confirmed
|
||||
* (or, in this case, failed to confirm) really happened -- ground truth
|
||||
* already changed outside this controller -- which is exactly dispatch_a_
|
||||
* job_'s own justification for going eager rather than deferred (design.md,
|
||||
* "Controller: job.c" Components section).
|
||||
*
|
||||
* Precondition: caller holds `job` under RTTI_LOCK_WRITE on entry and keeps
|
||||
* holding it until this call returns -- this function does not lock/unlock
|
||||
* `job` itself, matching apply_job_update_transition's own calling
|
||||
* convention (its other caller, mark_job_update above, is called under the
|
||||
* exact same already-held write lock).
|
||||
*
|
||||
* apply_job_update_transition()'s existing job_state_is_backward guard is
|
||||
* the ONLY commit/idempotency guard exercised here -- reused unchanged,
|
||||
* never duplicated: a terminal job's transition is rejected
|
||||
* (JOB_TRANSITION_REJECTED_BACKWARD, Req 4.4) and an already-PEND job's
|
||||
* transition is a no-op (JOB_TRANSITION_NOOP_DUPLICATE, Req 4.3) -- neither
|
||||
* outcome touches job->job.allocs or appends a WAL fact. Only on
|
||||
* JOB_TRANSITION_APPLIED does this release job->job.allocs (if non-NULL)
|
||||
* and durably append the fact via the existing generic persist_submit_job_
|
||||
* fact() -- the plain {jobid, state} shape is sufficient here, same as
|
||||
* every other WAL_JOB_UPDATE caller (no extra fields, unlike DISPATCH/
|
||||
* START/FINISH's richer siblings).
|
||||
*
|
||||
* Returns the job_transition_result_t of the underlying transition, cast to
|
||||
* int, so the caller can tell APPLIED apart from REJECTED_BACKWARD/
|
||||
* NOOP_DUPLICATE. Unlike record_job_finish_fact/record_job_fail_fact, this
|
||||
* function does NOT call persist_notify_job_updated_already_persisted --
|
||||
* the caller must do so itself, after releasing the write lock, exactly
|
||||
* once, for every JOB_TRANSITION_APPLIED result (cluster.h's doc comment on
|
||||
* the prototype carries this same contract). Omitting it reproduces
|
||||
* bug-1679 at this call site. */
|
||||
int record_job_reconcile_pending_fact(cluster_t* c, job_t* job)
|
||||
{
|
||||
job_transition_result_t result;
|
||||
|
||||
result = apply_job_update_transition(job, JOB_STAT_PEND);
|
||||
if (result != JOB_TRANSITION_APPLIED)
|
||||
return (int)result;
|
||||
|
||||
if (job->job.allocs != NULL) {
|
||||
free_JobAlloc_ptr(&(job->job.allocs));
|
||||
job->job.allocs = NULL;
|
||||
}
|
||||
|
||||
(void)persist_submit_job_fact(&c->persist, WAL_JOB_UPDATE, job->job.jobid,
|
||||
JOB_STAT_PEND);
|
||||
|
||||
return (int)result;
|
||||
}
|
||||
|
||||
/* Guarded state-transition core for a postdone fact, extracted from
|
||||
* on_job_postdone's old inline mutation (raft-apply-decoupling task 6.1
|
||||
* round 2, Req 3.3/7.1-7.3). Unlike a generic UPDATE, postdone's formula
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue