lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(controller): resync_resolve_round never told scheduler RECONCILING cleared (bug-1841)

Browse source 
Worker-rejoin-resync sets XCE_WORKER_RECONCILING and pushes it to the
scheduler, which correctly freezes that worker's resInUse mid-reconciliation
(bug-1799/1801). But resync_resolve_round() only cleared the bit on the
controller's own worker_t, never notifying the scheduler -- so once a
rejoin round resolved, the scheduler's copy stayed RECONCILING forever,
permanently freezing that worker's resource accounting even after every
job on it finished. Live-observed via a replay capture: a 22-slot worker
stuck at used=22.00 for the rest of the run with zero RUN jobs left.

Added persist_notify_worker_status_updated() (mirrors bug-1679's job-side
persist_notify_job_updated_already_persisted()) and call it from
resync_resolve_round() so HOOK_DETAIL_WORKER_UPDATE fires without a
spurious WAL append. TDD: new spy-hook regression test, RED confirmed
against the unfixed code, GREEN after the fix. Full controller_unit_test
suite: 1220 Tests, 0 Failures, 0 Ignored.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-09-22 09:28:10 +00:00
parent e1da9f1e9b
commit 1f6f219da6
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
5 changed files with 147 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

16
src/controller/persist.c
View file

@ -7983,6 +7983,22 @@ void persist_notify_job_updated_already_persisted(cluster_t* c, job_t* j)
t_apply_seam_notifying = false;
}
/* bug-1841: see persist.h's doc comment on this declaration for the full
* rationale. Same t_apply_seam_notifying-guarded shape as persist_notify_
* job_updated_already_persisted() above, applied to worker status instead
* of job state -- lets resync.c notify HOOK_DETAIL_WORKER_UPDATE's live
* subscribers (sched_worker_change/apiserver_worker_change) that `w`'s
* status changed WITHOUT persist_worker_hook_fun appending a WAL_WORKER_CTRL
* fact for it, since resync.c's XCE_WORKER_RECONCILING coordination bit is
* intentionally never durable. */
void persist_notify_worker_status_updated(cluster_t* c, worker_t* w)
{
t_apply_seam_notifying = true;
xce_run_hooks(&c->hooksys, XCE_CONTROLLER_HOOK_WORKER, HOOK_AFTER_EVENT,
HOOK_DETAIL_WORKER_UPDATE, c, w);
t_apply_seam_notifying = false;
}
/* P0 payload: just the detail value (4 bytes).
* Full Cap'n Proto serialization is wired in Task 15. */