--- id: "bug-1750" status: "fixed" status_updated: "2026-09-24" status_note: "" component: "worker" severity: "unknown" timestamp: "2026-09-01T09:00:00Z" last_seen: "2026-09-19T09:00:00Z" occurrences: 1 files: ["src/worker/src/controller_connector.c"] tags: ["worker-rejoin-resync", "task-4.3", "xcecd_resync-c", "controller_connector-c", "requirement-6.1", "requirement-3.8", "deadlock", "gate-placement", "checking_connection", "needs_rejoin", "review-caught"] related: ["bug-1749", "bug-1748"] legacy: true --- # bug-1750: worker-rejoin-resync task 4.3, round-1 review: the resync_pending gate was checked as… ## Description worker-rejoin-resync task 4.3, round-1 review: the resync_pending gate was checked as checking_connection()'s very first statement (an entry guard), returning early before xcecli_hello() or the needs_rejoin-triggered rejoin_call() could run. xcecd_resync_handle_query_page() sets resync_pending = true on receiving a job-state query's first page and only clears it when called again with a later page. If the worker's connection to the controller drops after a non-final page of a multi-page OPCode_jobStateQuery arrives (a query spanning more non-terminal jobs than fit in one page -- explicitly in scope per Requirement 8.1/8.3), no further page ever arrives, so xcecd_resync_handle_query_page() is never invoked again and resync_pending is stuck true forever -- permanently disabling checking_connection()'s entire body, including the worker's ONLY connection-recovery mechanism (xcecli_hello() / needs_rejoin-triggered rejoin). Directly violated Requirement 6.1 (the worker must be able to disconnect-and-rejoin during an incomplete reconciliation round) and regressed the intent of the pre-existing invariant test test_checking_connection_never_permanently_disables_worker, though none of round 1's own new tests drove the multi-page-then-disconnect sequence so it passed the full suite undetected. ## Environment _Not recorded (migrated from buglog.json)._ ## Reproduction _Not recorded (migrated from buglog.json)._ ## Root cause design.md directed mirroring the existing needs_rejoin boolean pattern for the new resync_pending gate, but needs_rejoin is self-healing by construction -- it gates one step INSIDE checking_connection() (the rejoin action itself), so the function's body including xcecli_hello() still runs every tick regardless of its value, giving the worker a fresh chance to detect and recover every cycle. Round 1 placed resync_pending's check as the function's entry guard instead, which is not actually the same pattern in the load-bearing respect that matters: it makes the gate's only exit condition (a later query page arriving) depend on the very connection health the gate is supposed to be resilient to, with no independent path back to a working state. ## Fix FIXED (2026-09-01, round-2 review, task 4.3). Two changes, defense-in-depth: (a) narrowed the gate so xcecd_resync_is_pending() is checked only inside the needs_rejoin branch, guarding solely rejoin_call() -- xcecli_hello() and the needs_rejoin check itself now run unconditionally on every checking_connection() tick, restoring the self-healing property. (b) added xcecd_resync_reset(), called from xcecd_on_reconnect() (the same signal that sets needs_rejoin, confirmed to fire only after xce_failover_rotate()/xcecli_invoke_rejoin() has already repaired a real, previously-detected transport failure -- never spuriously on a healthy connection), which clears resync_pending and discards any stalled partial multi-page accumulation (frees and zeroes the accumulator, not just the boolean), bounding the worst case to at most one reconnect cycle instead of forever. Verified: round-2 reviewer independently reintroduced the round-1 entry-guard bug, rebuilt, and confirmed a new structural regression test (test_checking_connection_hello_and_needs_rejoin_precede_resync_gate_in_source, asserting strict source-order) fails while the companion behavioral test (test_checking_connection_survives_permanently_stuck_resync_pending) does NOT catch the reintroduced bug alone -- establishing that placement-sensitive gates need a structural/ordering assertion, not just a behavioral one. Full worker suite green after the fix (xcecd_resync_test 28/28, controller_connector_test 22/22 including the pre-existing test_checking_connection_never_permanently_disables_worker invariant). One advisory, non-blocking follow-up noted but not fixed: resync_pending/query_accum are unsynchronized statics touched from two different OS threads (connector thread vs. a vtaskmgr pool thread) -- a narrow theoretical race on xcecd_resync_reset()'s free() vs. a concurrent read, mirroring needs_rejoin's own accepted unsynchronized-static risk profile but riskier since it frees a heap pointer. ## Verification _Not recorded (migrated from buglog.json)._ ## References - Related: bug-1749, bug-1748