--- id: "bug-1755" status: "fixed" status_updated: "2026-09-24" status_note: "" component: "controller" severity: "unknown" timestamp: "2026-09-02T00:30:00Z" last_seen: "2026-09-19T09:00:00Z" occurrences: 1 files: ["src/controller/worker.c"] tags: ["worker-rejoin-resync", "task-7.4", "worker-c", "worker_send_resync_query", "xce_job_state_query_req", "requirement-8.1", "requirement-8.3", "pagination", "design-reality-mismatch", "asymmetric-with-report-direction", "task-7.5"] related: ["bug-1746", "bug-1747"] legacy: true --- # bug-1755: Discovered while building worker-rejoin-resync task 7.4's large job-count pagination test… ## Description Discovered while building worker-rejoin-resync task 7.4's large job-count pagination test (test_large_job_count_pagination_correctness, test_worker_resync.c): the controller's real production OPCode_jobStateQuery send site, worker_send_resync_query() (src/controller/worker.c:730), calls xce_job_state_query_req(ep, c->myhost, &query, NULL, NULL, 0) with a hardcoded pktlimit of 0. xce_job_state_query_req()'s own pktlimit<=0 convention (request.c:1822-1824, "if (pktlimit <= 0) { pktlimit = nrows; }") means this ALWAYS sends the entire query as exactly ONE wire page, however many non-terminal jobs the controller queries a worker about -- there is no bound at all. This is asymmetric with the worker's own real report-send site (controller_connector.c's handle_jobstatequery -> XCECD_RESYNC_REPORT_PKTLIMIT == 100), which DOES genuinely cap and paginate OPCode_jobStateReport in production. The query-direction pagination mechanism (xce_job_state_query_req()'s own do/while page loop, task 1.4) is fully implemented and correct -- task 7.4's own test calls it directly with an explicit page size and proves it genuinely spans many wire pages with no drops/duplicates -- but it is effectively dead code in production today because its only call site never supplies a bound. ## Environment _Not recorded (migrated from buglog.json)._ ## Reproduction _Not recorded (migrated from buglog.json)._ ## Root cause worker_send_resync_query() was written to pass pktlimit=0 to xce_job_state_query_req(), which the callee treats as "unbounded / exactly one page" rather than "use a real per-page cap." No ClusterConfig field or fixed constant analogous to XCECD_RESYNC_REPORT_PKTLIMIT was ever wired in for the query direction, unlike the report direction. Requirement 8.1 ('regardless of how many non-terminal jobs... without... failing solely because of the number of jobs involved') is still technically satisfied -- an unbounded single message doesn't fail merely due to job count, and no hard message-size cap was found elsewhere in the transport/capnp layer that would make a very large single-page query actually fail or truncate. But design.md's own stated intent (both opcodes described as 'paginated'; the Testing Strategy's Performance/Load line about the exchange genuinely spanning multiple pages) is not actually exercised by any real caller for the query direction -- only the report direction is genuinely bounded/paginated in production as shipped. ## Fix FIXED (2026-09-01, worker-rejoin-resync task 7.5). worker_send_resync_query() (src/controller/worker.c:730, now ~745) no longer hardcodes pktlimit=0 in its xce_job_state_query_req() call -- it now passes a new, named page-size cap, XCE_RESYNC_QUERY_PKTLIMIT (worker.c, value 100), mirroring the worker's own already-wired XCECD_RESYNC_REPORT_PKTLIMIT (controller_connector.c, also 100) for the report direction. Same value used for both directions: a JobStateQuery entry (bare jobid) is smaller on the wire than a JobStateReport entry (jobid + status + exitCode + finishTimeMs + queried), so the query direction safely supports at least the same page size, with no reason found to invent a smaller one. This closes the query-direction gap task 7.4 found and correctly declined to fix (a test-only task) -- the query direction now genuinely, observably paginates in production, not just at the xce_job_state_query_req() mechanism level task 7.4's own test proved. Verified with a new RED->GREEN test exercising the REAL production call site directly (not a xce_job_state_query_req() bypass): test_worker_send_resync_query_paginates_across_multiple_wire_pages (test_worker_join.c) arms a reconciliation round over 250 non-terminal jobs (> the 100-job page cap, not a multiple of it) and asserts worker_send_resync_query() itself splits the query into exactly 3 wire pages (100+100+50), each with correct pos/count/n_total, every jobid appearing exactly once, no gaps or overlaps. Confirmed genuinely RED beforehand by reverting the fix and rerunning: failed with 'no single page may exceed the configured page cap' (a single collapsed page, matching the pre-fix hardcoded-0 behavior). Confirmed the fix does not regress small-job-count behavior: both existing task 6.1 tests (test_worker_send_resync_query_sends_armed_round, 2 jobs; test_worker_send_resync_query_sends_nothing_when_no_round, 0 jobs) and task 7.4's own test_large_job_count_pagination_correctness (which exercises xce_job_state_query_req() directly with an explicit page-size override, still bypassing the wrapper on purpose -- kept unmodified, it targets the mechanism, not the wrapper) all still pass unchanged. Full controller test suite (meson test -C src/xce.build 'controller test'): 6/6 OK, zero regressions. ## Verification _Not recorded (migrated from buglog.json)._ ## References - Related: bug-1746, bug-1747