test(worker-rejoin-resync): task 7.4 - large job-count pagination correctness

Add test_large_job_count_pagination_correctness, proving both
OPCode_jobStateQuery and OPCode_jobStateReport genuinely span many
wire pages (1042 jobs, 11 pages each direction, including a partial
final page) with zero jobs dropped/duplicated/truncated - exact-count
assertions throughout (page continuity, no-duplicate bitmap, exact WAL
delta, exact hook-fire count), extending task 6.2's 2-page proof by an
order of magnitude. Test-only, no production changes.

Logs bug-1755: the underlying query-pagination mechanism this test
proves correct is never exercised in production - worker_send_resync_
query() hardcodes pktlimit=0, so the controller's real query send
always produces one unbounded page regardless of job count, asymmetric
with the report direction's real XCECD_RESYNC_REPORT_PKTLIMIT bound.
No hard transport-layer size cap exists, so this isn't a demonstrated
failure mode, but it contradicts design.md's explicit "both paginated"
claim in this feature's own headline mechanism. Adds task 7.5 to close
it with a small, well-scoped fix before the feature-level gate, rather
than leaving a silently-false design claim in the completed feature.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y4M5MJM2zK6nU43da267Ku
This commit is contained in:
Rongsong Shen 2026-09-02 00:48:15 +00:00
parent 165fc1586a
commit 613b80e991
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
3 changed files with 715 additions and 1 deletions

View file

@ -20582,6 +20582,33 @@
],
"occurrences": 1,
"last_seen": "2026-09-01T13:00:00Z"
},
{
"id": "bug-1755",
"timestamp": "2026-09-02T00:30:00Z",
"error_message": "Discovered while building worker-rejoin-resync task 7.4's large job-count pagination test (test_large_job_count_pagination_correctness, test_worker_resync.c): the controller's real production OPCode_jobStateQuery send site, worker_send_resync_query() (src/controller/worker.c:730), calls xce_job_state_query_req(ep, c->myhost, &query, NULL, NULL, 0) with a hardcoded pktlimit of 0. xce_job_state_query_req()'s own pktlimit<=0 convention (request.c:1822-1824, \"if (pktlimit <= 0) { pktlimit = nrows; }\") means this ALWAYS sends the entire query as exactly ONE wire page, however many non-terminal jobs the controller queries a worker about -- there is no bound at all. This is asymmetric with the worker's own real report-send site (controller_connector.c's handle_jobstatequery -> XCECD_RESYNC_REPORT_PKTLIMIT == 100), which DOES genuinely cap and paginate OPCode_jobStateReport in production. The query-direction pagination mechanism (xce_job_state_query_req()'s own do/while page loop, task 1.4) is fully implemented and correct -- task 7.4's own test calls it directly with an explicit page size and proves it genuinely spans many wire pages with no drops/duplicates -- but it is effectively dead code in production today because its only call site never supplies a bound.",
"file": "src/controller/worker.c",
"root_cause": "worker_send_resync_query() was written to pass pktlimit=0 to xce_job_state_query_req(), which the callee treats as \"unbounded / exactly one page\" rather than \"use a real per-page cap.\" No ClusterConfig field or fixed constant analogous to XCECD_RESYNC_REPORT_PKTLIMIT was ever wired in for the query direction, unlike the report direction. Requirement 8.1 ('regardless of how many non-terminal jobs... without... failing solely because of the number of jobs involved') is still technically satisfied -- an unbounded single message doesn't fail merely due to job count, and no hard message-size cap was found elsewhere in the transport/capnp layer that would make a very large single-page query actually fail or truncate. But design.md's own stated intent (both opcodes described as 'paginated'; the Testing Strategy's Performance/Load line about the exchange genuinely spanning multiple pages) is not actually exercised by any real caller for the query direction -- only the report direction is genuinely bounded/paginated in production as shipped.",
"fix": "NOT FIXED -- out of scope for worker-rejoin-resync task 7.4 (a test-only pagination-correctness validation task, correctly declined to patch a production defect/gap it incidentally surfaced). Recommend a dedicated future task: either wire a real config-driven or fixed page cap into worker_send_resync_query()'s xce_job_state_query_req() call (mirroring XCECD_RESYNC_REPORT_PKTLIMIT, or reusing one of the two existing ClusterConfig fields this feature already added -- workerResyncWindowMs/workerResyncTimeoutMs are not a natural fit, so a new field would likely be needed), or explicitly document that the query direction is deliberately left unbounded by design (e.g. because a worker-side job count large enough to matter is not expected in practice). Until then, a worker rejoin with a very large non-terminal job count will always receive its OPCode_jobStateQuery as one very large single message, not multiple bounded pages, even though the receiving/decoding machinery on both sides already correctly supports true multi-page queries.",
"tags": [
"worker-rejoin-resync",
"task-7.4",
"worker-c",
"worker_send_resync_query",
"xce_job_state_query_req",
"requirement-8.1",
"requirement-8.3",
"pagination",
"design-reality-mismatch",
"discovered-not-fixed",
"asymmetric-with-report-direction"
],
"related_bugs": [
"bug-1746",
"bug-1747"
],
"occurrences": 1,
"last_seen": "2026-09-02T00:30:00Z"
}
]
}