fix(xce-test-e2e-primitives): jobsub clean-ack requires echoed JobId == submitted id (bug-786, surfaced by live soak 7.2)

This commit is contained in:
Rongsong Shen 2026-06-19 16:57:47 +00:00
parent 6315368b01
commit 860997634a

View file

@ -1521,13 +1521,25 @@ int cmd_jobsub(int argc, char* argv[])
FREEUP(submit.start_info->exdata); FREEUP(submit.start_info->exdata);
FREEUP(submit.start_info); FREEUP(submit.start_info);
/* Report success only on a clean ack with a well-formed JobId (bug-786, /* Report success only on a clean ack that echoes back the EXACT JobId we
* Req 7.1/7.2/7.3). xce_jobsubmit() writes *jobid only when the reply * submitted (bug-786, Req 7.1/7.2/7.3). xce_jobsubmit() writes *jobid only
* status is >= 0; XCECLI_FAILOVER_CALL surfaces a non-clean outcome as a * when the reply status is >= 0; XCECLI_FAILOVER_CALL surfaces a non-clean
* negative rc (e.g. -E_XCE_NOT_READY on no-primary budget exhaustion, * outcome as a negative rc (e.g. -E_XCE_NOT_READY on no-primary budget
* -E_XCE_REDIRECT / -E_XCE_NOMASTER on a standby, -ETIMEDOUT / -ENOTCONN on * exhaustion, -E_XCE_REDIRECT / -E_XCE_NOMASTER on a standby, -ETIMEDOUT /
* transport loss). A clean ack is therefore rc >= 0 AND a non-zero jobid; * -ENOTCONN on transport loss).
* anything else is a typed failure and must NOT print "ok" or set $jobid. */ *
* The `jobid != 0` gate alone is NOT sufficient: on a failover, libxce's
* handle_sub_reply() returns -1 on a non-jobid reply but LEAVES the stale
* previous value in ctx->reply.u.jobid, and xceapi.c copies it out whenever
* status >= 0 — so a rejected submit can surface as rc >= 0 with a NON-ZERO
* stale-garbage jobid (the live soak captured `63-15--1-1015808[32751]`
* while the controller logged "handle_jobsubmit fail to lock job" and never
* committed it). We therefore require the returned jobid to EQUAL the JobId
* we generated locally and submitted: on a genuine clean ack the controller
* commits and echoes back exactly submit->jobid (handle_jobsubmit replies
* reply.data.jobid = job_submit() = base_id = submit->jobid; verified
* round-trip). `jobid != jobsubmit.jobid` covers BOTH the never-written
* (zero) case AND the stale non-zero garbage case in one uint64 compare. */
if (rc < 0) { if (rc < 0) {
char ebuf[128]; char ebuf[128];
@ -1556,11 +1568,21 @@ int cmd_jobsub(int argc, char* argv[])
return rc; return rc;
} }
if (jobid == 0) { if (jobid != jobsubmit.jobid) {
/* Non-negative rc but no JobId was written: not a clean ack. Never /* Non-negative rc but the returned JobId does not match the one we
* print "ok" with an uninitialized/garbage id (Req 7.3). */ * submitted: not a clean ack. This rejects BOTH the never-written
* (jobid == 0) case AND the stale-garbage non-zero case left in the
* libxce reply union on a failover-rejected submit. Never print "ok"
* with an uninitialized/garbage id (Req 7.3). */
char wantStr[128];
char gotStr[128];
fmt_jobid(wantStr, sizeof(wantStr) - 1, jobsubmit.jobid);
fmt_jobid(gotStr, sizeof(gotStr) - 1, jobid);
fprintf(stderr, fprintf(stderr,
"jobsubmit failed: no clean ack (no JobId returned)\n"); "jobsubmit failed: no clean ack (controller did not confirm "
"the submitted JobId: wanted %s, got %s)\n",
wantStr, gotStr);
return -E_XCE_INTERNAL; return -E_XCE_INTERNAL;
} }