lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(libxce): bug-1077 ctrlstatus crash from shared reply-union type confusion

Browse source 
ctx->reply.u is a single untagged union shared across opcodes (debugmsg /
get(dataset_t*) / jobid / clusterconf / ctrlstatus alias the same 8 bytes).
Commit 61e1391 added speculative free-guards
`if (ctx->reply.u.ctrlstatus) free_CtrlStatus_ptr(...)` (and the clusterconf
twin). A non-NULL slot does NOT imply a CtrlStatus*: a failover-aborted
reslist/joblist leaves ctx->reply.u.get == &getreply (a dead STACK pointer)
because xce_get_with_condition/xce_resval only cleared the slot on the success
path, not the xce_get_req<0 / cbuf_get<0 early returns. The next ctrlstatus
reply (worker thread) freed that foreign pointer as a CtrlStatus -> ASan
SEGV / stack-use-after-return in free_CtrlStatus (controller.capnp.c:11638).

Fix (root cause, both layers):
- Revert the unsafe free-guards in xce_handle_reply: MOVE_PTR overwrite only,
  never free/deref an untagged union slot by type.
- Clear ctx->reply.u.get = NULL on ALL exit paths of xce_get_with_condition
  and xce_resval via a single `goto done` cleanup (forget-proof; the bug was
  one forgotten failover path). Also drops the now-unused controller.capnp.h.
- Defense-in-depth: handle_get_reply drops a reply when the slot is NULL.

Regression: expose xce_handle_reply and add
test_handle_reply_ctrlstatus_does_not_free_foreign_slot, which poisons the
shared slot with a foreign heap value and proves the ctrlstatus reply handler
does not free it (RED reproduced the exact free_CtrlStatus_ptr double-free;
GREEN after fix). xcelib_test 244/0, controller_unit_test 166/0, all binaries
build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSxwb2Xt5F84ATgiAZnH8V
This commit is contained in:
Rongsong Shen 2026-06-24 09:44:43 +00:00
parent 61e1391cd0
commit 8da8fd03dd
7 changed files with 943 additions and 767 deletions
Download patch file Download diff file Expand all files Collapse all files

19
src/lib/xceapi.c
View file

@ -106,16 +106,16 @@ int xce_get_with_condition(xce_ctx_t* ctx, xce_get_t* get, void*** data)
}
memset(&getreply, 0x0, sizeof(getreply));
ctx->reply.u.get = &getreply;
rc = xce_get_req(ctx->ep, get);
if (rc < 0) {
return rc;
goto done;
}
rc = cbuf_get(&(ctx->wbuf), &status);
if (rc < 0) {
return rc;
goto done;
}
if (status >= 0) {
@ -125,6 +125,12 @@ int xce_get_with_condition(xce_ctx_t* ctx, xce_get_t* get, void*** data)
rc = status;
}
done:
/* Always clear the destination — including on the xce_get_req/cbuf_get
* failover error paths — so the single, untagged reply union never retains
* the address of this now-returning stack `getreply`. A later ctrlstatus
* reply would otherwise free that dead-stack pointer as a CtrlStatus*
* (bug-1077, ASan stack-use-after-return). */
ctx->reply.u.get = NULL;
return rc;
@ -189,12 +195,12 @@ int xce_resval(xce_ctx_t* ctx, char* host, char* res, xce_resval_t*** vals)
ctx->reply.u.get = &getreply;
rc = xce_get_req(ctx->ep, &get);
if (rc < 0) {
return rc;
goto done;
}
rc = cbuf_get(&(ctx->wbuf), &status);
if (rc < 0) {
return rc;
goto done;
}
if (status >= 0) {
@ -204,6 +210,9 @@ int xce_resval(xce_ctx_t* ctx, char* host, char* res, xce_resval_t*** vals)
rc = status;
}
done:
/* Clear on every path (incl. the failover error returns) so the shared
* reply union never keeps this stack `getreply` address — bug-1077. */
ctx->reply.u.get = NULL;
return rc;