lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(controller): string resvals own their bytes -- rejoin use-after-free killed new primary (bug-1851)

Browse source 
update_val_res() memcpy'd the xce_val_t into w->host.resvals, so the
"type"/"model" string resources aliased the interned xstrings owned by
w->host.attrs. rejoin_worker() drops the old attrs strings, then
apply_join_state() -> resync_arm_round() resolves the previous join's
still-outstanding round, firing the worker hooks before
add_hostattrs_resvals() re-seeds resvals; the apiserver hook's
collect_resval() strdup()ed the freed hostmodel. Live: a newly promoted
primary died ~8s after election during an HA soak.

A string resval now stores a private strdup (taken before the old copy is
freed), and the owning arrays free their entries: host.resvals gets
kill_resval (replacing the FIXME NULL killer) and c->sharedinst's
resval_obj_killer (previously a no-op) frees the string and the value --
both also leaked every xce_val_t before. Also closes the same alias for
worker-reported string resvals and config values.

Regression test: test_rejoin_worker_string_resvals_survive_outstanding_round
(RED reproduced the live ASAN stack; GREEN after). meson test 109/109.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-09-24 08:41:55 +00:00
parent ac033f4783
commit ca4db13a79
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
5 changed files with 199 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

17
.wolf/buglog.json
View file

@ -25170,6 +25170,23 @@
"ha",
"scheduler"
]
},
{
"id": "bug-1851",
"timestamp": "2026-09-24T07:40:00Z",
"error_message": "Live sequential HA soak (run-ha-soak.sh --sequential 6, round 5): newly-promoted primary ctrl-3 (term 10) died ~8s after promotion with ASAN heap-use-after-free READ in strdup <- collect_resval (dataex.c:71) <- apiserver_worker_change (apiserver.c:450) <- apiserver_worker_hook_fun <- xce_run_hooks <- persist_notify_worker_status_updated (persist.c:7997) <- resync_resolve_round (resync.c:418) <- resync_arm_round (resync.c:315) <- apply_join_state (worker.c:351) <- rejoin_worker (worker.c:444) <- handle_worker_join. Freed by the same thread in rejoin_worker (worker.c:434, xstring_dereference of the old hostmodel). Soak surfaced it as 'FAIL: round 5: no new primary elected after kill (no serving primary at term>9)' -- the new primary crashed rather than failing to elect.",
"file": "src/controller/worker.c",
"root_cause": "update_val_res() (worker.c:~99) memcpy()s the xce_val_t into w->host.resvals, so the 'type'/'model' string resources seeded by add_hostattrs_resvals() ALIAS the interned xstrings owned by w->host.attrs->hosttype/hostmodel (no reference taken). rejoin_worker() dereferences (frees, at refcount 0 -- even when the new value is the same string, since it dereferences before makeString) the old hosttype/hostmodel, then apply_join_state() calls resync_arm_round() BEFORE add_hostattrs_resvals() re-seeds resvals. When the resync round resolves synchronously inside arm (resync_resolve_round -> persist_notify_worker_status_updated -> worker hooks), apiserver_worker_change -> collect_resval strdup()s the dangling svalue. Latent since the alias was introduced; made reachable by worker-rejoin-resync's synchronous resolve+notify path on rejoin (bug-1841 area).",
"fix": "FIXED (2026-09-24, TDD). update_val_res() (worker.c) now stores a private strdup() for Value_u_svalue (copy taken before freeing the previous one, so val may alias v) and frees an old string when a slot changes type; the owning arrays now free their entries: w->host.resvals gets kill_resval (lib/wobj.c, replacing the FIXME NULL killer) and c->sharedinst's resval_obj_killer (cluster.c, previously a no-op) frees svalue + the xce_val_t -- both previously also leaked every xce_val_t. Also closes the same alias hazard for worker-reported string resvals (update_resval_list aliased the decoded request) and config values (mark_host_res). Regression test test_worker_join.c::test_rejoin_worker_string_resvals_survive_outstanding_round: first join arms a round (non-terminal job on host), second rejoin_worker with new attrs resolves it and fires a registered XCE_CONTROLLER_HOOK_WORKER hook that reads the type/model resvals -- RED reproduced the live stack exactly (heap-use-after-free, freed by rejoin_worker worker.c:432 xstring_dereference); GREEN after. Full meson test 109/109 OK. Note: get_resval() still hands out a borrowed svalue pointer (opget.c), now into the resval's own copy -- same lifetime contract as before, not widened.",
"tags": [
"controller",
"asan",
"use-after-free",
"worker-rejoin-resync",
"ha-soak",
"resvals",
"xstring"
]
}
]
}