lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(xce-test-cli-ux): wrap every blocking RPC with the interrupt adapter

Browse source 
Routes all 26 blocking xcecli_* call sites across ~24 commands through
xcetest_run_interruptible(), wires xcetest_set_active_connection() at
execute_line()'s dispatch point (active_ctx->ctx, read fresh per
command), and fixes main()'s REPL loop to distinguish a real EOF from
an EINTR-interrupted readline() so Ctrl-C at an empty prompt discards
the line instead of exiting the session.

Live-discovered and fixed bug-1794: fill_job_alloc() never initializes
xce_job_alloc_t's nres_shared/allocs_shared fields -- harmless as stack
garbage before this task, genuine heap garbage once moved into a
heap-allocated ctx block (a necessary consequence of the adapter's
heap-only precondition). Fixed by switching all ~26 call-context
allocations from malloc to calloc uniformly. Also fixed an argv-aliasing
hazard: cmd_hello/cmd_jobckpt's -m values aliased optarg, which
execute_line() frees immediately after return including on the
abandoned path -- now strdup'd into the heap block.

Also closed a safety-critical gap this task's own design introduced
(bug-1795): before this task the REPL was fully synchronous, so
leave_ctx() (called by up/quit) could never run while an RPC was
outstanding. An abandoned worker can now persist past command
completion, so leave_ctx() tearing down the connection could
use-after-free it. Fixed via xcetest_wait_connection_idle(), which
blocks on the same per-connection semaphore the adapter already
tracks, then releases it, proving no worker is active before teardown.
Proven with a deterministic white-box RED->GREEN test, since the race
itself is too timing-dependent for a black-box test to reliably force.

1361/1361 across the full controller suite (+2), 0 regressions.

Task 3.2 of .kiro/specs/xce-test-cli-ux/tasks.md.
Requirements: 1.1, 1.2, 1.3, 1.4, 1.5

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
This commit is contained in:
Rongsong Shen 2026-09-09 20:58:18 +00:00
parent f152c59e75
commit 45120af2cc
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
7 changed files with 1887 additions and 70 deletions
Download patch file Download diff file Expand all files Collapse all files

38
src/controller/testsuite/interruptcall.c
View file

@ -314,3 +314,41 @@ int xcetest_run_interruptible(xcetest_blocking_fn fn, void* ctx)
xcetest_sleep_poll_interval();
}
}
/* See interruptcall.h for the full contract. Reuses xcetest_conn_lock_for()
* -- the SAME lazy-lookup/create used by xcetest_run_interruptible() above
* -- rather than a parallel registry, so this genuinely observes the same
* per-connection semaphore any in-flight or abandoned worker on `conn_key`
* is holding. */
void xcetest_wait_connection_idle(void* conn_key)
{
sem_t* lock;
int rc;
lock = xcetest_conn_lock_for(conn_key);
if (!lock) {
/* Allocation failure in the lazy-create path: this key has never
* been registered, so by definition no worker could be holding a
* lock for it via this registry. Nothing to wait for. */
return;
}
/* Plain BLOCKING wait, deliberately not the interruptible
* sem_trywait()-poll loop used above -- see interruptcall.h's
* "CONTRACT" section on this function. A signal delivered while
* blocked here can legitimately interrupt sem_wait() with EINTR even
* though nothing else ever posts spuriously; retry rather than treat
* that as "acquired" or bail out early, since either would defeat
* this function's entire purpose (proving the wait genuinely
* completed before the caller proceeds to tear the connection down). */
do {
rc = sem_wait(lock);
} while ((rc != 0) && (errno == EINTR));
assert(rc == 0); /* NDEBUG caveat: see xcetest_worker_main()'s comment. */
/* Immediately release: the semantic is "prove idle right now," not
* "hold the connection locked." */
rc = sem_post(lock);
(void)rc;
assert(rc == 0);
}

58
src/controller/testsuite/interruptcall.h
View file

@ -249,6 +249,64 @@ void xcetest_set_active_connection(void* conn_key);
*/
int xcetest_run_interruptible(xcetest_blocking_fn fn, void* ctx);
/*
* xcetest_wait_connection_idle (xce-test-cli-ux task 3.2 remediation round
* 1, bug-1795) -- blocks the calling thread until no worker thread
* (running or abandoned-but-still-running) is active on `conn_key`, per
* this module's own "at most one worker thread per connection" invariant,
* then returns immediately.
*
* WHY THIS EXISTS: xcetest_run_interruptible()'s per-connection lock
* registry above enforces that invariant BETWEEN calls on the same
* connection -- a second call on a connection an abandoned worker still
* holds waits (interruptibly) for that worker to finish. It does NOT by
* itself protect connection TEARDOWN: test.c's leave_ctx() (called by
* `up`/`quit`) tears down the connection object out from under an
* abandoned-but-still-running worker with no wait/join of its own,
* because leave_ctx() never calls xcetest_run_interruptible() -- it is
* connection teardown, not a new RPC. xcecli_wakeup() (called by
* leave_ctx() just before this function, best-effort) nudges that worker
* to unwind faster than its own ~10s deadline, but nothing forced
* leave_ctx() to actually wait for it -- a genuine, exploitable
* use-after-free window (any abandoned worker still reading/writing the
* connection object while leave_ctx()'s xcecli_stop()/xcecli_ctx_fini()
* frees it). This function closes that gap by reusing the SAME
* per-connection semaphore registry xcetest_run_interruptible() already
* builds and maintains, rather than inventing a second mechanism.
*
* CONTRACT: looks up (or lazily creates, via the exact same internal
* lookup xcetest_run_interruptible() itself uses) the sem_t for
* `conn_key`, does a PLAIN BLOCKING sem_wait() on it (deliberately NOT
* the interruptible sem_trywait()-poll variant used inside
* xcetest_run_interruptible() -- this is a narrow, safety-critical wait
* where correctness matters more than Ctrl-C responsiveness; it should
* resolve quickly in the ordinary case since the caller's own
* xcecli_wakeup() call already nudges the worker to finish sooner), then
* immediately sem_post()s it back before returning. It does NOT hold the
* semaphore locked on return -- the semantic this function provides is
* "prove that, at this instant, no worker is active on this connection,"
* not "hold this connection locked for some ongoing operation of the
* caller's own." A caller that needs the latter must acquire and hold the
* lock itself (which only xcetest_run_interruptible() currently does).
*
* A transient EINTR from sem_wait() itself (a signal delivered while
* blocked, distinct from `errno`-based break_out signal state) is
* retried internally rather than surfaced -- this function's whole
* purpose is to make the wait unconditional, not skippable.
*
* Safe to call with any `conn_key` previously passed to
* xcetest_set_active_connection() (or, for that matter, any key never
* seen before -- the lazy-create path simply creates a fresh, immediately
* uncontended semaphore for it and returns right away). Must be called
* from the same single REPL thread this module's other entry points are
* documented to run on (see this header's "CONCURRENCY OF THE REGISTRY
* ITSELF" note) -- it is not itself safe to call concurrently from
* multiple threads, though the semaphore it waits on is of course safe to
* be concurrently sem_post()'d by a worker thread while this function
* waits on it.
*/
void xcetest_wait_connection_idle(void* conn_key);
#ifdef XCE_TESTING
/*
* Test-only instrumentation, NOT part of the production API and never

1230
src/controller/testsuite/test.c
View file

File diff suppressed because it is too large Load diff

107
src/controller/testsuite/test_interruptcall.c
View file

@ -464,6 +464,112 @@ static void test_interrupted_while_waiting_for_lock_never_spawns_worker(void)
wait_for_abandoned_free_count(1);
}
/* -- (iv) xcetest_wait_connection_idle() (task 3.2 remediation round 1,
* bug-1795) -- deterministic proof of the exact mechanism leave_ctx() now
* relies on to close the connection-teardown use-after-free.
*
* test_test_cli.c's black-box test_interrupt_then_immediate_teardown_no_uaf
* exercises the SAME fix end-to-end through a real xce_test subprocess, but
* the actual race window there (between an abandoned worker being woken by
* xcecli_wakeup() and leave_ctx()'s teardown proceeding) is sub-millisecond
* and timing-dependent -- it did not reliably reproduce as a failing test
* even with the fix disabled in this environment (matching the original
* round-1 review's own observation that a clean ASAN run is evidence the
* race didn't fire on THAT run's timing, not evidence the gap doesn't
* exist). This test instead proves xcetest_wait_connection_idle() itself,
* in isolation, with a WIDE, deterministic timing margin (a 700ms abandoned
* worker, not a sub-millisecond one) -- a direct, reliable RED (this
* function returning early would fail it) / GREEN (the shipped
* implementation passes it) demonstration of the invariant leave_ctx()
* depends on. */
#define WCI_SLOW_SLEEP_MS 700
#define WCI_MIN_WAIT_MS 400 /* comfortably below the ~700ms - \
* SIGNAL_DELAY_MS remaining sleep once the \
* call is abandoned, comfortably above \
* scheduling noise if \
* xcetest_wait_connection_idle() did NOT \
* actually wait (that failure mode returns \
* in ~0ms). */
static int wci_slow_fn(void* arg)
{
(void)arg;
sleep_ms(WCI_SLOW_SLEEP_MS);
return SLOW_FN_RC;
}
static void test_wait_connection_idle_blocks_until_abandoned_worker_finishes(
void)
{
static int conn_e; /* private fake connection key for this test */
pthread_t signaler;
struct timespec wait_start;
long waited_ms;
int rc2;
g_test_break_out = 0;
atomic_store(&xcetest_test_abandoned_free_count, 0);
xcetest_install_sigint(test_sigint_handler);
xcetest_set_active_connection(&conn_e);
TEST_ASSERT_EQUAL_INT(
0, pthread_create(&signaler, NULL, send_sigint_after_delay, NULL));
int rc = xcetest_run_interruptible(wci_slow_fn, NULL);
TEST_ASSERT_EQUAL_INT(-EINTR, rc);
TEST_ASSERT_EQUAL_INT(0, pthread_join(signaler, NULL));
/* At this instant the abandoned worker is still running wci_slow_fn()
* -- it has NOT yet freed its call block or released conn_e's lock
* (interruptcall.c's abandoned path). This is exactly the production
* race leave_ctx() closes: xcetest_wait_connection_idle() below must
* genuinely block until that worker finishes, not return immediately. */
TEST_ASSERT_EQUAL_INT_MESSAGE(
0, atomic_load(&xcetest_test_abandoned_free_count),
"the abandoned worker must still be running at this point for "
"this test to actually exercise the wait -- if this fires, the "
"worker finished implausibly fast and the timing margin above "
"needs to grow, not shrink");
clock_gettime(CLOCK_MONOTONIC, &wait_start);
xcetest_wait_connection_idle(&conn_e);
waited_ms = elapsed_ms_since(&wait_start);
TEST_ASSERT_TRUE_MESSAGE(
waited_ms >= WCI_MIN_WAIT_MS,
"xcetest_wait_connection_idle() returned too quickly -- it must "
"genuinely block until the abandoned worker finishes, not return "
"immediately (this is precisely the bug-1795 use-after-free gap: "
"a caller proceeding to tear down the connection before this "
"point would race the still-running worker)");
TEST_ASSERT_EQUAL_INT_MESSAGE(
1, atomic_load(&xcetest_test_abandoned_free_count),
"the abandoned worker's cleanup must have genuinely completed by "
"the time xcetest_wait_connection_idle() returns");
/* The semaphore must be left available (posted back), not held --
* proven by a normal subsequent call on the SAME connection key
* completing without waiting at all. Reset the fake interrupt flag
* first -- xcetest_interrupted() is a pure, non-clearing read (see
* xcetest_interrupt.h's CONTRACT), so the flag the earlier SIGINT set
* is still "on" here and would otherwise spuriously abandon this call
* too, same reasoning as every other multi-call scenario in this
* file. */
g_test_break_out = 0;
clock_gettime(CLOCK_MONOTONIC, &wait_start);
rc2 = xcetest_run_interruptible(fast_fn, NULL);
waited_ms = elapsed_ms_since(&wait_start);
TEST_ASSERT_EQUAL_INT(NORMAL_FN_RC, rc2);
TEST_ASSERT_TRUE_MESSAGE(
waited_ms < WCI_MIN_WAIT_MS,
"xcetest_wait_connection_idle() must release the semaphore back "
"immediately after proving idleness, not hold it -- a subsequent "
"call on the same connection should not itself have to wait");
}
void test_interruptcall(void)
{
UnitySetTestFile(__FILE__);
@ -472,4 +578,5 @@ void test_interruptcall(void)
RUN_TEST(test_same_connection_key_serializes_overlapping_calls);
RUN_TEST(test_different_connection_keys_run_independently);
RUN_TEST(test_interrupted_while_waiting_for_lock_never_spawns_worker);
RUN_TEST(test_wait_connection_idle_blocks_until_abandoned_worker_finishes);
}

466
src/controller/testsuite/test_test_cli.c
View file

@ -151,8 +151,10 @@
* CONCERNS in this task's status report if a future task instead needs
* cross-file reuse.
*/
#include <arpa/inet.h>
#include <errno.h>
#include <limits.h>
#include <netinet/in.h>
#include <poll.h>
#include <pty.h>
#include <signal.h>
@ -160,8 +162,10 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/wait.h>
#include <termios.h>
#include <time.h>
#include <unistd.h>
#include <unity/unity.h>
@ -1434,6 +1438,465 @@ static void test_walcompare_unrecognized_flag_rejected_with_usage(void)
clitest_buf_free(&err);
}
/* ------------------------------------------------------------------ */
/* Interruptible Call Adapter end-to-end (task 3.2, Requirement 1) */
/* ------------------------------------------------------------------ */
/* clitest_start_blackhole_listener - starts a "black hole" TCP peer on
* 127.0.0.1:`port`: it accepts every incoming connection but never reads
* or writes a single byte on any of them, so a client that connects and
* sends a request gets no reply, ever. This is the vehicle for exercising
* Requirement 1's "blocked awaiting a cluster RPC reply" scenario
* deterministically: xce_test's own client library still completes a
* fast, successful TCP connect() (something IS listening and accepting),
* so the actual multi-second block this test needs to interrupt is
* exactly the one interruptcall.c's own header describes -- the blocking
* `xceapi.c` reply wait (`cbuf_timedget`'s fixed 10s deadline), not the
* OS-level connect() path. Returns the listener's pid (to be reaped via
* clitest_stop_blackhole_listener()) on success, or -1 on failure. */
static pid_t clitest_start_blackhole_listener(int port)
{
int lfd;
struct sockaddr_in addr;
int opt = 1;
pid_t pid;
lfd = socket(AF_INET, SOCK_STREAM, 0);
if (lfd < 0) {
return -1;
}
setsockopt(lfd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
addr.sin_port = htons((uint16_t)port);
if (bind(lfd, (struct sockaddr*)&addr, sizeof(addr)) != 0) {
close(lfd);
return -1;
}
if (listen(lfd, 16) != 0) {
close(lfd);
return -1;
}
pid = fork();
if (pid < 0) {
close(lfd);
return -1;
}
if (pid == 0) {
/* Child: accept forever, never read/write any accepted socket --
* a deliberate silent peer, so the parent xce_test's blocking RPC
* wait never completes on its own. */
for (;;) {
int cfd = accept(lfd, NULL, NULL);
if (cfd < 0) {
if (errno == EINTR) {
continue;
}
_exit(0);
}
/* Leave `cfd` open and untouched; just keep accepting more. */
}
}
close(lfd); /* the parent process only needs the child running */
return pid;
}
static void clitest_stop_blackhole_listener(pid_t pid)
{
if (pid > 0) {
kill(pid, SIGKILL);
waitpid(pid, NULL, 0);
}
}
static long clitest_elapsed_ms(const struct timespec* start)
{
struct timespec now;
clock_gettime(CLOCK_MONOTONIC, &now);
return ((long)(now.tv_sec - start->tv_sec) * 1000L)
+ ((now.tv_nsec - start->tv_nsec) / 1000000L);
}
/* test_interrupt_abandons_blocked_rpc_well_before_timeout -- Requirement
* 1.1/1.3/1.5, this task's Required Test Scenario 1+2: a command blocked
* on a real RPC reply (against the black-hole peer above, so the ~10s
* cbuf_timedget deadline is genuinely reachable) is interrupted by a real
* SIGINT; control must return to the prompt in a small fraction of that
* 10s deadline, the command must be reported as failed (Requirement 1.3),
* and the SAME connection/context must accept a normal, non-RPC follow-up
* command immediately afterward (proving the context/connection were left
* intact, Requirement 1.5), all without the session exiting. */
static void test_interrupt_abandons_blocked_rpc_well_before_timeout(void)
{
char xce_test_path[PATH_MAX];
char* const argv[] = { "xce_test", "--norc", NULL };
clitest_session_t sess;
clitest_buf_t out;
clitest_buf_t err;
int exit_code = -999;
int port = 19990 + (int)(getpid() % 1000);
char masters_cmd[64];
pid_t blackhole;
struct timespec interrupt_sent_at;
blackhole = clitest_start_blackhole_listener(port);
TEST_ASSERT_TRUE_MESSAGE(blackhole > 0,
"must be able to start the black-hole TCP "
"listener this test blocks against");
TEST_ASSERT_EQUAL_INT_MESSAGE(
0, resolve_xce_test_path(xce_test_path, sizeof(xce_test_path)),
"must be able to resolve the sibling xce_test binary's path");
clitest_buf_init(&out);
clitest_buf_init(&err);
TEST_ASSERT_EQUAL_INT_MESSAGE(
0, clitest_spawn(&sess, xce_test_path, argv),
"clitest_spawn must launch xce_test behind a real pty");
TEST_ASSERT_TRUE_MESSAGE(
clitest_wait_for_text(&sess, &out, &err, "xce>", false, 5000),
"expected the initial 'xce>' interactive prompt on stdout");
snprintf(masters_cmd, sizeof(masters_cmd), "set masters 127.0.0.1:%d",
port);
TEST_ASSERT_EQUAL_INT(0, clitest_send_line(&sess, masters_cmd));
TEST_ASSERT_EQUAL_INT(0, clitest_send_line(&sess, "client"));
TEST_ASSERT_TRUE_MESSAGE(
clitest_wait_for_text(&sess, &out, &err, "xce/client>", false, 5000),
"entering the client context must succeed even though its "
"connection target is a silent black hole (connect setup does not "
"itself block)");
/* `hello` issues exactly one blocking xcecli_hello() RPC (interrupt-
* call-adapter-wrapped, task 3.2) -- against a peer that never
* replies, this call would otherwise block for ~10s. */
TEST_ASSERT_EQUAL_INT(0, clitest_send_line(&sess, "hello"));
/* Give the command time to actually reach the blocking RPC wait
* (connect + send are fast; this margin is generous relative to
* both) before sending the interrupt, so this test genuinely
* exercises the "interrupt a wait already in flight" path rather
* than racing dispatch itself. */
usleep(300000); /* 300ms */
/* Reset the capture buffers before sending the interrupt: "xce/client>"
* already appears once in `out` from when this session first entered
* the client context above, so waiting for that SAME substring against
* the accumulated buffer would trivially "succeed" on the very first
* poll regardless of whether a NEW prompt is genuinely printed after
* the interrupt -- discarding prior output makes the wait below (and
* the elapsed-time measurement it gates) observe a real, fresh event. */
clitest_buf_free(&out);
clitest_buf_free(&err);
clitest_buf_init(&out);
clitest_buf_init(&err);
clock_gettime(CLOCK_MONOTONIC, &interrupt_sent_at);
TEST_ASSERT_EQUAL_INT(0, clitest_send_signal(&sess, SIGINT));
TEST_ASSERT_TRUE_MESSAGE(
clitest_wait_for_text(&sess, &out, &err, "xce/client>", false, 3000),
"the client-context prompt must reappear after the interrupt, well "
"before the RPC's own ~10s deadline");
TEST_ASSERT_LESS_THAN_MESSAGE(
2000L, clitest_elapsed_ms(&interrupt_sent_at),
"control must return to the prompt in a small fraction of the RPC's "
"~10s deadline (observed well under 2s in practice, given the "
"adapter's documented 50ms poll interval)");
/* Requirement 1.5: the SAME connection/context must accept a normal
* follow-up command immediately -- `echo` makes no RPC call at all,
* so an immediate, correct reply here proves the session (and its
* client context) is genuinely alive and responsive, not merely
* appearing to be so. */
TEST_ASSERT_EQUAL_INT(
0, clitest_send_line(&sess, "echo cli_after_interrupt_marker"));
TEST_ASSERT_TRUE_MESSAGE(
clitest_wait_for_text(&sess, &out, &err, "cli_after_interrupt_marker",
false, 3000),
"a follow-up command on the same connection/context must execute "
"normally right after the interrupt");
TEST_ASSERT_EQUAL_INT(0, clitest_send_line(&sess, "quit"));
TEST_ASSERT_EQUAL_INT_MESSAGE(
0, clitest_wait_for_exit(&sess, &out, &err, 5000, &exit_code),
"xce_test did not exit within the harness's safety timeout after "
"'quit'");
TEST_ASSERT_EQUAL_INT_MESSAGE(
1, exit_code,
"the interrupted 'hello' must be recorded as a failed command "
"(Requirement 1.3), matching this session's overall exit code");
clitest_kill_and_reap(&sess);
clitest_close(&sess);
clitest_stop_blackhole_listener(blackhole);
clitest_buf_free(&out);
clitest_buf_free(&err);
}
/* test_interrupt_at_empty_prompt_discards_line_without_exiting --
* Requirement 1.2/1.4, this task's Required Test Scenario 3: a Ctrl-C with
* nothing in flight (a partially-typed, unsubmitted line at the prompt)
* must discard that line and redisplay the prompt, never exit the session
* as a side effect -- proven by sending a real, valid command right after
* and confirming it executes normally. */
static void test_interrupt_at_empty_prompt_discards_line_without_exiting(void)
{
char xce_test_path[PATH_MAX];
char* const argv[] = { "xce_test", "--norc", NULL };
clitest_session_t sess;
clitest_buf_t out;
clitest_buf_t err;
int exit_code = -999;
TEST_ASSERT_EQUAL_INT_MESSAGE(
0, resolve_xce_test_path(xce_test_path, sizeof(xce_test_path)),
"must be able to resolve the sibling xce_test binary's path");
clitest_buf_init(&out);
clitest_buf_init(&err);
/* disable_echo=false (clitest_spawn_ex, not plain clitest_spawn): this
* test needs libedit's REAL interactive line-editing state (in
* particular, that a partial, uncommitted line is genuinely discarded
* from its edit buffer on the EINTR path) -- clitest_spawn()'s default
* echo-off session makes THIS project's linked libedit fall back to a
* silent/non-interactive input mode (task 2.3's own documented finding,
* see clitest_spawn_ex()'s header comment) that does NOT discard a
* pending partial line across an interrupted read(), so a truly
* interactive echo-on session is required to observe the correct
* (and, empirically confirmed against a genuine terminal-like pty,
* actually-happening) discard behavior here. */
TEST_ASSERT_EQUAL_INT_MESSAGE(
0, clitest_spawn_ex(&sess, xce_test_path, argv, false),
"clitest_spawn_ex must launch xce_test behind a real pty with echo "
"on");
TEST_ASSERT_TRUE_MESSAGE(
clitest_wait_for_text(&sess, &out, &err, "xce>", false, 5000),
"expected the initial 'xce>' interactive prompt on stdout");
/* A partial, unsubmitted line -- nothing in flight, no command has
* been dispatched yet. */
TEST_ASSERT_EQUAL_INT(0, clitest_send_partial(&sess, "ech"));
usleep(300000); /* let libedit's readline() actually be blocked on it */
/* Reset the capture buffers before sending the interrupt: "xce>"
* already appears once in `out` from the session's initial prompt
* above, so waiting for that SAME substring against the accumulated
* buffer would trivially "succeed" on the very first poll regardless
* of whether a NEW prompt is genuinely printed after the interrupt. */
clitest_buf_free(&out);
clitest_buf_free(&err);
clitest_buf_init(&out);
clitest_buf_init(&err);
TEST_ASSERT_EQUAL_INT(0, clitest_send_signal(&sess, SIGINT));
/* Wait for the REDISPLAYED prompt (main()'s new break_out-then-printf
* path) before sending anything else -- this is what makes the next
* step race-free: only once "xce>" has genuinely reappeared do we know
* the interrupted readline() call has already returned and a BRAND
* NEW readline() call, with a fresh/empty edit buffer, is active.
* Sending the next line immediately after clitest_send_signal()
* (without waiting for this) would race the child's own signal
* delivery/handling latency -- if that next line's bytes arrive before
* the child has actually re-entered its blocking read(), they get
* queued as more input to the STILL-PENDING original read() instead of
* starting a fresh one, undetectably from this harness's side (this
* was empirically confirmed while building this test: it manifests as
* a corrupted "echecho ..." command whose raw, still-echoed-back typed
* text nonetheless contains the search marker, making the naive
* text-search assertion below pass despite the underlying command
* actually failing -- caught instead by the exit-code assertion at the
* end, which is what motivated this explicit wait). */
TEST_ASSERT_TRUE_MESSAGE(
clitest_wait_for_text(&sess, &out, &err, "xce>", false, 3000),
"the prompt must be redisplayed after Ctrl-C at an empty/partial "
"prompt, before this test sends anything further");
/* Requirement 1.4: never exit as a side effect of a single Ctrl-C.
* Send a real, valid command and confirm it executes normally --
* this is the semantic, redraw-independent proof this session is
* still alive and its discarded partial line did not leak into the
* next command's input (e.g. becoming "echecho ..."). */
TEST_ASSERT_EQUAL_INT(
0, clitest_send_line(&sess, "echo cli_still_alive_marker"));
TEST_ASSERT_TRUE_MESSAGE(
clitest_wait_for_text(&sess, &out, &err, "cli_still_alive_marker",
false, 3000),
"a real command sent right after Ctrl-C at an empty/partial prompt "
"must execute normally, proving the session did not exit");
TEST_ASSERT_EQUAL_INT(0, clitest_send_line(&sess, "quit"));
TEST_ASSERT_EQUAL_INT_MESSAGE(
0, clitest_wait_for_exit(&sess, &out, &err, 5000, &exit_code),
"xce_test did not exit within the harness's safety timeout after "
"'quit'");
TEST_ASSERT_EQUAL_INT_MESSAGE(
0, exit_code,
"a session with no genuinely failing command must still exit 0");
clitest_kill_and_reap(&sess);
clitest_close(&sess);
clitest_buf_free(&out);
clitest_buf_free(&err);
}
/* test_interrupt_then_immediate_teardown_no_uaf -- xce-test-cli-ux task 3.2
* remediation round 1 regression guard (bug-1795). Interrupts a blocked RPC
* exactly like test_interrupt_abandons_blocked_rpc_well_before_timeout
* above, but where that test deliberately WAITS for the interrupted prompt
* to redisplay before sending anything else (so ITS OWN elapsed-time
* measurement is race-free), THIS test does the opposite on purpose: it
* sends ".." (the "go up a context" command, bound to cmd_up) IMMEDIATELY
* after the SIGINT, with no settling wait at all -- racing leave_ctx()'s
* connection teardown (invoked via cmd_up) against
* whatever worker thread was abandoned by the interrupt, which may still
* be running for up to its own ~10s deadline (interruptcall.c's
* xcetest_run_interruptible()/xcecli_wakeup()). This is exactly the crux
* the round-1 review flagged as a genuine use-after-free: before this
* remediation, leave_ctx() called xcecli_stop()/xcecli_ctx_fini() (which
* unconditionally frees the connection's api_ctx_t, src/lib/client.c)
* right after a merely best-effort xcecli_wakeup() nudge, with nothing
* forcing it to actually wait for that abandoned worker to finish
* touching the connection first. The fix under test is
* xcetest_wait_connection_idle() (interruptcall.c/.h), which leave_ctx()
* now calls, right before teardown, to block until the SAME per-connection
* semaphore registry xcetest_run_interruptible() itself uses proves no
* worker is active on this connection.
*
* The actual proof this test relies on is running under ASAN (this
* project's `asan` build profile) with a completely clean stderr -- a
* genuine heap-use-after-free is caught by ASAN the instant it happens,
* independent of anything this test does afterward, so the exit-code and
* "no AddressSanitizer text anywhere in stderr" assertions below are what
* actually catch a regression; a merely-non-hanging, prompt-reappeared
* session alone would not have caught the original bug on a timing-lucky
* run. */
static void test_interrupt_then_immediate_teardown_no_uaf(void)
{
char xce_test_path[PATH_MAX];
char* const argv[] = { "xce_test", "--norc", NULL };
clitest_session_t sess;
clitest_buf_t out;
clitest_buf_t err;
int exit_code = -999;
int port = 19990 + (int)(getpid() % 1000) + 1;
char masters_cmd[64];
pid_t blackhole;
blackhole = clitest_start_blackhole_listener(port);
TEST_ASSERT_TRUE_MESSAGE(blackhole > 0,
"must be able to start the black-hole TCP "
"listener this test blocks against");
TEST_ASSERT_EQUAL_INT_MESSAGE(
0, resolve_xce_test_path(xce_test_path, sizeof(xce_test_path)),
"must be able to resolve the sibling xce_test binary's path");
clitest_buf_init(&out);
clitest_buf_init(&err);
TEST_ASSERT_EQUAL_INT_MESSAGE(
0, clitest_spawn(&sess, xce_test_path, argv),
"clitest_spawn must launch xce_test behind a real pty");
TEST_ASSERT_TRUE_MESSAGE(
clitest_wait_for_text(&sess, &out, &err, "xce>", false, 5000),
"expected the initial 'xce>' interactive prompt on stdout");
snprintf(masters_cmd, sizeof(masters_cmd), "set masters 127.0.0.1:%d",
port);
TEST_ASSERT_EQUAL_INT(0, clitest_send_line(&sess, masters_cmd));
TEST_ASSERT_EQUAL_INT(0, clitest_send_line(&sess, "client"));
TEST_ASSERT_TRUE_MESSAGE(
clitest_wait_for_text(&sess, &out, &err, "xce/client>", false, 5000),
"entering the client context must succeed even though its "
"connection target is a silent black hole");
/* Same as the sibling test above: issue the blocking `hello` RPC
* against the black hole, then give it a moment to actually reach the
* blocking wait before interrupting -- this margin is only about
* getting the WORKER thread genuinely blocked inside the RPC, not
* about settling anything after the interrupt (see below, which is
* the entire point of this test). */
TEST_ASSERT_EQUAL_INT(0, clitest_send_line(&sess, "hello"));
usleep(300000); /* 300ms */
/* Reset capture buffers before the interrupt, same rationale as the
* sibling test above ("xce/client>" already appears once from
* entering the context). */
clitest_buf_free(&out);
clitest_buf_free(&err);
clitest_buf_init(&out);
clitest_buf_init(&err);
TEST_ASSERT_EQUAL_INT(0, clitest_send_signal(&sess, SIGINT));
/* THE POINT OF THIS TEST: send the "go up a context" command ("..",
* bound to cmd_up -- see test.c's command tables) IMMEDIATELY, with NO
* settling wait for the interrupted prompt to reappear first (unlike
* the sibling test above). This is deliberately the adversarial timing
* for the use-after-free the review found -- leave_ctx() (invoked
* here via cmd_up) must genuinely BLOCK, via
* xcetest_wait_connection_idle(), until the abandoned worker thread
* has actually finished with this connection before
* xcecli_stop()/xcecli_ctx_fini() free it, rather than racing it. */
TEST_ASSERT_EQUAL_INT(0, clitest_send_line(&sess, ".."));
TEST_ASSERT_TRUE_MESSAGE(
clitest_wait_for_text(&sess, &out, &err, "xce>", false, 5000),
"the top-level 'xce>' prompt must reappear after '..' tears down "
"the interrupted client connection, with no crash in between");
/* A real follow-up command on the brand-new top-level context proves
* the whole process (not merely the pty) is still alive and
* undamaged after the teardown race above. */
TEST_ASSERT_EQUAL_INT(
0, clitest_send_line(&sess, "echo cli_after_teardown_marker"));
TEST_ASSERT_TRUE_MESSAGE(
clitest_wait_for_text(&sess, &out, &err, "cli_after_teardown_marker",
false, 3000),
"a follow-up command after the interrupt-then-immediate-teardown "
"race must execute normally");
TEST_ASSERT_EQUAL_INT(0, clitest_send_line(&sess, "quit"));
TEST_ASSERT_EQUAL_INT_MESSAGE(
0, clitest_wait_for_exit(&sess, &out, &err, 5000, &exit_code),
"xce_test did not exit within the harness's safety timeout after "
"'quit'");
TEST_ASSERT_TRUE_MESSAGE(
exit_code >= 0,
"the session must exit normally, never via a fatal signal (e.g. "
"ASAN's SIGABRT/SIGSEGV on a detected heap-use-after-free)");
TEST_ASSERT_NULL_MESSAGE(
strstr(err.data, "AddressSanitizer"),
"no AddressSanitizer diagnostic (e.g. heap-use-after-free) may "
"appear anywhere in stderr across this whole session");
clitest_kill_and_reap(&sess);
clitest_close(&sess);
clitest_stop_blackhole_listener(blackhole);
clitest_buf_free(&out);
clitest_buf_free(&err);
}
void test_test_cli(void)
{
UnitySetTestFile(__FILE__);
@ -1446,4 +1909,7 @@ void test_test_cli(void)
RUN_TEST(test_qctrl_flag_forms_short_and_long_both_accepted);
RUN_TEST(test_migrated_commands_sample_reject_unrecognized_flag);
RUN_TEST(test_walcompare_unrecognized_flag_rejected_with_usage);
RUN_TEST(test_interrupt_abandons_blocked_rpc_well_before_timeout);
RUN_TEST(test_interrupt_at_empty_prompt_discards_line_without_exiting);
RUN_TEST(test_interrupt_then_immediate_teardown_no_uaf);
}