feat(xce-test-cli-ux): add per-connection lock to interruptible-call adapter
A registry of per-connection locks, acquired (interruptibly) before spawning a worker and released by whichever side finishes last, so an abandoned call's worker is never running concurrently with a new call on the same connection. Uses a POSIX semaphore (sem_t), not a mutex: review round 1 found the initial pthread_mutex_t implementation relied on behavior POSIX classifies as undefined for a non-owning-thread unlock (not just error-checking mutex types) -- this design's worker thread legitimately releases a lock the caller thread acquired, which sem_t supports with no ownership concept at all. bug-1791. Design.md synced to reflect the semaphore choice. 1285/1285 controller suite passing. Task 1.5 of .kiro/specs/xce-test-cli-ux/tasks.md. Requirements: 1.1, 1.5 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019jaMf9QEwpugH2W9KAuc7f
This commit is contained in:
parent
9a06047da2
commit
204f81d134
6 changed files with 1226 additions and 60 deletions
540
.kiro/specs/xce-test-cli-ux/design.md
Normal file
540
.kiro/specs/xce-test-cli-ux/design.md
Normal file
|
|
@ -0,0 +1,540 @@
|
|||
# Design Document: xce-test-cli-ux
|
||||
|
||||
## Overview
|
||||
|
||||
**Purpose**: This feature makes `xce_test`'s command-line surface predictable and fail-loud for the engineers who drive controller/worker/scheduler/HA testing through it — consistent flag validation across every command, a Ctrl-C that actually cancels a stuck RPC, structured output for scripted callers, complete per-command help, a safety gate on destructive actions, and in-session script composition via `source`. It also fixes two known display/default bugs (bug-1711, bug-1557) found during the CLI-UX audit that motivated this spec.
|
||||
|
||||
**Users**: SDETs and engineers running interactive soak/E2E sessions against live or degraded HA clusters, and script authors piping `xce_test` non-interactively in automated test harnesses.
|
||||
|
||||
**Impact**: Adds a small set of cross-cutting internal modules (flag parsing/validation, an interruptible-RPC wrapper, a confirmation gate, a JSON list formatter) to the existing single-file `xce_test` REPL client (`src/controller/testsuite/test.c`, ~4550 lines), migrates ~17 commands' flag parsing to a shared pattern, and corrects two existing defects — without changing wire protocol, RPC semantics, or any existing short flag's meaning.
|
||||
|
||||
### Goals
|
||||
- Every flag-accepting command uniformly rejects an unrecognized/malformed flag and offers a long-form equivalent, named consistently for the same concept across commands.
|
||||
- Ctrl-C reliably cancels a blocked RPC wait or the current input line — never the whole session, never silently doing nothing.
|
||||
- The five destructive commands (`jobkill`, `shutdown`, `qctrl -i/-c`, `hostctrl -c`, `upgradeconfig`) require confirmation, safely bypassable per-command or per-session for scripts.
|
||||
- `help <cmd>` is complete (description, full flag docs, example, context/related-command notes) and consistent for every command.
|
||||
- `workerlist`/`joblist`/`reslist` support a JSON output mode with the same fields as their tables, unabbreviated.
|
||||
- `source <file>` composes reusable command sequences, safely (cycle-detected).
|
||||
- bug-1711's display defects and bug-1557's stale `join` default are corrected.
|
||||
|
||||
### Non-Goals
|
||||
- No change to controller/worker/scheduler wire protocol or RPC semantics.
|
||||
- No `xce_test` → `xce_harness` rename.
|
||||
- No full curses/TUI mode, no color/theming.
|
||||
- No new `.xce_testrc` auto-executed startup-file feature (already shipped by the separate `xce-test-rc-file` spec).
|
||||
- No renaming or reassignment of any existing short flag's meaning.
|
||||
- No change to the auto-variable/exit-code-verdict/shell-escape mechanisms already shipped by `xce-test-e2e-primitives`.
|
||||
|
||||
## Boundary Commitments
|
||||
|
||||
### This Spec Owns
|
||||
- `xce_test`'s own CLI surface: flag parsing, validation, and typo-suggestion for every command that accepts flags (Flag Parsing Layer).
|
||||
- Client-side Ctrl-C cancellation of a blocked RPC wait (Interruptible Call Adapter) — cancellation is local only; it does not cancel the in-flight request server-side. Owns serializing any two calls that share a connection (including a new call issued right after a prior one was abandoned) so a connection's single-outstanding-reply state is never touched concurrently.
|
||||
- The destructive-command confirmation gate, its five-command enumeration, its bypass flag, and its session-scoped toggle (Confirmation Gate).
|
||||
- JSON output for `workerlist`/`joblist`/`reslist` (JSON List Formatter).
|
||||
- Command-name tab completion scoped to the active context (Tab Completion).
|
||||
- `help <cmd>` content and generation, including the active-context-only not-found behavior (Help Renderer).
|
||||
- The new `source <file>` command, its cycle detection, and its explicit missing-file error (Source Command).
|
||||
- The specific display/default corrections enumerated in Requirements 11 and 12 (bug-1711, bug-1557).
|
||||
- Process-level `-h`/`--help`/`--version` (Process Entry).
|
||||
|
||||
### Out of Boundary
|
||||
- `libxce`'s blocking RPC primitives (`cbuf_timedget`, `xceapi.c`'s `xce_*` functions) — not modified. Requirement 1 is achieved entirely client-side, inside `test.c`/testsuite.
|
||||
- Controller/worker/scheduler wire protocol and RPC semantics — unaffected by any part of this design.
|
||||
- `rcfile.c`'s own path-resolution and startup-time missing-file behavior — untouched; `source` reuses `rcfile_load` as-is and layers its own missing-file/cycle behavior on top, in the new command, not in `rcfile.c`.
|
||||
- Any existing short flag's accepted values, default, or meaning — never reassigned (only a long-form alias is added alongside).
|
||||
- The exit-code/auto-variable verdict machinery and shell-escape mechanism from `xce-test-e2e-primitives` — read-only dependency, not modified.
|
||||
- Authentication/connection setup to the controller/worker/scheduler — unchanged.
|
||||
- The controller daemon's HA/persistence/consensus invariants (`src/controller/CLAUDE.md`'s TLA+ mapping, election safety, commit durability, etc.) — checked and found not applicable: `xce_test` is a client binary linked against `libxce`, not the controller daemon process, and no part of this design touches daemon code, WAL, or replication state.
|
||||
|
||||
### Allowed Dependencies
|
||||
- cJSON (via the already-linked `struct2json_dep`) — for the JSON List Formatter. No new external dependency.
|
||||
- libedit's completion API (already linked) — for Tab Completion.
|
||||
- POSIX threads (already transitively linked via `libxce`) — for the Interruptible Call Adapter.
|
||||
- `rcfile_load()` (existing, unmodified) — called directly by the Source Command.
|
||||
- `cmd_set`'s existing settings mechanism — extended (not replaced) by the Confirmation Gate's toggle.
|
||||
|
||||
### Revalidation Triggers
|
||||
- Any change to `libxce`'s blocking-call deadline (`cbuf_deadline_10s`) changes the worst-case lifetime of an abandoned worker thread this design assumes.
|
||||
- Any change to `rcfile_load`'s signature or missing-file semantics invalidates the Source Command's reuse contract.
|
||||
- A new `cmd_*` command added after this spec ships must adopt the `flagspec_t` pattern to stay covered by Requirements 2/6/8/10 — otherwise it silently falls outside these guarantees.
|
||||
- Any future decision to reassign an existing short flag's meaning (explicitly deferred, not this spec) would invalidate the `flagspec_t` tables' short/long/concept mappings built here.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Existing Architecture Analysis
|
||||
|
||||
- **Flag parsing** is three uncoordinated patterns today: a plain-`getopt()` loop (~16 commands, each with its own hand-written `switch`/`default`), positional-only argument checking (offline/diagnostic commands), and one fully hand-rolled long-flag string-compare loop (`cmd_walcompare`). No shared helper exists for parsing, rejection, or usage rendering — this disconnect between parsing code and the hand-authored `USAGE_*` string table is the direct root cause of bug-1711's drift.
|
||||
- **Interrupt handling**: `SIGINT` → `sigbreak()` sets a static `break_out` flag, read in exactly one place (`cmd_hello`'s repeat loop). Every blocking `xce_*` RPC call waits on `pthread_cond_timedwait` with a fixed 10-second deadline (`cbuf_deadline_10s`/`cbuf_timedget`, in `libxce`) — a wait a signal-flag cannot interrupt.
|
||||
- **REPL input**: libedit (`readline()`/`add_history()` only), no completion or key-binding customization registered.
|
||||
- **Session settings**: a single global `settings[]` array shared (not actually per-context) by all four `command_ctx_t` instances; `cmd_set`'s `-e`/`+e`/`continue-on-error` toggle is the existing pattern for a session-scoped boolean.
|
||||
- **Serialization**: cJSON is already linked (via `struct2json_dep`) and used narrowly for job exdata blobs; Cap'n Proto is also linked but exclusively for wire-protocol/checkpoint-blob decoding, not general CLI output.
|
||||
- **Script loading**: `rcfile_load(path, runner)` is a generic, already-reusable "read a file line by line, execute each via a callback" primitive with no cycle protection and a silent-success policy on a missing file (correct for its one existing caller, the startup rc file).
|
||||
|
||||
### Architecture Pattern & Boundary Map
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
User[Operator or Script]
|
||||
Readline[Readline Loop]
|
||||
Dispatch[Command Dispatcher]
|
||||
FlagSpec[Flag Parsing Layer]
|
||||
Confirm[Confirmation Gate]
|
||||
CmdFunc[Command Handler]
|
||||
Interrupt[Interruptible Call Adapter]
|
||||
Json[JSON List Formatter]
|
||||
Source[Source Command]
|
||||
RcFile[Rcfile Loader]
|
||||
LibXce[Libxce Rpc Client]
|
||||
Help[Help Renderer]
|
||||
|
||||
User --> Readline
|
||||
Readline --> Dispatch
|
||||
Dispatch --> FlagSpec
|
||||
FlagSpec --> Confirm
|
||||
Confirm --> CmdFunc
|
||||
CmdFunc --> Interrupt
|
||||
Interrupt --> LibXce
|
||||
CmdFunc --> Json
|
||||
Dispatch --> Source
|
||||
Source --> RcFile
|
||||
RcFile --> Dispatch
|
||||
FlagSpec --> Help
|
||||
Dispatch --> Help
|
||||
```
|
||||
|
||||
**Architecture Integration**:
|
||||
- **Selected pattern**: a layered CLI pipeline — Readline → Dispatcher → Flag Parsing → (Confirmation Gate, destructive commands only) → Command Handler → Interruptible Call Adapter → `libxce`.
|
||||
- **Domain/feature boundaries**: Flag Parsing is one cohesive layer shared by every flag-accepting command; the Interruptible Call Adapter is a second cross-cutting layer used only where a blocking RPC occurs; the Confirmation Gate is a third, used only by the five destructive commands.
|
||||
- **Existing patterns preserved**: hand-authored `USAGE_*` prose for description/example/related-command text; the single-global session-settings pattern (`cmd_set`); `rcfile.c`'s file-loading primitive, reused unmodified.
|
||||
- **New components rationale**: each addresses one specific root cause found in discovery — flag drift, a non-interruptible wait, no cycle guard, no typo-suggestion utility.
|
||||
- **Steering compliance**: TDD (a `test_<module>.c` per new module, red-green-refactor), `gnu17`/Meson conventions, negative-errno-style (`errnox.h`) error propagation throughout.
|
||||
|
||||
### Technology Stack
|
||||
|
||||
| Layer | Choice / Version | Role in Feature | Notes |
|
||||
|-------|------------------|-----------------|-------|
|
||||
| CLI / testsuite | C (gnu17), Meson | Language/build for all new and modified files | Matches existing `test.c` |
|
||||
| CLI parsing | `getopt_long` (glibc) | Long-form flag parsing, replacing raw `getopt()` | Migrates ~17 commands |
|
||||
| Structured output | cJSON (already linked via `struct2json_dep`) | Requirement 5 JSON mode | Adopted, no new dependency |
|
||||
| Line editing | libedit (already linked) | Tab completion (Requirement 7) | `rl_attempted_completion_function`-style hook; exact API to confirm at implementation start |
|
||||
| Concurrency | POSIX threads (already transitively linked via `libxce`) | Interruptible RPC wrapper (Requirement 1) | Detached worker + CAS ownership handoff |
|
||||
| Test framework | Unity (project-wide) | New `test_*.c` files | `#ifdef XCE_TESTING` for internal-only symbols |
|
||||
|
||||
## File Structure Plan
|
||||
|
||||
`src/controller/testsuite/` is a flat directory; new modules follow its existing one-helper-module-per-file convention (`rcfile.c/h`, `waldump.c/h`, etc.).
|
||||
|
||||
### New Files
|
||||
- `flagspec.c` / `flagspec.h` — flag-descriptor table format, `getopt_long` option-table builder, shared rejection + typo-suggestion helper, help-flag-block renderer. (Requirements 2, 6, 8.2, 10)
|
||||
- `interruptcall.c` / `interruptcall.h` — `xcetest_run_interruptible()`: detached-worker-thread wrapper with CAS-based heap-ownership handoff, plus a per-connection lock registry that serializes overlapping calls (including a call issued right after a prior one was abandoned) on the same connection. (Requirement 1)
|
||||
- `confirm.c` / `confirm.h` — destructive-command confirmation gate, session-scoped toggle, stderr-routed prompt, EINTR-as-decline. (Requirement 9)
|
||||
- `listfmt_json.c` / `listfmt_json.h` — shared cJSON array/object helpers for the three listing commands' JSON mode. (Requirement 5)
|
||||
- `test_flagspec.c`, `test_interruptcall.c`, `test_confirm.c`, `test_listfmt_json.c` — Unity unit tests, one per new module, following the existing `test_rcfile.c`-style convention.
|
||||
- `test_test_cli.c` — new black-box/subprocess (pty-driven) test suite for `xce_test`'s own CLI surface (`-h`/`--version`, `help <cmd>` content, tab completion, command-not-found scoping, `source` cycle/missing-file cases) — this surface has zero prior coverage.
|
||||
|
||||
### Modified Files
|
||||
- `src/controller/testsuite/test.c` — exposes a small `xcetest_interrupted()` accessor over the existing `break_out` flag for `interruptcall.c` to poll; replaces the existing `signal(SIGINT, sigbreak)` installation in `main()` with an explicit `sigaction()` call with `SA_RESTART` cleared, so `confirm.c`'s confirmation-prompt read reliably observes `EINTR`; every flag-parsing `cmd_*` migrated from raw `getopt()`/hand-rolled parsing to `flagspec_t` + `getopt_long`; the ~19 blocking `xce_*` call sites wrapped via `xcetest_run_interruptible()`; `cmd_set` extended with the confirmation-toggle branch (mirroring its existing `-e`/`+e` style); `cmd_source` added and registered in all four `command_t` tables; `xcetest_complete`/`xcetest_complete_generator` registered against libedit in `main()`; `cmd_help` extended to call `flagspec_render_help()`; `USAGE_*` constants split into a generated flag-list portion and retained hand-authored prose; `worker_status_str`, `cmd_help`'s not-found message, `cmd_runtasks`, `cmd_reslist`, `cmd_joblist`, `cmd_getconf` corrected per Requirement 11; `cmd_join_worker`'s default `maxmem` corrected per Requirement 12; the `argp` `doc` string in `main()` extended to describe the in-session `help` command per Requirement 4.3 (4.1/4.2's `-h`/`--help`/`--version` exit-without-REPL behavior is already `argp`'s default and needs no new code, only verification via `test_test_cli.c`).
|
||||
- `src/controller/meson.build` — adds the four new `.c` files to `xce_test_src`, and their `test_*.c` counterparts plus `test_test_cli.c` to the appropriate test target(s).
|
||||
|
||||
## System Flows
|
||||
|
||||
### Interruptible RPC Call and Ctrl-C Abandonment
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Op as Operator
|
||||
participant Cmd as CommandHandler
|
||||
participant Adapter as InterruptibleAdapter
|
||||
participant Lock as ConnectionLock
|
||||
participant Worker as WorkerThread
|
||||
participant Rpc as LibxceRpc
|
||||
|
||||
Op->>Cmd: invoke a command with a blocking RPC
|
||||
Cmd->>Adapter: run_interruptible call
|
||||
Adapter->>Lock: acquire lock for the active connection, interruptibly
|
||||
Lock-->>Adapter: acquired
|
||||
Adapter->>Worker: spawn detached thread
|
||||
Worker->>Rpc: blocking xce call, up to ten second deadline
|
||||
Adapter->>Adapter: poll break_out every fifty milliseconds
|
||||
Op-->>Adapter: Ctrl-C sets break_out
|
||||
Adapter->>Adapter: CAS running to abandoned succeeds
|
||||
Adapter-->>Cmd: return EINTR immediately, lock stays held
|
||||
Cmd-->>Op: report command as failed, prompt returns
|
||||
Rpc-->>Worker: reply arrives later
|
||||
Worker->>Worker: CAS running to done fails, worker frees the block
|
||||
Worker->>Lock: release lock
|
||||
```
|
||||
|
||||
A second command issued against the *same* connection after an abandonment blocks on this same lock-acquisition step — itself interruptibly — until the abandoned worker finishes and releases it, bounding the unsafe-overlap window to zero rather than letting two calls share one connection's single-outstanding-reply state concurrently. A command against a *different* connection is unaffected, since each connection gets its own lock.
|
||||
|
||||
### Heap-Block Ownership Handoff
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> Running
|
||||
Running --> DoneUnclaimed: worker finishes, CAS succeeds
|
||||
Running --> Abandoned: caller interrupted, CAS succeeds
|
||||
DoneUnclaimed --> [*]: caller reads rc and frees the block
|
||||
Abandoned --> [*]: worker frees the block after finishing
|
||||
```
|
||||
|
||||
Exactly one atomic compare-and-swap decides who frees the block, regardless of which side "wins" the race — this is what prevents a dangling-stack or double-free bug in the abandonment path (see `research.md`, "Interruptible RPC via detached worker thread" decision). This block-ownership handoff is orthogonal to, and independent of, the per-connection lock shown in the sequence diagram above: the lock protects the *connection*'s exclusive use across the call's full lifetime; the CAS protects the *heap block*'s lifetime. Both are released/freed by whichever side is left holding them once the worker finishes.
|
||||
|
||||
### Destructive Command Confirmation Gate
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Start[Destructive command invoked]
|
||||
CheckBypass{Bypass flag given}
|
||||
CheckToggle{Session toggle set to skip}
|
||||
CheckTty{Input from interactive TTY}
|
||||
Prompt[Prompt operator on stderr]
|
||||
Answer{Operator answers}
|
||||
Run[Execute command]
|
||||
NotRun[Report not run]
|
||||
|
||||
Start --> CheckBypass
|
||||
CheckBypass -- yes --> Run
|
||||
CheckBypass -- no --> CheckToggle
|
||||
CheckToggle -- yes --> Run
|
||||
CheckToggle -- no --> CheckTty
|
||||
CheckTty -- no --> Run
|
||||
CheckTty -- yes --> Prompt
|
||||
Prompt --> Answer
|
||||
Answer -- yes --> Run
|
||||
Answer -- no or ctrl c --> NotRun
|
||||
```
|
||||
|
||||
The TTY check evaluates the top-level session's own stdin (`g_interactive`), not whether the current line originates from a `source`d file (Requirement 13.7) — a sourced destructive command in an otherwise-interactive session still prompts unless the toggle is set.
|
||||
|
||||
## Requirements Traceability
|
||||
|
||||
| Requirement | Summary | Components | Interfaces | Flows |
|
||||
|-------------|---------|------------|------------|-------|
|
||||
| 1.1, 1.2, 1.3, 1.4, 1.5 | Ctrl-C cancels a blocked RPC or the input line, never the session | Interruptible Call Adapter, Command Dispatcher | `xcetest_run_interruptible` | Interrupt sequence + ownership state diagram |
|
||||
| 2.1, 2.2, 2.3, 2.4 | Uniform rejection of unrecognized/malformed flags | Flag Parsing Layer | `flagspec_reject` | - |
|
||||
| 3.1, 3.2, 3.3, 3.4 | stderr/stdout discipline | Command Dispatcher, all `cmd_*` handlers | - | - |
|
||||
| 4.1, 4.2, 4.3 | Process-level `-h`/`--help`/`--version` | Process Entry | `argp` options | - |
|
||||
| 5.1, 5.2, 5.3, 5.4 | JSON output for listing commands | JSON List Formatter | `listfmt_json_*` | - |
|
||||
| 6.1, 6.2, 6.3, 6.4 | Long-form, concept-consistent flags | Flag Parsing Layer | `flagspec_t` table | - |
|
||||
| 7.1, 7.2, 7.3, 7.4 | Command-name tab completion | Tab Completion | `xcetest_complete*` | - |
|
||||
| 8.1, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7 | Detailed, context-scoped help content | Help Renderer, Flag Parsing Layer | `flagspec_render_help`, `cmd_help` | - |
|
||||
| 9.1, 9.2, 9.3, 9.4, 9.5, 9.6, 9.7, 9.8, 9.9 | Confirmation gate for destructive commands | Confirmation Gate | `confirm_destructive`, `cmd_set` toggle | Confirmation flowchart |
|
||||
| 10.1, 10.2, 10.3 | Actionable flag-typo suggestions | Flag Parsing Layer | `flagspec_reject` (suggestion path) | - |
|
||||
| 11.1, 11.2, 11.3, 11.4, 11.5, 11.6, 11.7, 11.8 | bug-1711 display-defect fixes | Help Renderer, and `worker_status_str`/`cmd_runtasks`/`cmd_reslist`/`cmd_joblist`/`cmd_getconf` | - | - |
|
||||
| 12.1, 12.2 | bug-1557 `join` maxmem default fix | `cmd_join_worker`, Help Renderer | - | - |
|
||||
| 13.1, 13.2, 13.3, 13.4, 13.5, 13.6, 13.7 | `source` command | Source Command, `rcfile_load` (reused) | `cmd_source` | Confirmation flowchart (13.7) |
|
||||
|
||||
## Components and Interfaces
|
||||
|
||||
| Component | Domain/Layer | Intent | Req Coverage | Key Dependencies (P0/P1) | Contracts |
|
||||
|-----------|--------------|--------|---------------|--------------------------|-----------|
|
||||
| Flag Parsing Layer | CLI Core | `getopt_long` parsing, rejection, suggestion, help-flag rendering from one descriptor table | 2, 6, 8.2, 10 | `getopt_long` (P0) | Service |
|
||||
| Interruptible Call Adapter | CLI Core | Client-side cancellable wrapper around blocking `xce_*` calls | 1 | pthread (P0), `libxce` `xceapi` (P0) | Service |
|
||||
| Confirmation Gate | CLI Core | Gate + session toggle for destructive commands | 9 | `cmd_set` (P1), Flag Parsing Layer (P1) | Service |
|
||||
| JSON List Formatter | Output | cJSON-based structured output for the three listing commands | 5 | cJSON (P0) | Service |
|
||||
| Tab Completion | REPL | libedit completion hook scoped to the active context | 7 | libedit (P0) | Service |
|
||||
| Help Renderer | REPL | Assembles `help <cmd>` from generated flags + hand-authored prose; scopes not-found to the active context | 8, 11.2, 12.2 | Flag Parsing Layer (P0) | Service |
|
||||
| Source Command | REPL | `source <file>`, cycle detection, reuse of `rcfile_load` | 13 | `rcfile_load` (P0), Confirmation Gate (P1) | Service |
|
||||
| Process Entry | Process | `argp`-based `-h`/`--help`/`--version` | 4 | `argp` (glibc) (P0) | Service |
|
||||
| Bug-1711 Fixes | Output | Text/formatting corrections in existing `cmd_*` bodies | 11 | — | — |
|
||||
| Bug-1557 Fix | Data | `join`'s `maxmem` default correction | 12 | — | — |
|
||||
|
||||
Only components introducing a new boundary get a full detail block below; Process Entry and the two bug-fix rows are simple, localized edits to existing functions and need no further elaboration beyond the summary and File Structure Plan.
|
||||
|
||||
### CLI Core
|
||||
|
||||
#### Flag Parsing Layer
|
||||
|
||||
| Field | Detail |
|
||||
|-------|--------|
|
||||
| Intent | One small library, used by every flag-accepting command, that parses via `getopt_long` from a per-command descriptor table and uniformly rejects/suggests/documents against that same table |
|
||||
| Requirements | 2.1, 2.2, 2.3, 2.4, 6.1, 6.2, 6.3, 6.4, 8.2, 10.1, 10.2, 10.3 |
|
||||
|
||||
**Responsibilities & Constraints**
|
||||
- Owns the `flagspec_t` descriptor format, the shared rejection message format, and the shared help-flag-block format.
|
||||
- Does not own each command's business-logic switch on parsed values — only the parse/reject/describe surface.
|
||||
- Invariant: a `flagspec_t[]` table's long-form name for a repeated concept (host, jobid, command line, resource name) is identical across every command that has it — enforced by table review, not at runtime.
|
||||
|
||||
**Dependencies**
|
||||
- Outbound: `getopt_long` (glibc) — P0
|
||||
- Outbound: a small edit-distance utility introduced by this component itself (not an external dependency) — P0
|
||||
|
||||
**Contracts**: Service [x] / API [ ] / Event [ ] / Batch [ ] / State [ ]
|
||||
|
||||
##### Service Interface
|
||||
```c
|
||||
typedef struct {
|
||||
char short_opt; /* 0 if long-only */
|
||||
const char* long_opt; /* stable, cross-command-consistent name */
|
||||
const char* concept; /* e.g. "host", "jobid", "command-line" */
|
||||
int has_arg; /* no_argument | required_argument */
|
||||
int required; /* 1 if the command fails without it */
|
||||
const char* default_desc; /* human text, NULL if none */
|
||||
} flagspec_t;
|
||||
|
||||
int flagspec_build_opts(const flagspec_t* spec, size_t n,
|
||||
struct option* out_opts, char* out_optstring,
|
||||
size_t out_optstring_cap);
|
||||
|
||||
int flagspec_reject(const char* cmd_name, const flagspec_t* spec, size_t n,
|
||||
const char* bad_flag, const char* usage_text);
|
||||
|
||||
char* flagspec_render_help(const flagspec_t* spec, size_t n);
|
||||
```
|
||||
- Preconditions: `spec` is a stable static array for the process lifetime; the caller sets `opterr = 0` before its `getopt_long` loop so libc's own diagnostic never doubles up with `flagspec_reject`'s.
|
||||
- Postconditions: `flagspec_reject` always returns a negative value and never partially executes the command.
|
||||
- Invariants: the built optstring never begins with `:` — missing-argument detection is `flagspec_reject`'s job, not silent `getopt` behavior.
|
||||
|
||||
**Implementation Notes**
|
||||
- Integration: every `getopt()`-using command and the one hand-rolled long-flag command (`cmd_walcompare`) migrate to `getopt_long` + a static `flagspec_t[]` + a `case '?': return flagspec_reject(...)` arm.
|
||||
- Validation: `test_flagspec.c` covers the suggestion threshold (exactly-one-close-match / no-match / ambiguous), the optstring builder, and the help-render format, independent of any real command.
|
||||
- Risks: existing short-flag collisions (`-m` five ways, `-c` six ways) mean the same short letter maps to a *different* `concept` in different commands' tables — expected, not a defect; Requirement 6.2 only requires the long-form name to be consistent per concept, not that short letters be globally unique.
|
||||
- Note (bug-1704): `cmd_qctrl`'s existing `action` variable has no initializer and no post-loop check that one of `-a`/`-i`/`-o`/`-c` was actually given, unlike its sibling `cmd_hostctrl`, which already initializes to a sentinel and rejects if unset. This migration must apply the same sentinel-and-reject pattern to `cmd_qctrl` — not just for correctness, but because the Confirmation Gate (Requirement 9.5) depends on reliably knowing whether `-i`/`-c` was given, which is unsafe to check against an uninitialized value.
|
||||
|
||||
#### Interruptible Call Adapter
|
||||
|
||||
| Field | Detail |
|
||||
|-------|--------|
|
||||
| Intent | Let a command make a blocking `xce_*` RPC call cancellably from the operator's perspective, without modifying `libxce` |
|
||||
| Requirements | 1.1, 1.2, 1.3, 1.4, 1.5 |
|
||||
|
||||
**Responsibilities & Constraints**
|
||||
- Owns the heap-block ownership protocol (the CAS state machine) and the poll interval.
|
||||
- Owns a per-connection lock registry that serializes any two calls sharing the same connection — including a new call issued after a prior one on that connection was abandoned — so a connection's single-outstanding-reply state is never touched by two threads at once.
|
||||
- Never touches `libxce`/`xceapi.c` internals — the wrapped call is opaque.
|
||||
- Invariant: a heap-allocated call block's lifetime is resolved by exactly one atomic CAS; no code path frees it from both sides.
|
||||
- Invariant: at most one worker thread — running or abandoned-but-still-running — is ever active per connection at a time.
|
||||
|
||||
**Dependencies**
|
||||
- Outbound: pthread create/detach, POSIX semaphore (`sem_t`, one per connection, in an internal registry) — P0 (a mutex was considered and rejected during implementation review: POSIX classifies a non-owning-thread `pthread_mutex_unlock()` as undefined behavior for every mutex type, including the default, and this design's cross-thread release — the worker thread releasing a lock the caller thread acquired — needs a primitive with no ownership concept; see bug-1791)
|
||||
- Outbound: the wrapped `xce_*` call, treated as opaque — P0
|
||||
- Inbound: every command performing a blocking RPC (~19 call sites) — P0
|
||||
|
||||
**Contracts**: Service [x]
|
||||
|
||||
##### Service Interface
|
||||
```c
|
||||
typedef int (*xcetest_blocking_fn)(void* ctx);
|
||||
|
||||
typedef enum { XCT_RUNNING, XCT_DONE_UNCLAIMED, XCT_ABANDONED } xcetest_call_state_t;
|
||||
|
||||
/* Serializes against any other in-flight or abandoned-but-still-running
|
||||
* call that shares the same active connection (keyed internally by the
|
||||
* current command_ctx_t's connection pointer, looked up in a small,
|
||||
* lazily-populated, process-lifetime registry - at most one entry per
|
||||
* context, so a linear scan suffices). Acquiring that per-connection lock
|
||||
* is itself interruptible: if break_out fires while waiting for a PRIOR
|
||||
* call's abandoned worker to release it, this returns -EINTR before ever
|
||||
* spawning fn. */
|
||||
int xcetest_run_interruptible(xcetest_blocking_fn fn, void* ctx);
|
||||
```
|
||||
- Preconditions: `fn` touches only what `ctx` itself heap-owns — never caller-stack memory.
|
||||
- Postconditions: on `-EINTR`, the caller must not touch `ctx` again — ownership has passed to the still-running worker thread. The per-connection lock is released by whichever side (caller or worker) is the last to finish with the call, never left held once both sides are done.
|
||||
- Invariants: exactly one of {caller, worker} frees the call block, decided by which side wins the `RUNNING -> {DONE_UNCLAIMED, ABANDONED}` CAS.
|
||||
|
||||
**Implementation Notes**
|
||||
- Integration: each of the ~19 blocking-RPC call sites gets a small per-site static thunk matching `xcetest_blocking_fn`; none of them need to pass a connection identity explicitly — the per-connection lock key is read internally from the currently active `command_ctx_t`'s connection pointer (exact global/field name to confirm against `test.c` at implementation start, alongside the libedit completion API verification already noted in Tab Completion's Risks). A call site that ever operates on a connection *other than* the currently active one would break this assumption — none is known to today, but this is worth a one-line check per call site during migration.
|
||||
- Integration: the lock registry's lookup/insert step needs no synchronization of its own — a *new* registry entry is only ever created from the single REPL thread that issues commands (detached worker threads never initiate a new top-level call; they only run the `fn` they were given and release the lock/free the block they were handed), so there is no concurrent-insert case to guard against.
|
||||
- Validation: `test_interruptcall.c` uses a fake slow `fn` to exercise both the normal-completion and forced-abandon paths, verifying no double-free and no leak; a second test spawns two overlapping calls against the same fake connection key (one abandoned, one issued immediately after) and asserts they never run concurrently — this is the regression test for the connection-race this component's lock exists to prevent.
|
||||
- Risks: none material remaining — the connection-race risk this component previously left as an unverified follow-up is now closed by the per-connection lock described above.
|
||||
|
||||
#### Confirmation Gate
|
||||
|
||||
| Field | Detail |
|
||||
|-------|--------|
|
||||
| Intent | Gate the five destructive commands behind interactive confirmation, a per-command bypass flag, or a session-scoped toggle |
|
||||
| Requirements | 9.1, 9.2, 9.3, 9.4, 9.5, 9.6, 9.7, 9.8, 9.9 |
|
||||
|
||||
**Responsibilities & Constraints**
|
||||
- Owns the destructive-command enumeration (9.5), the session toggle state, the stderr-routed prompt, and Ctrl-C-as-decline.
|
||||
- Does not own the destructive action itself — only whether it proceeds.
|
||||
|
||||
**Dependencies**
|
||||
- Outbound: `cmd_set` (extended with the toggle branch) — P1
|
||||
- Outbound: Flag Parsing Layer (the per-command `--yes` bypass is itself a `flagspec_t` entry) — P1
|
||||
- Outbound: the process's SIGINT installation (`main()`) — must be installed without `SA_RESTART` for the confirmation read to reliably observe `EINTR` — P0
|
||||
- Inbound: `cmd_jobkill`, `cmd_shutdown`, `cmd_qctrl`, `cmd_hostctrl`, `cmd_upgradeconfig` — P0
|
||||
|
||||
**Contracts**: Service [x]
|
||||
|
||||
##### Service Interface
|
||||
```c
|
||||
int confirm_destructive(const char* action_description);
|
||||
void confirm_set_skip(int skip);
|
||||
int confirm_is_skip_set(void);
|
||||
```
|
||||
- Preconditions: called only at the call sites enumerated by Requirement 9.5 (the enumeration's implementation is authoritative — no other command calls this). The process's SIGINT handler must already be installed without `SA_RESTART` (see Implementation Notes) — this is a one-time startup precondition, not something each call re-checks.
|
||||
- Postconditions: a 0 return means the caller reports the command as "not run," never "failed" (9.2, 9.9).
|
||||
- Invariants: the prompt always goes to stderr (9.8); the TTY check is the top-level session's `g_interactive`, not "is this line from a sourced file" (13.7).
|
||||
|
||||
**Implementation Notes**
|
||||
- Integration: each of the five call sites checks `confirm_is_skip_set() || !g_interactive || <its own --yes flag>` before calling `confirm_destructive`.
|
||||
- Integration: the existing `signal(SIGINT, sigbreak)` installation in `main()` (`test.c:4457`) is replaced with an explicit `sigaction()` call with `SA_RESTART` cleared. This is verified, not assumed: glibc's `signal()` uses BSD semantics, which historically restart an interrupted slow syscall by default — if left as-is, the confirmation prompt's blocking read could silently retry instead of returning `EINTR`, and Requirement 9.9 would never fire. This is a one-time, global change made once at startup; it does not affect the Interruptible Call Adapter, which never relies on `EINTR` (it polls `break_out` on a timer instead).
|
||||
- Validation: `test_confirm.c` simulates non-TTY stdin for auto-bypass, and a piped decline/EOF/simulated-EINTR answer for the decline paths; `test_test_cli.c` adds a real pty-driven case that sends an actual SIGINT to the process while a confirmation prompt is displayed, proving the corrected `sigaction()` installation actually delivers `EINTR` end-to-end (not just in a simulated unit test).
|
||||
- Risks: none material remaining — the EINTR-reliance risk this component previously left unverified is now closed by the explicit `sigaction()`/`SA_RESTART` fix and its end-to-end test.
|
||||
|
||||
### Output
|
||||
|
||||
#### JSON List Formatter
|
||||
|
||||
| Field | Detail |
|
||||
|-------|--------|
|
||||
| Intent | Shared cJSON array/object plumbing so the three listing commands can emit their existing per-row data as JSON |
|
||||
| Requirements | 5.1, 5.2, 5.3, 5.4 |
|
||||
|
||||
**Responsibilities & Constraints**
|
||||
- Owns only the array/object lifecycle helpers — not each command's field mapping. No generic "record" abstraction is introduced (per the Simplification lens — each command's row struct already exists and is stable).
|
||||
|
||||
**Dependencies**
|
||||
- Outbound: cJSON (already linked) — P0
|
||||
- Inbound: `cmd_workerlist`, `cmd_joblist`, `cmd_reslist` — P0
|
||||
|
||||
**Contracts**: Service [x]
|
||||
|
||||
##### Service Interface
|
||||
```c
|
||||
cJSON* listfmt_json_begin(void);
|
||||
void listfmt_json_add_string(cJSON* row, const char* key, const char* val);
|
||||
void listfmt_json_add_number(cJSON* row, const char* key, double val);
|
||||
cJSON* listfmt_json_new_row(cJSON* array);
|
||||
void listfmt_json_finish(cJSON* array);
|
||||
```
|
||||
- Preconditions: called only when the command's `--json` flag (a boolean `flagspec_t` entry, the same long-form name across all three commands per Requirement 6.2) is set.
|
||||
- Postconditions: every field present in the human-readable table row is also present in the JSON object (5.1); no truncation regardless of value length (5.3).
|
||||
|
||||
**Implementation Notes**
|
||||
- Integration: each command's existing print loop gains an `if (json_mode) { ... } else { existing table row } ` branch.
|
||||
- Validation: `test_listfmt_json.c` covers array/object lifecycle and long-value non-truncation; per-command field mapping is covered by that command's own test or the black-box CLI suite.
|
||||
- Risks: none material.
|
||||
|
||||
### REPL
|
||||
|
||||
#### Tab Completion
|
||||
|
||||
| Field | Detail |
|
||||
|-------|--------|
|
||||
| Intent | Register a libedit completion function scoped to the active context's command table |
|
||||
| Requirements | 7.1, 7.2, 7.3, 7.4 |
|
||||
|
||||
**Responsibilities & Constraints**
|
||||
- Owns the completion-candidate lookup against `active_ctx->commands[]` only.
|
||||
|
||||
**Dependencies**
|
||||
- Outbound: libedit completion API — P0 (exact signature to confirm at implementation start, see `research.md` Risks)
|
||||
|
||||
**Contracts**: Service [x]
|
||||
|
||||
##### Service Interface
|
||||
```c
|
||||
char** xcetest_complete(const char* text, int start, int end);
|
||||
char* xcetest_complete_generator(const char* text, int state);
|
||||
```
|
||||
- Preconditions: registered only when `g_interactive` is true.
|
||||
- Postconditions: never offers a command invalid in the active context (7.4); an unambiguous match completes in full (7.2); multiple matches list candidates without altering the input line (7.3).
|
||||
|
||||
**Implementation Notes**
|
||||
- Integration: registered once in `main()` alongside the existing `readline`/`add_history` setup. No new header — small enough to live in `test.c` directly.
|
||||
- Validation: covered by `test_test_cli.c` (pty-driven — libedit completion isn't practically unit-testable in isolation).
|
||||
- Risks: libedit API confirmation (`research.md` Risks).
|
||||
|
||||
#### Help Renderer
|
||||
|
||||
| Field | Detail |
|
||||
|-------|--------|
|
||||
| Intent | Compose `help <cmd>` from the generated flag block plus each command's hand-authored prose; scope "not found" strictly to the active context |
|
||||
| Requirements | 8.1, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7, 11.2, 12.2 |
|
||||
|
||||
**Responsibilities & Constraints**
|
||||
- Owns the assembly order (description -> flags -> example -> related-command/context notes) and the not-found message text.
|
||||
- Does not own the prose content — still hand-authored per command in the `USAGE_*` constants, now split into a prose part and a flagspec-rendered part.
|
||||
|
||||
**Dependencies**
|
||||
- Outbound: Flag Parsing Layer's `flagspec_render_help` — P0
|
||||
|
||||
**Contracts**: Service [x]
|
||||
|
||||
##### Service Interface
|
||||
Extends the existing `cmd_help` in `test.c` — no new header. `command_t` gains a `flagspec_t*`/count pair alongside its existing `usage` pointer; `cmd_help`'s per-command branch calls `flagspec_render_help(cmd->flagspec, cmd->flagspec_count)` and concatenates it with the command's (now flag-list-free) prose constant.
|
||||
|
||||
**Implementation Notes**
|
||||
- Integration: every `USAGE_*` constant is edited to drop its hand-written flag-list lines (now generated) while keeping description/example/related-command prose.
|
||||
- Validation: `test_test_cli.c` checks a representative sample of `help <cmd>` outputs for both a generated flag line and the retained hand-authored example, plus the corrected grammar (11.2) and `join`'s corrected default (12.2).
|
||||
- Risks: touches all 40 `USAGE_*` constants — mechanical but broad; sequence as its own task after the Flag Parsing Layer lands.
|
||||
|
||||
#### Source Command
|
||||
|
||||
| Field | Detail |
|
||||
|-------|--------|
|
||||
| Intent | `source <file>`, reusing `rcfile_load`, with cycle detection and an explicit missing-file error |
|
||||
| Requirements | 13.1, 13.2, 13.3, 13.4, 13.5, 13.6, 13.7 |
|
||||
|
||||
**Responsibilities & Constraints**
|
||||
- Owns the in-flight source-path stack (cycle detection) and the missing-file check.
|
||||
- Does not own `rcfile.c`'s file-reading loop or its startup-time missing-file policy — unchanged.
|
||||
|
||||
**Dependencies**
|
||||
- Outbound: `rcfile_load` (unmodified) — P0
|
||||
- Outbound: Confirmation Gate (13.7's TTY-condition clarification) — P1
|
||||
|
||||
**Contracts**: Service [x]
|
||||
|
||||
##### Service Interface
|
||||
```c
|
||||
int cmd_source(int argc, char* argv[]); /* registered command_t entry */
|
||||
int source_push_path(const char* canonical_path); /* 0 ok, -ELOOP on cycle */
|
||||
void source_pop_path(void);
|
||||
```
|
||||
- Preconditions: `argv[1]` is a path; `access(path, R_OK)` must succeed or `cmd_source` reports the error itself (bypassing `rcfile_load`'s silent-missing behavior).
|
||||
- Postconditions: the path stack is popped exactly once per successful push, on every exit path including an aborted line under `-e` semantics.
|
||||
- Invariants: `realpath()` is applied before pushing, so a cycle through symlinks or a differently-spelled relative path is still caught.
|
||||
|
||||
**Implementation Notes**
|
||||
- Integration: registered as a new entry in `top_commands`, `worker_commands`, `client_commands`, and `scheduler_commands`.
|
||||
- Validation: `test_test_cli.c` covers a two-file mutual-cycle case and a non-existent-file case as black-box scenarios (these need real files on disk).
|
||||
- Risks: the path stack must be popped on every exit path, not just the success path — call out explicitly in the implementing task.
|
||||
|
||||
## Data Models
|
||||
|
||||
Minimal — `xce_test` has no persistent storage. The only new data shapes are the C structs already shown above (`flagspec_t`, the interruptible-call state enum, the per-connection lock registry entry — a connection pointer paired with a `sem_t`, at most one per live context — and the source-path stack). No database, no schema, no migration.
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Error Strategy
|
||||
Every new or modified path returns a negative errno-style code per this project's existing C convention (`errnox.h`), and every error/diagnostic message is written to stderr per Requirement 3 — this design extends the existing error-reporting philosophy uniformly rather than introducing a new one.
|
||||
|
||||
### Error Categories and Responses
|
||||
- **Malformed/unrecognized flag** -> `flagspec_reject` (stderr + usage + optional suggestion, nonzero return) — Requirements 2, 10.
|
||||
- **Blocked RPC interrupted** -> `-EINTR` from `xcetest_run_interruptible`, reported as "failed" (1.3) except at a confirmation prompt, reported as "not run" (9.9).
|
||||
- **Destructive command declined** -> 0 return, explicit "not run" reporting (9.2).
|
||||
- **`source` missing file / cycle** -> stderr error, nonzero return, no partial execution (13.2, 13.5).
|
||||
|
||||
### Monitoring
|
||||
Not applicable — `xce_test` is an interactive/scripted client, not a long-running service. No new logging/metrics infrastructure beyond the existing stderr diagnostics this design extends.
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
**Unit Tests**
|
||||
- `test_flagspec.c` — optstring/option-table builder correctness; suggestion logic (exactly-one-match / no-match / ambiguous-match-suppressed); help-render format; `cmd_qctrl`'s corrected sentinel-and-reject validation (bug-1704).
|
||||
- `test_interruptcall.c` — normal completion path and forced-abandon path via a fake slow `fn`, verifying no double-free under either; two overlapping calls against the same fake connection key (one abandoned, one issued immediately after) never run concurrently, proving the per-connection lock.
|
||||
- `test_confirm.c` — non-TTY auto-bypass; decline path; simulated-EINTR-as-decline path; toggle on/off.
|
||||
- `test_listfmt_json.c` — array/object lifecycle; no truncation of a long field value.
|
||||
|
||||
**Integration Tests**
|
||||
- `cmd_source` two-file mutual-cycle detection; missing-file error; nested (non-cyclic) sourcing composing correctly.
|
||||
- Flag rejection end-to-end through a real migrated command (e.g. `jobkill -z`), confirming stderr routing + usage + nonzero exit, not just the shared helper in isolation.
|
||||
- Confirmation gate against a real destructive command (`qctrl -c`) with the session toggle flipped, confirming no prompt appears and the command executes.
|
||||
|
||||
**E2E / Black-box Tests** (`test_test_cli.c`, subprocess/pty-driven — the only viable vehicle for these)
|
||||
- `-h`/`--help`/`--version` print and exit without entering the REPL (Requirement 4).
|
||||
- Tab completion at the prompt: partial command name + Tab yields the expected single completion or candidate list, scoped to the active context (Requirement 7).
|
||||
- `help <cmd>` output contains both a generated flag line and the hand-authored example for a representative command in each context; `help <unknown>` reports not-found without a cross-context hint (Requirement 8, 11.2).
|
||||
- A destructive command run non-interactively (piped stdin) executes without hanging on a prompt (Requirement 9.4).
|
||||
- A real SIGINT delivered to the process while a confirmation prompt is displayed is observed as `EINTR` and reported as "not run" (Requirement 9.9), proving the `sigaction()`/`SA_RESTART` fix end-to-end rather than only in a simulated unit test.
|
||||
|
||||
No dedicated Performance/Load section is included — requirements.md states no performance target, and the Interruptible Call Adapter's 50ms poll interval is the only latency-relevant parameter, already justified in its component detail above.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
`xce_test` remains an internal test-only client; no new authentication/authorization surface is introduced, and connection/credential handling to the controller/worker/scheduler is unchanged and out of this spec's boundary. The Confirmation Gate (Requirement 9) is this design's one safety-relevant feature — it exists specifically to reduce the blast radius of an operator's typo or stale jobid against a live or degraded HA cluster during soak testing, not to guard against any external threat actor.
|
||||
|
|
@ -31,7 +31,7 @@
|
|||
- _Boundary: Interruptible Call Adapter_
|
||||
- _Depends: 1.3_
|
||||
|
||||
- [ ] 1.5 Add the per-connection lock registry to the interruptible-call adapter
|
||||
- [x] 1.5 Add the per-connection lock registry to the interruptible-call adapter
|
||||
- Add a small registry, keyed by connection identity, that serializes any two calls sharing a connection so an abandoned call's worker thread is never running concurrently with a newly issued call on the same connection
|
||||
- Make lock acquisition itself interruptible, so a new call waiting on a connection still held by a prior abandoned call can itself be cancelled
|
||||
- Observable completion: the unit test suite shows two overlapping calls against the same fake connection identity (one abandoned, one issued immediately after) never executing concurrently, while two calls against different connection identities run independently
|
||||
|
|
@ -178,5 +178,6 @@
|
|||
## Implementation Notes
|
||||
- `src/tools/xce-build.sh controller asan xce.build --incremental` writes its build dir at `src/xce.build/` (relative to the script's own location), not a repo-root `xce.build/` — meson logs are at `src/xce.build/meson-logs/testlog.txt`. (task 1.1)
|
||||
- New Unity test files need two wiring points, not just meson.build's source list: also register an `extern void test_<module>(void);` + `TEST(test_<module>);` in `src/controller/testsuite/controller_test.c`, mirroring the existing `test_rcfile` registration, or the suite silently never runs. (task 1.1)
|
||||
- The per-connection lock is a POSIX semaphore (`sem_t`), not a `pthread_mutex_t` — a mutex was tried first but rejected on review (POSIX classifies a non-owning-thread `pthread_mutex_unlock()` as UB for every mutex type, not just error-checking ones, and this design needs the worker thread to release a lock the caller thread acquired). `sem_t` has no ownership concept and is fully POSIX-defined for this pattern. See bug-1791. (task 1.5)
|
||||
- `interruptcall.c`'s forced-abandonment test proves `-EINTR` via return-value disambiguation against a slow `fn`, not a direct wall-clock elapsed-time assertion — sound today (poll interval is 50ms, matching design.md) but a future regression widening the poll interval wouldn't be caught unless the test also asserts an elapsed-time bound. Non-blocking; noted for task 4.1's end-to-end Ctrl-C validation to consider. (task 1.4)
|
||||
- After task 1.3's SIGINT/EINTR fix alone (before task 3.2 wires the REPL loop to check `xcetest_interrupted()`), a real Ctrl-C at `xce_test`'s top-level interactive prompt now causes the process to EXIT (readline's EINTR-driven NULL return is currently treated the same as EOF) rather than gracefully redisplaying the prompt. This is expected/deferred, not a regression — no current test exercises that path — but task 3.2's implementer must fix it as part of wiring the REPL loop, not just the RPC call sites. (task 1.3)
|
||||
|
|
|
|||
|
|
@ -21465,6 +21465,31 @@
|
|||
],
|
||||
"occurrences": 1,
|
||||
"last_seen": "2026-09-08T00:00:00Z"
|
||||
},
|
||||
{
|
||||
"id": "bug-1791",
|
||||
"timestamp": "2026-09-09T08:50:58.000Z",
|
||||
"error_message": "controller_unit_test: interruptcall.c:142: xcetest_worker_main: Assertion \\`unlock_rc == 0' failed (pthread_mutex_unlock returned EPERM)",
|
||||
"file": "src/controller/testsuite/interruptcall.c",
|
||||
"root_cause": "xce-test-cli-ux task 1.5's per-connection lock registry initialized each connection's pthread_mutex_t with PTHREAD_MUTEX_ERRORCHECK, intended to catch double-unlock/never-unlocked regressions. But this design's own ownership-handoff protocol (design.md/research.md's 'per-connection lock' decision) REQUIRES a legitimate cross-thread release on the abandonment path: the calling (REPL) thread acquires the lock via pthread_mutex_trylock(), but on abandonment the still-running detached WORKER thread is the one that must release it later, once fn() finishes -- ownership of 'release the lock' passes to the worker at the same moment ownership of the heap call block does. PTHREAD_MUTEX_ERRORCHECK (and _RECURSIVE) mutexes track a single owning thread and reject pthread_mutex_unlock() from any other thread with EPERM, which fired immediately on the very first same-key-serialization test.",
|
||||
"fix": "Switched the per-connection lock registry (xcetest_conn_lock_for() in interruptcall.c) to initialize each mutex with the DEFAULT type (pthread_mutex_init(&node->mutex, NULL)) instead of PTHREAD_MUTEX_ERRORCHECK. glibc default (PTHREAD_MUTEX_NORMAL-equivalent) mutexes do not track a per-thread owner, so a worker thread may validly unlock a mutex the REPL thread locked. Kept the assert(pthread_mutex_unlock(...) == 0) safety-net checks at every release site (computed outside the assert() expression itself, so the unlock call is never skipped under NDEBUG) as a lower-cost guard against genuine corruption, without relying on error-checking-mutex semantics that conflict with the intentional cross-thread handoff. SUPERSEDED 2026-09-09 (task 1.5 remediation round 1, independent review): an external reviewer correctly pointed out that POSIX classifies non-owning-thread pthread_mutex_unlock() as undefined behavior for PTHREAD_MUTEX_NORMAL/DEFAULT too, not just ERRORCHECK/_RECURSIVE -- NORMAL is merely *permitted* not to detect the violation, not declared well-defined for it. The DEFAULT-mutex fix above was therefore empirically safe on XCE's glibc/NPTL-exclusive Nix build but relied on non-portable, POSIX-UB behavior, under-documented as such. Replaced the pthread_mutex_t per-connection lock with a POSIX semaphore (sem_t, binary, sem_init(&sem, 0, 1)) throughout: sem_trywait() in place of pthread_mutex_trylock() (EAGAIN in place of EBUSY) in the interruptible poll loop, and sem_post() in place of pthread_mutex_unlock() at every release site (both the caller-normal-completion path and the worker-abandoned-cleanup cross-thread path). sem_t has no ownership concept, so this cross-thread release is now fully POSIX-defined with no UB reliance at all, at the same code complexity. All 1285 controller_unit_test tests (including the 3 task-1.5-specific tests) continued to pass unchanged after the migration.",
|
||||
"tags": [
|
||||
"pthread",
|
||||
"mutex",
|
||||
"EPERM",
|
||||
"cross-thread-unlock",
|
||||
"ownership-handoff",
|
||||
"xce-test-cli-ux",
|
||||
"task-1.5",
|
||||
"interruptcall.c",
|
||||
"design-review-caught-by-test",
|
||||
"posix-ub",
|
||||
"sem_t",
|
||||
"semaphore-migration"
|
||||
],
|
||||
"related_bugs": [],
|
||||
"occurrences": 1,
|
||||
"last_seen": "2026-09-09T08:50:58.000Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,7 +1,9 @@
|
|||
#include "interruptcall.h"
|
||||
|
||||
#include <assert.h>
|
||||
#include <errno.h>
|
||||
#include <pthread.h>
|
||||
#include <semaphore.h>
|
||||
#include <stdlib.h>
|
||||
#include <time.h>
|
||||
|
||||
|
|
@ -17,11 +19,19 @@
|
|||
* call. `state` is the single atomic word whose CAS transitions decide
|
||||
* who frees this block -- see interruptcall.h's xcetest_call_state_t
|
||||
* comment and xcetest_run_interruptible()'s own header comment for the
|
||||
* full protocol. */
|
||||
* full protocol. `lock` is the per-connection lock (task 1.5) this call
|
||||
* acquired before spawning its worker; whichever side -- caller or
|
||||
* worker -- ends up owning the block also ends up owning the job of
|
||||
* releasing `lock`, per the same protocol. It is a sem_t*, not a
|
||||
* pthread_mutex_t* -- see interruptcall.h's "WHY THE LOCK IS A POSIX
|
||||
* SEMAPHORE" section for why: the release below is a genuine cross-
|
||||
* thread handoff, which sem_t (unlike pthread_mutex_t, of any type)
|
||||
* defines cleanly. */
|
||||
typedef struct {
|
||||
xcetest_blocking_fn fn;
|
||||
void* ctx;
|
||||
int rc;
|
||||
sem_t* lock;
|
||||
_Atomic xcetest_call_state_t state;
|
||||
} xcetest_call_block_t;
|
||||
|
||||
|
|
@ -29,6 +39,80 @@ typedef struct {
|
|||
_Atomic int xcetest_test_abandoned_free_count = 0;
|
||||
#endif
|
||||
|
||||
/* --- Per-connection lock registry (task 1.5) ---
|
||||
*
|
||||
* See interruptcall.h's "PER-CONNECTION LOCK REGISTRY" and "WHY THE LOCK
|
||||
* IS A POSIX SEMAPHORE" file-header sections for the full rationale,
|
||||
* including why this is a singly-linked list (pointer stability for a
|
||||
* sem_t* a still-running abandoned worker may hold onto indefinitely)
|
||||
* rather than a growable array, and why each entry is a sem_t rather
|
||||
* than a pthread_mutex_t.
|
||||
*/
|
||||
typedef struct xcetest_conn_lock_node {
|
||||
void* key;
|
||||
sem_t sem;
|
||||
struct xcetest_conn_lock_node* next;
|
||||
} xcetest_conn_lock_node_t;
|
||||
|
||||
/* Only ever read/written from the single REPL thread that issues
|
||||
* top-level commands -- see interruptcall.h's "CONCURRENCY OF THE
|
||||
* REGISTRY ITSELF" note. Neither variable needs its own lock. */
|
||||
static xcetest_conn_lock_node_t* g_conn_lock_registry = NULL;
|
||||
static void* g_active_connection = NULL;
|
||||
|
||||
void xcetest_set_active_connection(void* conn_key)
|
||||
{
|
||||
g_active_connection = conn_key;
|
||||
}
|
||||
|
||||
/* Looks up (by pointer equality) or lazily creates the lock entry for
|
||||
* `key` in the process-lifetime registry. Returns NULL only if the new
|
||||
* node's allocation fails. Nodes are prepended to the list and NEVER
|
||||
* freed or moved for the rest of the process's life, so a pointer this
|
||||
* function has already returned to a caller stays valid forever,
|
||||
* regardless of how many later, unrelated connections are subsequently
|
||||
* registered. */
|
||||
static sem_t* xcetest_conn_lock_for(void* key)
|
||||
{
|
||||
xcetest_conn_lock_node_t* node;
|
||||
|
||||
for (node = g_conn_lock_registry; node; node = node->next) {
|
||||
if (node->key == key)
|
||||
return &node->sem;
|
||||
}
|
||||
|
||||
node = malloc(sizeof(*node));
|
||||
if (!node)
|
||||
return NULL;
|
||||
|
||||
node->key = key;
|
||||
/* A binary semaphore (initial value 1), used as a mutual-exclusion
|
||||
* lock, NOT a pthread_mutex_t: this design's own ownership-handoff
|
||||
* protocol (see xcetest_run_interruptible()'s and
|
||||
* xcetest_worker_main()'s comments) REQUIRES a cross-thread release
|
||||
* on the abandonment path -- the calling thread acquires the lock,
|
||||
* but the still-running WORKER thread is the one that releases it
|
||||
* once fn() finishes, since ownership of "release the lock" passed
|
||||
* to the worker at the same moment ownership of the call block did.
|
||||
* A pthread_mutex_t (ERRORCHECK/_RECURSIVE, or even the DEFAULT/
|
||||
* NORMAL type) is the wrong primitive for this: POSIX classifies an
|
||||
* unlock from a non-owning thread as undefined behavior for every
|
||||
* mutex type -- ERRORCHECK/_RECURSIVE are simply *required* to
|
||||
* detect and report it (which is why an earlier version of this code
|
||||
* hit an immediate EPERM here), while NORMAL/DEFAULT are merely
|
||||
* *permitted* not to detect it, not declared well-defined for it.
|
||||
* sem_t has no ownership concept at all -- sem_wait()/sem_trywait()/
|
||||
* sem_post() are fully defined regardless of which thread calls
|
||||
* which -- so it is the textbook-correct, portable primitive for
|
||||
* this exact cross-thread signal, with no UB reliance whatsoever.
|
||||
* pshared=0 since this registry is intra-process only. */
|
||||
sem_init(&node->sem, 0, 1);
|
||||
|
||||
node->next = g_conn_lock_registry;
|
||||
g_conn_lock_registry = node;
|
||||
return &node->sem;
|
||||
}
|
||||
|
||||
/* Runs on the detached worker thread. Invokes fn(ctx), stores its result,
|
||||
* then resolves ownership of `block` via the RUNNING -> DONE_UNCLAIMED
|
||||
* CAS: on success, the caller (still polling, or about to read the result
|
||||
|
|
@ -40,26 +124,51 @@ static void* xcetest_worker_main(void* arg)
|
|||
{
|
||||
xcetest_call_block_t* block = (xcetest_call_block_t*)arg;
|
||||
xcetest_call_state_t expected;
|
||||
sem_t* lock = block->lock;
|
||||
|
||||
block->rc = block->fn(block->ctx);
|
||||
|
||||
expected = XCT_RUNNING;
|
||||
if (atomic_compare_exchange_strong(&block->state, &expected,
|
||||
XCT_DONE_UNCLAIMED)) {
|
||||
/* Won the CAS: the caller still owns the block. Do not touch it
|
||||
* again from this thread. */
|
||||
/* Won the CAS: the caller still owns the block AND the
|
||||
* per-connection lock. Do not touch either again from this
|
||||
* thread. */
|
||||
return NULL;
|
||||
}
|
||||
|
||||
/* CAS failed. The caller is the only other party that ever writes
|
||||
* block->state, and the only value it ever writes is XCT_ABANDONED --
|
||||
* so `expected` (and the block's actual state) can only be
|
||||
* XCT_ABANDONED here. The caller returned -EINTR without freeing;
|
||||
* this thread now owns the block and must free it. */
|
||||
* XCT_ABANDONED here. The caller returned -EINTR without touching
|
||||
* the block or the lock again; this thread now owns both. Free the
|
||||
* block (and record the test-only counter) BEFORE releasing the
|
||||
* lock, so that any other thread waiting on this same connection's
|
||||
* lock only ever observes it become available once this cleanup has
|
||||
* already fully happened. */
|
||||
free(block);
|
||||
#ifdef XCE_TESTING
|
||||
atomic_fetch_add(&xcetest_test_abandoned_free_count, 1);
|
||||
#endif
|
||||
{
|
||||
/* sem_post(), unlike pthread_mutex_unlock(), is well-defined from
|
||||
* any thread regardless of which thread called sem_wait()/
|
||||
* sem_trywait() -- see interruptcall.h's "WHY THE LOCK IS A
|
||||
* POSIX SEMAPHORE" section. This cross-thread release is exactly
|
||||
* the case that primitive was chosen for. */
|
||||
int post_rc = sem_post(lock);
|
||||
(void)post_rc; /* checked via assert() below, never inside it --
|
||||
* an assert() argument's side effects vanish
|
||||
* under NDEBUG, which must never apply to the
|
||||
* post call itself. NOTE: this project's
|
||||
* `release` build profile compiles with -DNDEBUG
|
||||
* (unlike asan/debug), so this assert() -- like
|
||||
* every other one in this file -- compiles out
|
||||
* entirely there; it is a debug/asan-only safety
|
||||
* net, not a guarantee in a release-profile
|
||||
* xce_test binary. */
|
||||
assert(post_rc == 0);
|
||||
}
|
||||
return NULL;
|
||||
}
|
||||
|
||||
|
|
@ -79,6 +188,7 @@ static void xcetest_sleep_poll_interval(void)
|
|||
int xcetest_run_interruptible(xcetest_blocking_fn fn, void* ctx)
|
||||
{
|
||||
xcetest_call_block_t* block;
|
||||
sem_t* lock;
|
||||
pthread_attr_t attr;
|
||||
pthread_t worker;
|
||||
int rc;
|
||||
|
|
@ -86,32 +196,81 @@ int xcetest_run_interruptible(xcetest_blocking_fn fn, void* ctx)
|
|||
if (!fn)
|
||||
return -EINVAL;
|
||||
|
||||
block = malloc(sizeof(*block));
|
||||
if (!block)
|
||||
lock = xcetest_conn_lock_for(g_active_connection);
|
||||
if (!lock)
|
||||
return -ENOMEM;
|
||||
|
||||
block->fn = fn;
|
||||
block->ctx = ctx;
|
||||
block->rc = 0;
|
||||
/* Interruptible lock acquisition (task 1.5): poll sem_trywait() on
|
||||
* the same interval used for the completion-wait poll below,
|
||||
* checking xcetest_interrupted() between attempts. A plain
|
||||
* sem_wait() would block uninterruptibly, defeating the whole point
|
||||
* of letting a second call on this same connection be Ctrl-C'd out
|
||||
* of waiting for a prior abandoned call to release it.
|
||||
* sem_trywait() returns -1/EAGAIN when the semaphore is currently
|
||||
* at 0 (held) -- the direct equivalent of pthread_mutex_trylock()'s
|
||||
* EBUSY for this poll loop's purposes. If interrupted here, this
|
||||
* call never acquires the lock and never spawns a worker -- there
|
||||
* is nothing to release. */
|
||||
for (;;) {
|
||||
if (sem_trywait(lock) == 0)
|
||||
break; /* acquired */
|
||||
if (errno != EAGAIN)
|
||||
return -errno; /* unexpected sem_trywait() error; nothing held */
|
||||
|
||||
if (xcetest_interrupted())
|
||||
return -EINTR; /* never acquired; nothing to release */
|
||||
|
||||
xcetest_sleep_poll_interval();
|
||||
}
|
||||
|
||||
block = malloc(sizeof(*block));
|
||||
if (!block) {
|
||||
rc = sem_post(lock);
|
||||
(void)rc;
|
||||
assert(rc == 0);
|
||||
return -ENOMEM;
|
||||
}
|
||||
|
||||
block->fn = fn;
|
||||
block->ctx = ctx;
|
||||
block->rc = 0;
|
||||
block->lock = lock;
|
||||
atomic_init(&block->state, XCT_RUNNING);
|
||||
|
||||
rc = pthread_attr_init(&attr);
|
||||
if (rc != 0) {
|
||||
free(block);
|
||||
{
|
||||
/* NOTE: this assert(), like every other one in this file,
|
||||
* compiles out under -DNDEBUG (this project's `release`
|
||||
* build profile) -- a documented debug/asan-only safety net,
|
||||
* not a release-build guarantee. */
|
||||
int post_rc = sem_post(lock);
|
||||
(void)post_rc;
|
||||
assert(post_rc == 0);
|
||||
}
|
||||
return -rc;
|
||||
}
|
||||
/* Detached: no one ever joins this thread. That is precisely why
|
||||
* ownership of `block` must be resolved by the CAS protocol below
|
||||
* rather than by a join -- an abandoned caller returns without
|
||||
* waiting for this thread at all. */
|
||||
* ownership of `block` (and of `lock`) must be resolved by the CAS
|
||||
* protocol below rather than by a join -- an abandoned caller
|
||||
* returns without waiting for this thread at all. */
|
||||
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED);
|
||||
|
||||
rc = pthread_create(&worker, &attr, xcetest_worker_main, block);
|
||||
pthread_attr_destroy(&attr);
|
||||
if (rc != 0) {
|
||||
/* The worker never started, so fn() was never invoked and this
|
||||
* thread unambiguously still owns the block by itself. */
|
||||
* thread unambiguously still owns the block, and the lock, by
|
||||
* itself. */
|
||||
free(block);
|
||||
{
|
||||
/* NDEBUG caveat: see the pthread_attr_init() failure branch
|
||||
* above. */
|
||||
int post_rc = sem_post(lock);
|
||||
(void)post_rc;
|
||||
assert(post_rc == 0);
|
||||
}
|
||||
return -rc;
|
||||
}
|
||||
|
||||
|
|
@ -121,6 +280,13 @@ int xcetest_run_interruptible(xcetest_blocking_fn fn, void* ctx)
|
|||
if (state == XCT_DONE_UNCLAIMED) {
|
||||
int result = block->rc;
|
||||
free(block);
|
||||
{
|
||||
/* NDEBUG caveat: see the pthread_attr_init() failure
|
||||
* branch above. */
|
||||
int post_rc = sem_post(lock);
|
||||
(void)post_rc;
|
||||
assert(post_rc == 0);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
|
|
@ -129,9 +295,9 @@ int xcetest_run_interruptible(xcetest_blocking_fn fn, void* ctx)
|
|||
if (atomic_compare_exchange_strong(&block->state, &expected,
|
||||
XCT_ABANDONED)) {
|
||||
/* Won the CAS: the worker (still running fn()) will see
|
||||
* ABANDONED via its own failed CAS once fn() returns and
|
||||
* free the block itself. Must not touch `block` again
|
||||
* from here. */
|
||||
* ABANDONED via its own failed CAS once fn() returns,
|
||||
* free the block, and release `lock` itself. Must not
|
||||
* touch `block` or `lock` again from here. */
|
||||
return -EINTR;
|
||||
}
|
||||
/* CAS failed: the worker is the only other party that ever
|
||||
|
|
|
|||
|
|
@ -34,15 +34,82 @@
|
|||
* pthread_cancel() was rejected (cancellation-safety of
|
||||
* pthread_cond_timedwait()'s internal state is not established).
|
||||
*
|
||||
* NOTE ON SCOPE (task 1.4 boundary): this module has NO notion of
|
||||
* "connection identity" at all -- it does not know or care what `ctx`
|
||||
* represents, and does not serialize two calls against each other in any
|
||||
* way. The per-connection lock registry that serializes overlapping calls
|
||||
* sharing a connection (including a call issued right after a prior one
|
||||
* on that connection was abandoned) is task 1.5's addition, layered on
|
||||
* top of this exact xcetest_run_interruptible() signature -- deliberately
|
||||
* unchanged by task 1.5 so it can add that behavior without this module
|
||||
* needing to guess its shape.
|
||||
* PER-CONNECTION LOCK REGISTRY (task 1.5, Requirements 1.1, 1.5): an
|
||||
* abandoned worker thread from task 1.4's mechanism above keeps running
|
||||
* against whatever connection state `ctx` closes over. If the operator's
|
||||
* very next command reuses that SAME connection, its own call would race
|
||||
* the still-running abandoned worker for that connection's single-
|
||||
* outstanding-reply state -- not a rare edge case, but the direct, likely
|
||||
* consequence of Ctrl-C's own purpose (cancel, then immediately try
|
||||
* something else). xcetest_run_interruptible() closes this by acquiring
|
||||
* a per-connection lock -- looked up or lazily created in a small,
|
||||
* process-lifetime registry keyed by whatever `xcetest_set_active_
|
||||
* connection()` was last called with -- BEFORE spawning the worker, and
|
||||
* holding it for the call's full lifetime, including through an
|
||||
* abandonment (ownership of "release the lock" passes to the worker in
|
||||
* that case, exactly mirroring the heap-block CAS handoff above). Lock
|
||||
* acquisition is itself interruptible (the same break_out poll loop), so
|
||||
* a second call waiting on a connection a prior abandoned call still
|
||||
* holds can itself be Ctrl-C'd out of that wait, before ever acquiring
|
||||
* the lock or spawning a worker of its own. See design.md's "Interruptible
|
||||
* RPC Call and Ctrl-C Abandonment" sequence diagram and research.md's
|
||||
* "Decision: Per-connection lock serializes overlapping calls on the
|
||||
* Interruptible Call Adapter" for the full rationale.
|
||||
*
|
||||
* WHY THE LOCK IS A POSIX SEMAPHORE (sem_t), NOT A MUTEX: the handoff
|
||||
* above is a genuine CROSS-THREAD release -- the REPL/caller thread
|
||||
* acquires the lock before spawning the worker, but on the abandonment
|
||||
* path it is the still-running WORKER thread that later releases it,
|
||||
* once fn() finishes. A pthread_mutex_t (of ANY type, including the
|
||||
* default PTHREAD_MUTEX_NORMAL) is the wrong primitive for this: POSIX
|
||||
* defines pthread_mutex_t in terms of a single owning thread, and
|
||||
* classifies pthread_mutex_unlock() from a non-owning thread as
|
||||
* undefined behavior for every mutex type -- PTHREAD_MUTEX_ERRORCHECK/
|
||||
* _RECURSIVE are simply *required* to detect and report that UB (hence
|
||||
* the EPERM an earlier version of this code hit immediately), while
|
||||
* NORMAL/DEFAULT are merely *permitted* not to detect it. POSIX does NOT
|
||||
* separately declare cross-thread unlock well-defined for NORMAL; it
|
||||
* just does not mandate an error return for it. A pthread_mutex_t-based
|
||||
* fix that swaps ERRORCHECK for the default type therefore only trades a
|
||||
* loud crash for a silent reliance on non-portable, POSIX-classified-UB
|
||||
* behavior -- something that happens to hold on this project's actual
|
||||
* target (glibc/NPTL's PTHREAD_MUTEX_NORMAL has had no ownership
|
||||
* tracking for decades, and XCE's Nix-hermetic build is glibc/Linux-only)
|
||||
* but is the wrong thing to build on when a fully-defined alternative
|
||||
* exists at the same code complexity.
|
||||
*
|
||||
* sem_t has NO ownership concept at all: sem_wait()/sem_trywait()/
|
||||
* sem_post() may be called from any thread regardless of which thread
|
||||
* called which, with fully-defined POSIX semantics for exactly this
|
||||
* cross-thread signal pattern. Each registry entry is therefore a binary
|
||||
* semaphore (sem_init(&sem, 0, 1)) used as a mutual-exclusion lock:
|
||||
* sem_trywait() takes the place of pthread_mutex_trylock() in the
|
||||
* interruptible poll loop below (EAGAIN where trylock would have
|
||||
* returned EBUSY), and sem_post() takes the place of
|
||||
* pthread_mutex_unlock() at both release sites -- with no UB caveat on
|
||||
* either.
|
||||
*
|
||||
* WHY THE REGISTRY IS A SINGLY-LINKED LIST, NOT A GROWABLE ARRAY: a
|
||||
* sem_t* handed out by this registry may be cached inside a call block
|
||||
* that a still-running abandoned worker thread holds onto indefinitely
|
||||
* (up to ~10s, per the heap-block ownership handoff above). A growable
|
||||
* array's realloc() can MOVE every previously-returned element the
|
||||
* moment a later, unrelated connection is first seen -- silently
|
||||
* invalidating that worker's pointer into stale/reused memory. A linked
|
||||
* list's nodes are individually allocated and, per this registry's own
|
||||
* "entries are never removed" contract, never freed or moved for the
|
||||
* rest of the process's life, so a pointer into it stays valid forever
|
||||
* once handed out.
|
||||
*
|
||||
* CONCURRENCY OF THE REGISTRY ITSELF: per design.md's Implementation
|
||||
* Notes, "the registry's lookup/insert step needs no synchronization of
|
||||
* its own -- a new registry entry is only ever created from the single
|
||||
* REPL thread that issues commands" (detached worker threads never
|
||||
* initiate a new top-level call; they only run the `fn` they were given
|
||||
* and release the lock/free the block they were handed). The SEMAPHORE
|
||||
* each entry owns, once handed out, is of course safe for concurrent
|
||||
* wait/post from different threads -- that safety is sem_t's own job,
|
||||
* not the registry's.
|
||||
*/
|
||||
|
||||
#include <stdatomic.h>
|
||||
|
|
@ -74,6 +141,42 @@ typedef enum {
|
|||
XCT_ABANDONED
|
||||
} xcetest_call_state_t;
|
||||
|
||||
/*
|
||||
* Sets the connection-identity key that xcetest_run_interruptible() below
|
||||
* reads INTERNALLY when a new call starts, to look up (or lazily create)
|
||||
* that connection's entry in the per-connection lock registry (task 1.5,
|
||||
* see the file header above). This is a simple settable-elsewhere,
|
||||
* read-internally piece of process state -- deliberately mirroring how
|
||||
* break_out/xcetest_interrupted() already works -- NOT a parameter added
|
||||
* to xcetest_run_interruptible()'s own signature, which stays exactly
|
||||
* `(fn, ctx)` per this task's design constraint.
|
||||
*
|
||||
* SEAM FOR A FUTURE CALLER (task 3.2): production code (test.c's cmd_*
|
||||
* functions) does not exist inside this module and has no notion of
|
||||
* "the currently active command_ctx_t's connection pointer" that this
|
||||
* header can reach into directly -- test.c itself is never linked into
|
||||
* this binary's unit tests (see this file's own header and
|
||||
* xcetest_interrupt.h's). The intended real wiring is: before invoking
|
||||
* xcetest_run_interruptible() for a blocking RPC, a cmd_* function (or a
|
||||
* shared dispatch point above all of them) calls
|
||||
* xcetest_set_active_connection(<that connection's own stable pointer,
|
||||
* e.g. the command_ctx_t's connection field>). Any stable, comparable-by-
|
||||
* identity pointer works as `conn_key` -- this module never dereferences
|
||||
* it, only compares it by pointer equality against the registry's keys.
|
||||
* Until task 3.2 wires that in, xce_test's real binary calls this
|
||||
* exactly once, from wherever main()/the dispatcher sets up each
|
||||
* command's active connection context.
|
||||
*
|
||||
* Concurrency: like break_out, callable only from the single REPL thread
|
||||
* that issues top-level commands -- see the file header's "CONCURRENCY
|
||||
* OF THE REGISTRY ITSELF" note. Passing the same key across unrelated
|
||||
* connections would incorrectly serialize them against each other;
|
||||
* passing a fresh/different key per connection is what keeps unrelated
|
||||
* connections independent (Requirement 1.5, "leave ... its underlying
|
||||
* connection intact").
|
||||
*/
|
||||
void xcetest_set_active_connection(void* conn_key);
|
||||
|
||||
/*
|
||||
* Runs `fn(ctx)` on a detached worker thread while THIS (the calling)
|
||||
* thread polls xcetest_interrupted() every ~50ms (design.md's Interruptible
|
||||
|
|
@ -81,6 +184,16 @@ typedef enum {
|
|||
* it immediately instead of blocking for fn()'s own, possibly much
|
||||
* longer, duration.
|
||||
*
|
||||
* Before spawning that worker, this function first acquires -- also
|
||||
* interruptibly, via the same ~50ms xcetest_interrupted() poll, but
|
||||
* using sem_trywait() instead of a blocking sem_wait() -- the
|
||||
* per-connection lock for whichever connection key was last passed to
|
||||
* xcetest_set_active_connection() (task 1.5; see the file header's "PER-
|
||||
* CONNECTION LOCK REGISTRY" section above). This serializes this call
|
||||
* against any other in-flight or abandoned-but-still-running call on the
|
||||
* same connection, so that connection's single-outstanding-reply state
|
||||
* is never touched by two threads at once.
|
||||
*
|
||||
* Preconditions:
|
||||
* - `fn` is non-NULL.
|
||||
* - `fn` touches only what `ctx` itself heap-owns -- NEVER memory on
|
||||
|
|
@ -93,18 +206,32 @@ typedef enum {
|
|||
* - Returns fn()'s own return value when fn() finishes before this
|
||||
* thread's abandon-CAS wins the race (whether or not
|
||||
* xcetest_interrupted() ever became true during the wait) -- see the
|
||||
* header comment's "near-simultaneous" case below.
|
||||
* - Returns -EINTR when this thread observes an interrupt and its own
|
||||
* CAS (RUNNING -> ABANDONED) wins the race. On this return, the
|
||||
* caller MUST NOT touch `ctx` again -- ownership has passed to the
|
||||
* still-running worker thread, which will finish fn() and free the
|
||||
* call block on its own, asynchronously, with no further signal back
|
||||
* header comment's "near-simultaneous" case below. On this path, the
|
||||
* per-connection lock is released by this (the caller's) thread,
|
||||
* after reading rc and freeing the block.
|
||||
* - Returns -EINTR when this thread observes an interrupt WHILE
|
||||
* WAITING FOR THE PER-CONNECTION LOCK -- i.e. before ever acquiring
|
||||
* it. No worker is spawned for this call at all in this case, and
|
||||
* nothing needs releasing, since this call never held the lock to
|
||||
* begin with.
|
||||
* - Returns -EINTR when this thread observes an interrupt AFTER
|
||||
* acquiring the lock and spawning the worker, and its own CAS
|
||||
* (RUNNING -> ABANDONED) wins the race. On this return, the caller
|
||||
* MUST NOT touch `ctx` again -- ownership of both the call block AND
|
||||
* the per-connection lock has passed to the still-running worker
|
||||
* thread, which will finish fn(), free the call block, and release
|
||||
* the lock on its own, asynchronously, with no further signal back
|
||||
* to this caller.
|
||||
* - Returns -EINVAL if `fn` is NULL (fn is never invoked in this case).
|
||||
* - Returns a negative errno from pthread_attr_init()/pthread_create()
|
||||
* if the worker thread could not be set up or spawned at all; the
|
||||
* call block is freed before returning and fn() is never invoked in
|
||||
* this case either.
|
||||
* - Returns -EINVAL if `fn` is NULL (fn is never invoked in this case;
|
||||
* no lock is acquired).
|
||||
* - Returns a negative errno from the per-connection lock registry
|
||||
* lookup (e.g. -ENOMEM), from sem_trywait() itself for an
|
||||
* unexpected error other than EAGAIN, or from pthread_attr_init()/
|
||||
* pthread_create() if the worker thread could not be set up or
|
||||
* spawned at all; in every one of these cases fn() is never invoked,
|
||||
* any call block already allocated is freed before returning, and
|
||||
* the per-connection lock -- if this call had already acquired it --
|
||||
* is released before returning.
|
||||
*
|
||||
* The near-simultaneous race (this thread notices the interrupt in the
|
||||
* same window the worker finishes fn()) is resolved by the SAME single
|
||||
|
|
@ -113,14 +240,12 @@ typedef enum {
|
|||
* attempt} executes first wins, and the loser observes that the state is
|
||||
* no longer RUNNING and falls back to the OTHER side's outcome -- a
|
||||
* caller who loses the abandon-CAS falls through to the normal
|
||||
* completion path (reads rc, frees the block, returns rc) instead of
|
||||
* returning -EINTR; a worker who loses the done-CAS frees the block
|
||||
* itself instead of leaving it for the caller. Exactly one free() of the
|
||||
* block is therefore reachable on every possible interleaving -- never
|
||||
* zero, never two.
|
||||
*
|
||||
* Does not acquire any lock and has no notion of connection identity --
|
||||
* see the file header's "NOTE ON SCOPE" above.
|
||||
* completion path (reads rc, frees the block, releases the lock, returns
|
||||
* rc) instead of returning -EINTR; a worker who loses the done-CAS frees
|
||||
* the block and releases the lock itself instead of leaving them for the
|
||||
* caller. Exactly one free() of the block, and exactly one release of
|
||||
* the lock, are therefore reachable on every possible interleaving --
|
||||
* never zero, never two.
|
||||
*/
|
||||
int xcetest_run_interruptible(xcetest_blocking_fn fn, void* ctx);
|
||||
|
||||
|
|
|
|||
|
|
@ -7,6 +7,18 @@
|
|||
* and frees the block itself, proven via a side channel rather than
|
||||
* assumed from a fixed sleep.
|
||||
*
|
||||
* ALSO proves (task 1.5, Requirements 1.1, 1.5) the per-connection lock
|
||||
* registry added on top of that same call: two calls sharing a
|
||||
* connection identity (set via xcetest_set_active_connection()) never
|
||||
* run their fn bodies concurrently -- including a call issued right
|
||||
* after a prior one on that same connection was abandoned, which must
|
||||
* wait (interruptibly) for the abandoned worker to finish and release
|
||||
* the lock -- while two calls against DIFFERENT connection identities
|
||||
* run independently/concurrently. A third scenario proves that a call
|
||||
* interrupted WHILE waiting for a lock still held by a prior abandoned
|
||||
* call returns -EINTR without ever acquiring the lock or spawning a
|
||||
* worker for itself.
|
||||
*
|
||||
* xcetest_interrupted() (declared in xcetest_interrupt.h) is normally
|
||||
* defined in test.c, over test.c's own private break_out flag -- but
|
||||
* test.c itself is never linked into controller_unit_test (see
|
||||
|
|
@ -73,7 +85,10 @@ static int fast_fn(void* ctx)
|
|||
|
||||
static void test_normal_completion_returns_real_result(void)
|
||||
{
|
||||
static int conn_normal; /* private fake connection key for this test */
|
||||
|
||||
g_test_break_out = 0;
|
||||
xcetest_set_active_connection(&conn_normal);
|
||||
|
||||
int rc = xcetest_run_interruptible(fast_fn, NULL);
|
||||
|
||||
|
|
@ -119,11 +134,36 @@ static void* send_sigint_after_delay(void* arg)
|
|||
return NULL;
|
||||
}
|
||||
|
||||
/* Polls the XCE_TESTING-only xcetest_test_abandoned_free_count side
|
||||
* channel (NOT part of the production API) until it reaches `expected`
|
||||
* or a generous timeout elapses, then asserts it landed exactly on
|
||||
* `expected` -- so a test proves an abandoned worker's asynchronous
|
||||
* cleanup (free + lock release, see interruptcall.c's abandoned path)
|
||||
* actually completed, rather than merely hoping enough wall-clock time
|
||||
* passed before the process moves on. Shared by every scenario below
|
||||
* that produces an abandoned-but-still-running worker. */
|
||||
static void wait_for_abandoned_free_count(int expected)
|
||||
{
|
||||
int waited_ms = 0;
|
||||
while (atomic_load(&xcetest_test_abandoned_free_count) < expected
|
||||
&& waited_ms < WAIT_FOR_FREE_MS) {
|
||||
struct timespec poll;
|
||||
poll.tv_sec = 0;
|
||||
poll.tv_nsec = (long)POLL_STEP_MS * 1000L * 1000L;
|
||||
nanosleep(&poll, NULL);
|
||||
waited_ms += POLL_STEP_MS;
|
||||
}
|
||||
TEST_ASSERT_EQUAL_INT(expected, atomic_load(&xcetest_test_abandoned_free_count));
|
||||
}
|
||||
|
||||
static void test_forced_abandonment_returns_eintr_and_worker_cleans_up(void)
|
||||
{
|
||||
static int conn_abandon; /* private fake connection key for this test */
|
||||
|
||||
g_test_break_out = 0;
|
||||
atomic_store(&xcetest_test_abandoned_free_count, 0);
|
||||
xcetest_install_sigint(test_sigint_handler);
|
||||
xcetest_set_active_connection(&conn_abandon);
|
||||
|
||||
pthread_t signaler;
|
||||
TEST_ASSERT_EQUAL_INT(
|
||||
|
|
@ -140,27 +180,296 @@ static void test_forced_abandonment_returns_eintr_and_worker_cleans_up(void)
|
|||
TEST_ASSERT_EQUAL_INT(0, pthread_join(signaler, NULL));
|
||||
|
||||
/* The abandoned worker is still running slow_fn() in the background
|
||||
* at this point. Poll the XCE_TESTING-only side channel (NOT part of
|
||||
* the production API) instead of a single fixed sleep, so this test
|
||||
* proves the abandoned worker's asynchronous cleanup actually
|
||||
* happened -- confirming no leak -- rather than merely hoping enough
|
||||
* wall-clock time passed before the process exits. */
|
||||
int waited_ms = 0;
|
||||
while (atomic_load(&xcetest_test_abandoned_free_count) == 0
|
||||
&& waited_ms < WAIT_FOR_FREE_MS) {
|
||||
struct timespec poll;
|
||||
poll.tv_sec = 0;
|
||||
poll.tv_nsec = (long)POLL_STEP_MS * 1000L * 1000L;
|
||||
nanosleep(&poll, NULL);
|
||||
waited_ms += POLL_STEP_MS;
|
||||
}
|
||||
* at this point; wait_for_abandoned_free_count() proves its
|
||||
* asynchronous cleanup actually happened -- confirming no leak --
|
||||
* rather than merely hoping enough wall-clock time passed before the
|
||||
* process exits. */
|
||||
wait_for_abandoned_free_count(1);
|
||||
}
|
||||
|
||||
/* ---- Scenario (c): per-connection lock registry (task 1.5) ----
|
||||
*
|
||||
* Requirements 1.1, 1.5. Three scenarios, per the task's Observable
|
||||
* Completion line:
|
||||
* (i) two overlapping calls sharing the SAME fake connection key
|
||||
* (one abandoned, one issued immediately after) never run their
|
||||
* fn bodies concurrently -- the second waits for the first's
|
||||
* still-running abandoned worker to release the lock;
|
||||
* (ii) two calls against DIFFERENT fake connection keys run
|
||||
* independently -- one's abandonment does not block the other;
|
||||
* (iii) a call interrupted WHILE waiting for a lock still held by a
|
||||
* prior abandoned call returns -EINTR without ever acquiring
|
||||
* the lock or spawning a worker for itself.
|
||||
*
|
||||
* All three reuse the same real-SIGINT-delivery pattern as scenario (b)
|
||||
* above (xcetest_install_sigint() + a helper thread's kill(getpid(),
|
||||
* SIGINT)) rather than flipping g_test_break_out by hand, for the same
|
||||
* reason given in this file's header comment. g_test_break_out is
|
||||
* intentionally reset to 0 between two calls in the same scenario
|
||||
* whenever the SECOND call is expected to complete normally (not be
|
||||
* interrupted itself) -- xcetest_interrupted() is a pure, non-clearing
|
||||
* read (see xcetest_interrupt.h's CONTRACT), so a stale "interrupted"
|
||||
* flag left set by an earlier call in the same test would otherwise be
|
||||
* observed by the second call's own lock-wait or completion-wait poll
|
||||
* and spuriously abandon it too.
|
||||
*/
|
||||
|
||||
/* Shared probe: counts how many fn instances are concurrently "inside"
|
||||
* their body right now, and how many times an instance started while
|
||||
* that count was already nonzero. Used two ways below: scenario (i)
|
||||
* asserts overlap_count stays 0 (same key must serialize); scenario
|
||||
* (ii) asserts it becomes nonzero (different keys must NOT serialize). */
|
||||
typedef struct {
|
||||
_Atomic int running;
|
||||
_Atomic int overlap_count;
|
||||
_Atomic int ran;
|
||||
} overlap_probe_t;
|
||||
|
||||
static int probe_enter(overlap_probe_t* p)
|
||||
{
|
||||
int prev = atomic_fetch_add(&p->running, 1);
|
||||
if (prev != 0)
|
||||
atomic_fetch_add(&p->overlap_count, 1);
|
||||
atomic_store(&p->ran, 1);
|
||||
return prev;
|
||||
}
|
||||
|
||||
static void probe_leave(overlap_probe_t* p)
|
||||
{
|
||||
atomic_fetch_sub(&p->running, 1);
|
||||
}
|
||||
|
||||
static void sleep_ms(long ms)
|
||||
{
|
||||
struct timespec ts;
|
||||
ts.tv_sec = ms / 1000;
|
||||
ts.tv_nsec = (ms % 1000) * 1000L * 1000L;
|
||||
nanosleep(&ts, NULL);
|
||||
}
|
||||
|
||||
static long elapsed_ms_since(const struct timespec* start)
|
||||
{
|
||||
struct timespec now;
|
||||
clock_gettime(CLOCK_MONOTONIC, &now);
|
||||
return (now.tv_sec - start->tv_sec) * 1000L
|
||||
+ (now.tv_nsec - start->tv_nsec) / 1000000L;
|
||||
}
|
||||
|
||||
/* -- (i) same connection key serializes -- */
|
||||
|
||||
#define SAME_KEY_SLOW_SLEEP_MS 900
|
||||
#define SAME_KEY_FAST_SLEEP_MS 50
|
||||
#define SAME_KEY_FAST_RC 42
|
||||
#define SAME_KEY_MIN_WAIT_MS 300 /* well below the ~700ms this call
|
||||
* should actually wait for the
|
||||
* first call's abandoned worker,
|
||||
* comfortably above scheduling
|
||||
* noise if the lock were NOT
|
||||
* enforced (that failure mode
|
||||
* returns in tens of ms). */
|
||||
|
||||
static int same_key_slow_fn(void* arg)
|
||||
{
|
||||
overlap_probe_t* p = arg;
|
||||
probe_enter(p);
|
||||
sleep_ms(SAME_KEY_SLOW_SLEEP_MS);
|
||||
probe_leave(p);
|
||||
return SLOW_FN_RC;
|
||||
}
|
||||
|
||||
static int same_key_fast_fn(void* arg)
|
||||
{
|
||||
overlap_probe_t* p = arg;
|
||||
probe_enter(p);
|
||||
sleep_ms(SAME_KEY_FAST_SLEEP_MS);
|
||||
probe_leave(p);
|
||||
return SAME_KEY_FAST_RC;
|
||||
}
|
||||
|
||||
static void test_same_connection_key_serializes_overlapping_calls(void)
|
||||
{
|
||||
static int conn_a; /* address used purely as an opaque fake key */
|
||||
overlap_probe_t probe = {0};
|
||||
pthread_t signaler;
|
||||
struct timespec wait_start;
|
||||
|
||||
g_test_break_out = 0;
|
||||
atomic_store(&xcetest_test_abandoned_free_count, 0);
|
||||
xcetest_install_sigint(test_sigint_handler);
|
||||
xcetest_set_active_connection(&conn_a);
|
||||
|
||||
TEST_ASSERT_EQUAL_INT(
|
||||
0, pthread_create(&signaler, NULL, send_sigint_after_delay, NULL));
|
||||
|
||||
int rc1 = xcetest_run_interruptible(same_key_slow_fn, &probe);
|
||||
|
||||
TEST_ASSERT_EQUAL_INT(-EINTR, rc1);
|
||||
TEST_ASSERT_EQUAL_INT(0, pthread_join(signaler, NULL));
|
||||
TEST_ASSERT_TRUE_MESSAGE(atomic_load(&probe.ran),
|
||||
"the slow call must have started running "
|
||||
"before it was abandoned");
|
||||
|
||||
/* conn_a's lock is now held by the still-running abandoned worker.
|
||||
* Reset the fake interrupt flag -- this second call must NOT be
|
||||
* treated as interrupted itself; it is expected to wait for, then
|
||||
* acquire, the same lock once the first worker releases it. */
|
||||
g_test_break_out = 0;
|
||||
|
||||
clock_gettime(CLOCK_MONOTONIC, &wait_start);
|
||||
int rc2 = xcetest_run_interruptible(same_key_fast_fn, &probe);
|
||||
long waited_ms = elapsed_ms_since(&wait_start);
|
||||
|
||||
TEST_ASSERT_EQUAL_INT(SAME_KEY_FAST_RC, rc2);
|
||||
TEST_ASSERT_EQUAL_INT(0, atomic_load(&probe.overlap_count));
|
||||
TEST_ASSERT_TRUE_MESSAGE(
|
||||
waited_ms >= SAME_KEY_MIN_WAIT_MS,
|
||||
"second call on the same connection returned too quickly -- the "
|
||||
"per-connection lock did not make it wait for the first call's "
|
||||
"abandoned worker to finish and release it");
|
||||
|
||||
/* By the time rc2's call could acquire the lock, the abandoned
|
||||
* worker's cleanup (free + counter increment, THEN lock release --
|
||||
* see interruptcall.c's abandoned path) must already have happened. */
|
||||
TEST_ASSERT_EQUAL_INT(1, atomic_load(&xcetest_test_abandoned_free_count));
|
||||
}
|
||||
|
||||
/* -- (ii) different connection keys run independently -- */
|
||||
|
||||
#define DIFF_KEY_SLOW_SLEEP_MS 900
|
||||
#define DIFF_KEY_FAST_SLEEP_MS 20
|
||||
#define DIFF_KEY_FAST_RC 77
|
||||
|
||||
static int diff_key_slow_fn(void* arg)
|
||||
{
|
||||
overlap_probe_t* p = arg;
|
||||
probe_enter(p);
|
||||
sleep_ms(DIFF_KEY_SLOW_SLEEP_MS);
|
||||
probe_leave(p);
|
||||
return SLOW_FN_RC;
|
||||
}
|
||||
|
||||
static int diff_key_fast_fn(void* arg)
|
||||
{
|
||||
overlap_probe_t* p = arg;
|
||||
probe_enter(p);
|
||||
sleep_ms(DIFF_KEY_FAST_SLEEP_MS);
|
||||
probe_leave(p);
|
||||
return DIFF_KEY_FAST_RC;
|
||||
}
|
||||
|
||||
static void test_different_connection_keys_run_independently(void)
|
||||
{
|
||||
static int conn_b, conn_c; /* two distinct opaque fake keys */
|
||||
overlap_probe_t probe = {0};
|
||||
pthread_t signaler;
|
||||
|
||||
g_test_break_out = 0;
|
||||
atomic_store(&xcetest_test_abandoned_free_count, 0);
|
||||
xcetest_install_sigint(test_sigint_handler);
|
||||
xcetest_set_active_connection(&conn_b);
|
||||
|
||||
TEST_ASSERT_EQUAL_INT(
|
||||
0, pthread_create(&signaler, NULL, send_sigint_after_delay, NULL));
|
||||
|
||||
int rc1 = xcetest_run_interruptible(diff_key_slow_fn, &probe);
|
||||
|
||||
TEST_ASSERT_EQUAL_INT(-EINTR, rc1);
|
||||
TEST_ASSERT_EQUAL_INT(0, pthread_join(signaler, NULL));
|
||||
|
||||
/* Reset the fake flag before the second call, same reasoning as
|
||||
* scenario (i) -- this call is on a DIFFERENT connection and must
|
||||
* complete normally. */
|
||||
g_test_break_out = 0;
|
||||
xcetest_set_active_connection(&conn_c);
|
||||
|
||||
int rc2 = xcetest_run_interruptible(diff_key_fast_fn, &probe);
|
||||
|
||||
TEST_ASSERT_EQUAL_INT(DIFF_KEY_FAST_RC, rc2);
|
||||
TEST_ASSERT_TRUE_MESSAGE(
|
||||
atomic_load(&probe.overlap_count) >= 1,
|
||||
"different connection keys must not serialize against each "
|
||||
"other -- expected the second call to run concurrently with the "
|
||||
"first call's still-running abandoned worker");
|
||||
|
||||
wait_for_abandoned_free_count(1);
|
||||
}
|
||||
|
||||
/* -- (iii) interrupted while waiting for a held lock: no worker spawned -- */
|
||||
|
||||
#define WAIT_KEY_SLOW_SLEEP_MS 900
|
||||
|
||||
static int wait_key_slow_fn(void* arg)
|
||||
{
|
||||
(void)arg;
|
||||
sleep_ms(WAIT_KEY_SLOW_SLEEP_MS);
|
||||
return SLOW_FN_RC;
|
||||
}
|
||||
|
||||
static _Atomic int g_wait_key_fn_invoked = 0;
|
||||
|
||||
static int wait_key_never_should_run_fn(void* arg)
|
||||
{
|
||||
(void)arg;
|
||||
atomic_store(&g_wait_key_fn_invoked, 1);
|
||||
return 0;
|
||||
}
|
||||
|
||||
static void test_interrupted_while_waiting_for_lock_never_spawns_worker(void)
|
||||
{
|
||||
static int conn_d;
|
||||
pthread_t signaler1, signaler2;
|
||||
struct timespec t0;
|
||||
|
||||
g_test_break_out = 0;
|
||||
atomic_store(&xcetest_test_abandoned_free_count, 0);
|
||||
atomic_store(&g_wait_key_fn_invoked, 0);
|
||||
xcetest_install_sigint(test_sigint_handler);
|
||||
xcetest_set_active_connection(&conn_d);
|
||||
|
||||
TEST_ASSERT_EQUAL_INT(
|
||||
0, pthread_create(&signaler1, NULL, send_sigint_after_delay, NULL));
|
||||
|
||||
int rc1 = xcetest_run_interruptible(wait_key_slow_fn, NULL);
|
||||
|
||||
TEST_ASSERT_EQUAL_INT(-EINTR, rc1);
|
||||
TEST_ASSERT_EQUAL_INT(0, pthread_join(signaler1, NULL));
|
||||
|
||||
/* conn_d's lock is now held by the still-running abandoned worker
|
||||
* (up to ~750ms left of its 900ms sleep). Reset the fake flag, then
|
||||
* arm a SECOND delayed SIGINT that fires while THIS next call is
|
||||
* still waiting to acquire that same lock -- well before the
|
||||
* abandoned worker could ever release it. */
|
||||
g_test_break_out = 0;
|
||||
TEST_ASSERT_EQUAL_INT(
|
||||
0, pthread_create(&signaler2, NULL, send_sigint_after_delay, NULL));
|
||||
|
||||
clock_gettime(CLOCK_MONOTONIC, &t0);
|
||||
int rc2 = xcetest_run_interruptible(wait_key_never_should_run_fn, NULL);
|
||||
long elapsed_ms = elapsed_ms_since(&t0);
|
||||
|
||||
TEST_ASSERT_EQUAL_INT(0, pthread_join(signaler2, NULL));
|
||||
TEST_ASSERT_EQUAL_INT(-EINTR, rc2);
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(
|
||||
0, atomic_load(&g_wait_key_fn_invoked),
|
||||
"a call interrupted while waiting for a held connection lock "
|
||||
"must never spawn a worker / invoke fn for itself");
|
||||
TEST_ASSERT_TRUE_MESSAGE(
|
||||
elapsed_ms < (WAIT_KEY_SLOW_SLEEP_MS - SIGNAL_DELAY_MS),
|
||||
"the interrupted lock-wait should return promptly, well before "
|
||||
"the held lock would ever naturally free");
|
||||
|
||||
/* The first call's abandoned worker is still running in the
|
||||
* background; wait for it to finish so this test leaves no live
|
||||
* thread behind when it returns. */
|
||||
wait_for_abandoned_free_count(1);
|
||||
}
|
||||
|
||||
void test_interruptcall(void)
|
||||
{
|
||||
UnitySetTestFile(__FILE__);
|
||||
RUN_TEST(test_normal_completion_returns_real_result);
|
||||
RUN_TEST(test_forced_abandonment_returns_eintr_and_worker_cleans_up);
|
||||
RUN_TEST(test_same_connection_key_serializes_overlapping_calls);
|
||||
RUN_TEST(test_different_connection_keys_run_independently);
|
||||
RUN_TEST(test_interrupted_while_waiting_for_lock_never_spawns_worker);
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue