scheduler-sim-diagnostics, scheduler-sim-live-allocation, xce-test-cli-ux and eliminate-frontend-makefile had every sub-task checked but their parent boxes and spec.json phase still read tasks-generated. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
201 lines
40 KiB
Markdown
201 lines
40 KiB
Markdown
# Implementation Plan
|
|
|
|
- [x] 1. Foundation: shared libraries, process-level fixes, and test harness
|
|
- [x] 1.1 Build the flag-descriptor table format and getopt_long-based parsing/rejection core
|
|
- Define the flag-descriptor structure (short form, long form, concept name, required flag, default description) and a builder that produces a getopt_long-compatible option table from it
|
|
- Implement the shared rejection path: an unrecognized or malformed flag prints the offending flag plus the command's usage summary to stderr and returns a nonzero result
|
|
- Write the unit test first (red), then implement until it passes (green): a fake command's descriptor table correctly builds its option table, and both an unrecognized flag and a flag missing its required value are rejected with the right message
|
|
- Observable completion: running the new unit test suite shows the descriptor-driven rejection producing the same messages for a made-up two-flag test command regardless of which flag is malformed, and confirms the rejection text lands on stderr, not stdout
|
|
- _Requirements: 2.1, 2.2, 2.3, 2.4, 3.1_
|
|
- _Boundary: Flag Parsing Layer_
|
|
|
|
- [x] 1.2 Build the flag typo-suggestion utility and the generated help-flag-block renderer
|
|
- Add a small edit-distance comparison so an unrecognized flag that closely matches exactly one flag in the descriptor table is named as the likely intended one in the rejection message
|
|
- Add a renderer that produces the flag-listing portion of help text (short form, long form, required/optional, default) directly from the descriptor table
|
|
- Observable completion: the unit test suite shows a single close-match flag typo producing a "did you mean" suggestion, a non-close-match producing no suggestion, and the renderer producing a correctly formatted block for a multi-flag test table
|
|
- _Requirements: 6.1, 6.2, 6.3, 6.4, 8.2, 10.1, 10.2, 10.3_
|
|
- _Boundary: Flag Parsing Layer_
|
|
- _Depends: 1.1_
|
|
|
|
- [x] 1.3 Correct the SIGINT installation and expose an interruption accessor
|
|
- Replace the process's existing signal-based SIGINT installation with an explicit installation that clears the automatic-restart behavior, so a blocking read reliably returns an interrupted result instead of silently retrying
|
|
- Expose a small accessor over the existing interrupt flag so the interruptible-call adapter (built next, in 1.4/1.5) can poll it without depending on the flag's storage details
|
|
- Observable completion: a small standalone test that sends an interrupt signal to the process during a blocking read confirms the read returns immediately with an interrupted result rather than continuing to block
|
|
- _Requirements: 9.9_
|
|
|
|
- [x] 1.4 Build the interruptible-call adapter's heap-block-ownership mechanism
|
|
- Implement a wrapper that runs a given blocking call on a detached worker thread while the caller polls the interruption accessor from 1.3 on a short interval
|
|
- Implement the atomic ownership handoff so that whichever side (the polling caller or the worker) is left holding the call's heap-allocated state is the one that frees it, with no path that frees it from both sides
|
|
- Observable completion: the unit test suite shows a normal (uninterrupted) call returning its real result and freeing cleanly, and a forced-timeout call returning immediately with no double-free and no leak (verified under the project's leak/ASAN tooling)
|
|
- _Requirements: 1.1, 1.2, 1.3, 1.4_
|
|
- _Boundary: Interruptible Call Adapter_
|
|
- _Depends: 1.3_
|
|
|
|
- [x] 1.5 Add the per-connection lock registry to the interruptible-call adapter
|
|
- Add a small registry, keyed by connection identity, that serializes any two calls sharing a connection so an abandoned call's worker thread is never running concurrently with a newly issued call on the same connection
|
|
- Make lock acquisition itself interruptible, so a new call waiting on a connection still held by a prior abandoned call can itself be cancelled
|
|
- Observable completion: the unit test suite shows two overlapping calls against the same fake connection identity (one abandoned, one issued immediately after) never executing concurrently, while two calls against different connection identities run independently
|
|
- _Requirements: 1.1, 1.5_
|
|
- _Boundary: Interruptible Call Adapter_
|
|
- _Depends: 1.4_
|
|
|
|
- [x] 1.6 Add process-level help and version discovery
|
|
- Ensure invoking the process with a help or version flag prints the appropriate output and exits without starting the interactive/scripted session
|
|
- Extend the process-level help output to describe the in-session help command as the way to list and learn available commands
|
|
- Observable completion: invoking the process with the help flag prints usage text mentioning the in-session help command, and invoking with the version flag prints version information, both without entering the REPL
|
|
- _Requirements: 4.1, 4.2, 4.3_
|
|
- _Boundary: Process Entry_
|
|
|
|
- [x] 1.7 Build the black-box CLI test harness
|
|
- Add a subprocess/pty-driven test harness capable of launching the built client, sending input lines (including partial input followed by a tab keypress and raw signals), and capturing its stdout/stderr/exit code separately
|
|
- Observable completion: a smoke test using the new harness launches the client, sends a trivial command, and confirms the expected output arrives on the correct stream with the expected exit code
|
|
- _Boundary: Testing Infrastructure_
|
|
|
|
- [x] 2. Core: independent new capabilities and standalone fixes
|
|
- [x] 2.1 (P) Build the destructive-command confirmation gate and its session-scoped toggle
|
|
- Implement a confirmation check that prompts the operator on stderr before proceeding, returns a distinct "not run" outcome on decline, and treats an interrupt during the prompt the same as a decline
|
|
- Add a bypass flag usable per invocation and a session-scoped toggle (mirroring the existing continue-on-error toggle's style) that both default to requiring confirmation and both reset at process start
|
|
- Ensure the confirmation check is skipped automatically when input is not from an interactive session
|
|
- Observable completion: the unit test suite shows a simulated non-interactive session skipping the prompt, a simulated decline and a simulated interrupt both producing the "not run" outcome, and the toggle switching prompting on and off
|
|
- _Requirements: 9.1, 9.2, 9.3, 9.4, 9.6, 9.7, 9.8, 9.9_
|
|
- _Boundary: Confirmation Gate_
|
|
- _Depends: 1.1, 1.3_
|
|
|
|
- [x] 2.2 (P) Build the JSON list-output helper library
|
|
- Implement shared helpers for building a JSON array of row objects and printing it, usable by any listing command's existing per-row data
|
|
- Observable completion: the unit test suite shows a multi-row, multi-field test payload serializing to valid JSON with no field truncated regardless of value length
|
|
- _Requirements: 5.1, 5.3_
|
|
- _Boundary: JSON List Formatter_
|
|
|
|
- [x] 2.3 (P) Build command-name tab completion scoped to the active context
|
|
- Implement completion that, when the session is interactive, offers only command names valid in the currently active context, completes an unambiguous partial match in full, and lists candidates without altering the input line when more than one match exists
|
|
- Observable completion: a black-box test (using the harness from 1.7) drives a partial command name plus a tab keypress in two different contexts and confirms the completion offered differs by context and matches the single-match/multiple-match behavior
|
|
- _Requirements: 7.1, 7.2, 7.3, 7.4_
|
|
- _Boundary: Tab Completion_
|
|
- _Depends: 1.7_
|
|
|
|
- [x] 2.4 (P) Build the in-session source command with missing-file and cycle detection
|
|
- Implement a command that reads and executes another file's lines in order, reusing the existing rc-file loading primitive, reporting an explicit error (rather than silently doing nothing) when the file does not exist or cannot be read
|
|
- Add cycle detection using a canonicalized in-flight path stack so a file that sources itself, directly or transitively, is rejected with an error instead of hanging
|
|
- Ensure a failing line inside the sourced file follows the session's current abort-on-failure/continue-on-error setting, and that nested sourcing (one file sourcing another, non-cyclically) composes correctly
|
|
- Observable completion: the unit/integration test suite shows a two-file mutual-cycle case rejected with an error, a missing-file case rejected with an error, and a nested non-cyclic case executing every line from both files
|
|
- _Requirements: 13.1, 13.2, 13.3, 13.4, 13.5, 13.6_
|
|
- _Boundary: Source Command_
|
|
|
|
- [x] 2.5 (P) Fix bug-1711's display and text defects
|
|
- Correct the misspelled unrecognized-worker-status text and the misspelled default launch-type label (the malformed help-not-found grammar is fixed in 3.5, alongside that same code path's broader rework)
|
|
- Left-justify the resource-listing command's text columns to match the worker-listing command's convention, and label/prefix its flags column as hexadecimal
|
|
- Format the job-listing command's submit/start/finish time columns using the existing human-readable time formatting instead of raw epoch seconds, split its combined task-count column into two correctly labeled columns, and add a raw-state fallback for its ambiguous run-plus-exit state
|
|
- Replace the configuration-listing command's silently empty group/config sections with either their real contents or an explicit not-implemented marker
|
|
- Observable completion: running each affected command against a small fixture shows the corrected text/formatting for every defect listed above
|
|
- _Requirements: 11.1, 11.3, 11.4, 11.5, 11.6, 11.7, 11.8_
|
|
- _Boundary: Bug-1711 Fixes_
|
|
|
|
- [x] 2.6 (P) Fix bug-1557's join maxmem default
|
|
- Correct the simulated worker-join command's default memory value to the codebase's current bytes-canonical convention
|
|
- Observable completion: joining a simulated worker without specifying memory reports the corrected byte-scale default instead of the stale value
|
|
- _Requirements: 12.1_
|
|
- _Boundary: Bug-1557 Fix_
|
|
|
|
- [x] 3. Integration: migrate existing commands onto the new foundations
|
|
- [x] 3.1 Migrate every flag-accepting command's parsing to the shared validation library
|
|
- Convert every command currently parsing flags via the older mechanisms (including the one command using a fully hand-rolled long-flag parser) to the shared descriptor-table-driven parsing built in 1.1/1.2, so every one uniformly rejects an unrecognized or malformed flag and gains a long-form equivalent for each concept
|
|
- While migrating the queue-control command, initialize its action selection to an explicit unset sentinel and reject the invocation if none of its mutually-exclusive action flags was given, matching its sibling host-control command's existing pattern
|
|
- Observable completion: every migrated command rejects a deliberately invalid flag with a usage summary and, where applicable, a suggestion, and the queue-control command rejects an invocation given none of its action flags instead of transmitting an indeterminate value
|
|
- _Requirements: 2.1, 2.2, 2.3, 2.4, 6.1, 6.2, 6.3, 6.4, 10.1, 10.2, 10.3_
|
|
- _Boundary: Flag Parsing Layer_
|
|
- _Depends: 1.1, 1.2_
|
|
|
|
- [x] 3.2 Wrap every blocking request with the interruptible-call adapter
|
|
- Route every command's blocking request through the interruptible-call adapter built in 1.4/1.5, so an interrupt during any of them abandons the wait immediately, reports the command as failed, and leaves the active connection and context intact
|
|
- Ensure the session never exits as a side effect of a single interrupt keypress, and that an interrupt at an empty prompt simply discards the current input line
|
|
- Observable completion: interrupting a command that is blocked waiting on a request returns control to the prompt immediately (well before the request's own timeout) and the same connection accepts a subsequent command normally
|
|
- _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5_
|
|
- _Boundary: Interruptible Call Adapter_
|
|
- _Depends: 1.4, 1.5, 3.1_
|
|
|
|
- [x] 3.3 Wire the five destructive commands to the confirmation gate
|
|
- Gate the job-kill, shutdown, queue-close/inactivate, host-close, and configuration-upgrade actions behind the confirmation gate built in 2.1, using each command's now-correctly-parsed flags to decide whether a given invocation counts as destructive
|
|
- Add the bypass flag to each of these commands via the shared flag-descriptor table
|
|
- Observable completion: each of the five destructive actions prompts interactively by default, proceeds without prompting when the bypass flag or the session toggle is used, and proceeds without prompting when input is not interactive; separately, confirm that a reversal invocation (queue-active/open, host-open) and the fake-worker job-lifecycle commands never prompt at all, since they are excluded from the destructive set
|
|
- _Requirements: 9.1, 9.2, 9.3, 9.4, 9.5, 9.6, 9.7, 9.8, 9.9_
|
|
- _Boundary: Confirmation Gate_
|
|
- _Depends: 2.1, 3.1_
|
|
|
|
- [x] 3.4 Add JSON output mode to the three listing commands
|
|
- Bring the worker-listing, job-listing, and resource-listing commands onto the shared flag-descriptor library from 1.1 for the first time (none currently accepts any flag), adding a consistently named output-format flag that uses the JSON list-output library from 2.2 to emit every field present in each command's human-readable table, unabbreviated
|
|
- Leave each command's existing human-readable output unchanged when the new flag is not given
|
|
- Observable completion: running each of the three listing commands with the new flag against a fixture with a long field value produces valid JSON containing every table field with no truncation, while the default (no flag) invocation still prints the existing table
|
|
- _Requirements: 5.1, 5.2_
|
|
- _Boundary: JSON List Formatter_
|
|
- _Depends: 1.1, 2.2, 2.5_
|
|
|
|
- [x] 3.5 Generate and consolidate every command's help content
|
|
- Regenerate each command's flag-listing help text from the shared descriptor table (built in 1.2), and revise each command's remaining help text to include a purpose description, at least one example invocation, which contexts it is valid in when shared across contexts, and any related command or scripting side effect
|
|
- Where a shared command's behavior differs by active context, describe the current context's specific behavior rather than a generic description
|
|
- Ensure a help lookup for a name not found in the active context reports it as not found there without checking other contexts, and correct the existing malformed not-found message text
|
|
- Document the join command's corrected default from 2.6, and the JSON output flag added in 3.4, in their respective commands' help text
|
|
- Observable completion: looking up help for a representative command in each context shows a description, full flag documentation, an example, and (where applicable) related-command notes; looking up help for one of the three listing commands explicitly shows how to request its JSON output mode; and looking up an unrecognized name shows the corrected not-found message with no other-context hint
|
|
- _Requirements: 5.4, 8.1, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7, 11.2, 12.2_
|
|
- _Boundary: Help Renderer_
|
|
- _Depends: 1.2, 3.1, 2.6, 3.4_
|
|
|
|
- [x] 3.6 Audit and correct stdout/stderr stream routing across existing error paths
|
|
- Review every command's remaining error paths not already covered by the shared flag-rejection path built in 1.1 — request failures and internal errors — and ensure diagnostic/error text goes to stderr while a command's successful result data goes to stdout only
|
|
- Confirm the same routing holds identically whether stdout and stderr are both attached to a terminal or redirected to separate destinations (a script or pipe)
|
|
- Observable completion: redirecting only stdout to a file while triggering a request failure or an internal error on a representative command shows the diagnostic text still appearing on the terminal while the redirected file contains no diagnostic text; separately, running a successful command with only stdout redirected shows its result data captured in the redirected file with nothing sent to stderr
|
|
- _Requirements: 3.1, 3.2, 3.3, 3.4_
|
|
- _Boundary: Command Dispatcher_
|
|
- _Depends: 3.1, 3.2, 3.3, 3.4, 3.5_
|
|
|
|
- [x] 4. Validation: end-to-end verification
|
|
- [x] 4.1 Verify Ctrl-C cancellation end-to-end
|
|
- Confirm interrupting a command blocked on a real request abandons the wait well before its own timeout, reports the command as failed, and leaves the session's context and connection usable
|
|
- Confirm that issuing a second command on the same connection immediately after an interrupt does not race the abandoned request, and that a command on a different connection is unaffected
|
|
- Observable completion: a live end-to-end run demonstrates both the immediate-cancellation behavior and the no-overlap behavior on the same connection
|
|
- _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5_
|
|
- _Depends: 3.2, 1.7_
|
|
- _Done (requirements 1.1-1.5 MET): run twice independently (implementer + reviewer), each against a separately-deployed single-node live controller (`/xce-deploy-ha`-style, `deploy-ha.sh`), freezing the controller mid-RPC with `SIGSTOP` to create a genuine blocked-request condition and sending real `SIGINT` to the real `xce_test` process. AC 1.1: a `hello` blocked against the frozen controller (confirmed hung, no reply) was abandoned in ~0.00026-0.00028s of wall-clock time after SIGINT (both runs closely agreed), vs. the ~10s `cbuf_deadline_10s` RPC timeout. AC 1.2: a partial, un-submitted input line was discarded cleanly on SIGINT, prompt redisplayed, session immediately usable. AC 1.3: the interrupted command reported `hello: Interrupted system call` on stderr and `$?` == -4 (-EINTR), the same failure signaling as any other failing command. AC 1.4: the process survived 3 separate SIGINTs across the run, exiting only on an explicit `quit`. AC 1.5: a second `hello` issued immediately after the first abandonment, on the same connection, with the controller still frozen, itself blocked (queuing on the per-connection lock) rather than racing or returning stale data; after `SIGCONT`, a third `hello` on that same connection completed normally, proving full reuse after two abandonments. The different-connection-independence sub-clause was validated via existing white-box coverage (`test_different_connection_keys_run_independently`, `test_interruptcall.c`) rather than a second live controller node — both implementer and reviewer independently read that test and confirmed it exercises the same connection-key abstraction production uses (`active_ctx->ctx`) with a real abandonment on one key and asserted concurrency on a distinct key, and design.md confirms the mechanism is entirely client-side, so a second live node would add no further assurance. No source files were changed; both deploys were fully cleaned up (processes killed, deploy dirs removed, confirmed via `pgrep`)._
|
|
|
|
- [x] 4.2 Verify the confirmation gate end-to-end
|
|
- Confirm each of the five destructive commands prompts by default, respects its bypass flag and the session toggle, and auto-skips under non-interactive input
|
|
- Confirm that a real interrupt signal delivered while the confirmation prompt is displayed is observed as an interruption and reported as not run, not as failed
|
|
- Observable completion: a live end-to-end run against each destructive command demonstrates the prompt, the bypass, the toggle, the non-interactive auto-skip, and the real-signal-during-prompt case
|
|
- _Requirements: 9.1, 9.2, 9.3, 9.4, 9.5, 9.6, 9.7, 9.8, 9.9_
|
|
- _Depends: 3.3, 1.7_
|
|
- _Done (requirements 9.1-9.9 MET): run twice independently (implementer + reviewer), each against separately-deployed live single-node clusters (one with a real worker), for all five destructive commands (jobkill, shutdown, qctrl -i/-c, hostctrl -c, upgradeconfig). Default prompt + decline reports "<cmd>: not run (declined)" on stderr only, never "failed" (AC1/AC2/AC8). A real SIGINT delivered while blocked inside the prompt's read produces the IDENTICAL "not run (declined)" text for all five (AC9) -- concretely contrasted against task 4.1's own finding that Ctrl-C on an ordinary (non-destructive) command instead reports "<cmd>: Interrupted system call" as a failure, proving AC9's exception to Requirement 1.3 is real, not assumed. `--yes` and non-interactive (piped, non-tty) input both skip the prompt and proceed straight to the RPC for all five (AC3/AC4), including full real round trips for jobkill (a real submitted job actually signaled) and upgradeconfig (the real generated cluster.bin actually resubmitted, 3 times, safely idempotent) and, via two disposable throwaway controllers, shutdown itself (actually terminated for real to prove the bypass path executes all the way, not just "reached the RPC call"). The session toggle `set -y`/`set +y` (AC6/AC7) works for all five once issued from the top-level context (`set` is registered only in `top_commands[]`, not `client_commands[]`/`worker_commands[]`/`scheduler_commands[]` -- a cosmetic UX gap, not a Requirement 9 defect, confirmed by both runs). AC5's exact enumeration verified live both ways: `qctrl -a`/`-o`, `hostctrl -o`, and worker-context `jstart` never prompt; `qctrl -i`/`-c` and `hostctrl -c` do. No defect found in the confirmation gate itself. Two genuine but pre-existing, out-of-boundary defects were found along the way and logged rather than fixed: bug-1796 (cmd_qctrl/cmd_hostctrl/cmd_jobkill's RPC-error perror() calls print stale/misleading errno text, root-caused and reproduced independently by the reviewer, confirmed unrelated to the interrupt adapter or confirmation gate) and bug-1797 (cmd_qctrl leaks its strdup'd queue name on the RPC-error return path, ASAN-confirmed). No source files were changed; both sets of deploys were fully cleaned up._
|
|
|
|
- [x] 4.3 Verify source's interaction with the confirmation gate
|
|
- Confirm a destructive command inside a sourced file, run within an otherwise-interactive session, still prompts for confirmation unless the session toggle is set, proving the gate's interactive check is based on the top-level session rather than on whether the current line came from a file
|
|
- Observable completion: a live end-to-end run sources a file containing a destructive command from an interactive session and observes the prompt appear, then observes it skipped once the toggle is set
|
|
- _Requirements: 13.7_
|
|
- _Depends: 2.4, 3.3, 1.7_
|
|
- _Done (requirement 13.7 MET): run twice independently (implementer + reviewer), each against separately-deployed live single-node controllers, sourcing a script file containing a destructive `qctrl -i` line from an otherwise-interactive session. With the confirmation toggle at its default, the prompt appeared on stderr with wording byte-identical to a directly-typed invocation of the same command, and decline reported "qctrl: not run (declined)" -- proving the gate fires for a sourced-file line exactly as it would for a typed one. After `set -y` (issued at the top-level context), re-sourcing the same file produced zero prompt text, and `set +y` afterward restored prompting for a directly-typed invocation, confirming no lingering session-state confusion. The reviewer additionally confirmed this structurally, not just behaviorally: `confirm_destructive()`'s only gating checks are `confirm_is_skip_set()` and `!xcetest_is_interactive()` (a read-only accessor over `g_interactive`, set once at startup from `isatty(STDIN_FILENO)` and never reassigned), and the only per-source-invocation state that exists (`g_source_depth`, used solely for cycle detection) has zero references anywhere near the gate -- there is no code path capable of distinguishing a sourced line from a typed one for this decision. Also verified the bonus interaction flagged by task 3.3's Implementation Note: with `set -e` active, a sourced file's declined destructive line does NOT trip abort-on-failure (a subsequent `echo` line in the same file still ran), confirming decline's `return 0` per `confirm.h`'s contract. No defect found. The already-logged bug-1797 (`cmd_qctrl`'s RPC-error-path queue-name leak) recurred via the `source` call path in both runs -- confirmed to be the identical pre-existing leak, not a new one; occurrence count bumped in buglog.json. No source files were changed; both deploys were fully cleaned up._
|
|
|
|
- [x] 4.4 Verify process-level help, tab completion, and consolidated help content end-to-end
|
|
- Confirm the help/version flags and in-session help behavior from 1.6 and 3.5, and the tab-completion behavior from 2.3, hold together in the full built client, not just in isolation
|
|
- Observable completion: a black-box run confirms the help flag's output describes the in-session help command, a representative command's help output includes its generated flag block and hand-authored example, and tab completion in two different contexts offers the expected candidates
|
|
- _Requirements: 4.1, 4.2, 4.3, 7.1, 7.2, 7.3, 7.4, 8.1, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7, 11.2_
|
|
- _Depends: 1.6, 2.3, 3.5, 1.7_
|
|
- _Done (requirements 4.1-4.3, 7.1-7.4, 8.1-8.7, 11.2 MET): no live cluster needed (purely client-side). Existing per-mechanism coverage from tasks 1.6/2.3/3.5 was confirmed solid, but no test chained context-switching + tab completion + help lookup together in one continuous session -- a genuine "holds together, not just in isolation" gap. Added one new black-box test, `test_help_completion_and_context_switching_hold_together_in_one_session` (test_test_cli.c), covering: top-level vs. client-context double-Tab on the identical "w" prefix producing different, context-correct candidate sets; an unambiguous Tab-completion executed as a real (safely no-op, pre-RPC-validated) command; a mid-session `help jobkill` showing its purpose description, generated flag block, and hand-authored example; a further switch to worker context; and a `help jobkill` lookup there correctly reported not-found with the corrected grammar, leaking no client-context text. Both implementer and reviewer independently reproduced all three observable-completion criteria via their own standalone driver runs against the real binary, and the reviewer additionally cross-checked the completion candidate sets directly against test.c's four command tables. No defect found; no product code changed, only the one new permanent test (1331/1331 tests, 0 ASAN diagnostics)._
|
|
|
|
## Implementation Notes
|
|
- Audited all 139 `printf(` call sites in test.c; 33 were genuine diagnostic/warning/failure-path messages misrouted to stdout (not covered by the already-correct `flagspec_reject`/`perror` mechanisms) and were converted to `fprintf(stderr, ...)` — concentrated in `cmd_expect_jobs`/`cmd_expect_job`/`cmd_wait_primary` (entirely unrouted; these predate the flagspec migration since they take positional args, not flags), plus `get_setting`, `cmd_help`'s not-found path (a shared `FILE* out` var, stderr only when not-found; bare `help`'s listing and a successful lookup stay stdout), `cmd_ctrlstatus`, `cmd_jobrun`, `cmd_jobsub`, `cmd_jobstart`/`jobfinish`/`jobfail`, `resupdate_report_one`/`cmd_resupdate`, `cmd_set`. Deliberately left as stdout (Requirement 3 doesn't apply): `cmd_resval`'s intentional always-succeeds "not found" design, and the entire async worker/scheduler RPC-handler subsystem (`handle_notify`/`run_job`/etc.) — these are inbound-request callbacks dispatched outside the REPL's `cmd_*` table, not "a command's own diagnostic or result data." Found but explicitly left alone as pre-existing and out of THIS task's scope (stream routing, not return-code correctness): `cmd_resupdate` still unconditionally `return 0`s even on an outright RPC failure or a per-resource rejection — worth a follow-up bug entry if this area is touched again. Requirement 3.4 (redirection-invariance) holds by construction: no code branches a stream CHOICE on `isatty`/`g_interactive` anywhere in the file. (task 3.6)
|
|
- `command_t` gained `flagspec`/`flagspec_count`/`usage_tail` fields; `cmd_help` now composes description(`usage` head) -> generated `flagspec_render_help()` block -> `usage_tail` (example/related notes) for all 22 flag-accepting commands, with hand-written per-flag lines deleted from their `USAGE_X`/`USAGE_X_TAIL` constants so the flagspec table is the only place a flag's short/long/required/default lives. Since the four `command_t` tables (`top_commands` etc.) appear before the 22 `flagspec_t` tables in the file, each is forward-declared with an explicit literal size (e.g. `static const flagspec_t jobkill_flags[3];`) rather than the previous `#define X_NFLAGS (sizeof(x_flags)/sizeof(x_flags[0]))` self-computing macro — the literal must now be kept in sync by hand, though any mismatch between the forward declaration and the real definition is a compiler "conflicting types" build error, never a silent drift (review verified this is standard, portable C, and hand-counted several tables' entries against their literals). A follow-up could reorder the flagspec_t tables above the command_t tables to restore the sizeof-based macro and drop this literal-maintenance requirement entirely, but it wasn't required for this task. `join_flags`/`JOIN_FLAGS_COUNT` (a separate translation unit, join_worker_args.c/.h) uses the standard `extern const T arr[N]` header/source split instead, since `extern` against a `static` definition isn't valid — no literal-sync risk there. The five destructive commands' help text now names the confirmation gate and `-y`/`set -y` bypass, scoped accurately (`qctrl`'s note covers only `-i`/`-c`, `hostctrl`'s only `-c`, matching Requirement 9.5's exact enumeration, not their full flag sets). (task 3.5)
|
|
- `workerlist`/`joblist`/`reslist` all take the identical long-only `--json` flag (via each command's own `flagspec_t` table); JSON rows emit RAW numeric values for fields the table displays in a humanized/formatted form (`flags` as an int not a hex string, timestamps as raw epoch seconds not `format_job_time`'s human string, `maxmem`/`maxswap`/`maxtmp` as raw bytes not `format_bytes_human`'s string) — deliberate, since Requirement 5.1 asks for "the same fields" (value present) not identical display formatting, and a raw value is strictly more script-consumable. The field-mapping logic was extracted to a new pure module, `listrow_json.c`/`.h` (no RPC/networking/global-state coupling), following the `joblist_display.c` extraction precedent, with full unit coverage in `test_listrow_json.c` (a new standalone `listrow_json_unit_test` meson target, not counted in the main suite's test total) including the fractional-value (`cpufactor` 12.34) real cJSON-print-then-reparse round-trip the task-2.2 note asked for. `joblist_query` gained a 5th `json_mode` parameter; its other two callers (`expect-jobs`/`expect-job`) now pass a literal `0` and are behaviorally unchanged (verified: both `print_table` and `json_mode` false skips both branches exactly as before). Live-cluster E2E visual inspection of real JSON output was not done — same "genuinely untestable without a live cluster" situation task 2.5 already documented for these commands' inlined fetch+print bodies; coverage instead comes from the extracted module's unit tests plus a black-box test proving the flag parses/executes without a flag-parse error. (task 3.4)
|
|
- `src/tools/xce-build.sh controller asan xce.build --incremental` writes its build dir at `src/xce.build/` (relative to the script's own location), not a repo-root `xce.build/` — meson logs are at `src/xce.build/meson-logs/testlog.txt`. (task 1.1)
|
|
- New Unity test files need two wiring points, not just meson.build's source list: also register an `extern void test_<module>(void);` + `TEST(test_<module>);` in `src/controller/testsuite/controller_test.c`, mirroring the existing `test_rcfile` registration, or the suite silently never runs. (task 1.1)
|
|
- Confirmed (review traced it concretely): a declined destructive command inside a `source`d file does NOT trip `set -e`'s abort-on-failure, since decline maps to `return 0` per confirm.h's own documented contract, not `execute_line`'s `rc<0` failure check. This only matters when a human is actively present declining (never a blind/unattended failure mode), and any later destructive line re-prompts independently — but task 4.3 should add an explicit `set -e` + `source <file with a declined destructive command>` case to prove/document this interaction, since it's currently untested. (task 3.3)
|
|
- **Safety-critical finding**: wrapping RPC calls in an abandonable worker thread means `leave_ctx()` (called by `up`/`quit`) can now run while a previously-interrupted command's worker is still using the connection — before this task the REPL was fully synchronous so this UAF couldn't exist. Fixed via `xcetest_wait_connection_idle(conn_key)` (interruptcall.c), which blocks on the SAME per-connection semaphore the adapter already uses, then immediately releases it, proving no worker is active before teardown proceeds. Call it before any future connection-teardown path, not just `leave_ctx()`, if one is added later. A timing-dependent black-box test alone could NOT reliably prove this race (ran clean 20/20 even with the fix disabled) — the reliable proof came from a white-box test directly exercising the wait function against a real abandoned worker. When a fix's failure mode is a sub-millisecond race, don't trust a black-box test's clean run as RED-phase evidence; get a deterministic white-box reproduction instead. (task 3.2, bug-1795)
|
|
- All 17 flag-accepting commands migrated to `flagspec_t`+`getopt_long` (review verified essentially every command line-by-line via git diff context, not just a sample — zero regressions to any existing short flag's letter/value-format/default). `cmd_walcompare`'s migration needs a leading `+` in its optstring (`getopt_long`'s "stop at first non-option") to preserve its pre-migration "flags only before positionals" behavior — without it, argv permuting would change which tokens are treated as flags vs. member paths. Only 7 of the 17 migrated commands got dedicated black-box tests (`qctrl`, `jobinfo`, `jobkill`, `hostctrl`, `setpend`, `resupdate`, `walcompare`); the other 10 (`hello`, `jobrun`, `jobsub`, `resval`, `jobckpt`, `runtasks`, `jobstart`, `jobfinish`, `jobfail`, `join_worker_parse_attrs`) have no valid-flag regression test at all beyond this review's manual line-by-line diff check — worth a follow-up smoke test per command if this area gets touched again. (task 3.1)
|
|
- `cmd_join_worker`'s sibling fields `maxswap`/`maxtmp` have the exact same stale-MB-scale bug as bug-1557's `maxmem` (1024/2048 bytes instead of KB/MB-appropriate values) but were deliberately left unfixed here: `USAGE_JOIN_WORKER` currently documents these exact stale values accurately, so fixing only the code would create a new code/docs mismatch outside this task's mandate. Worth a coordinated follow-up bug fixing both fields' defaults and their help text together. A new `test_join_worker_sibling_defaults_unchanged` test locks in today's values so that follow-up isn't confused with this task's maxmem-specific assertions. (task 2.6)
|
|
- When a fix touches a pure function stuck inside `test.c` (no networking/REPL/global-state dependency), extract it into its own module with a standalone meson `test()` target rather than settling for a throwaway/manual verification — this codebase already has two precedents for exactly this (`resval_typeresolve.c`, `misc.c`'s runusage helpers; now also `joblist_display.c` for `worker_status_str`/`job_state_str`/`format_job_time`). `test.c` itself is never linked into `controller_unit_test`, so this extract-to-standalone-module pattern is the only way to get permanent automated coverage for such functions — a scratch/deleted verification harness is not sufficient and will be sent back on review. `cmd_reslist`/`joblist_query`/`cmd_getconf`'s table-printing bodies remain genuinely untestable without a live cluster (fetch+print are inlined together, confirmed on review) and stay uncovered by permanent tests for now. (task 2.5)
|
|
- `cmd_source` translates `rcfile_load`'s raw positive "aborted" return to `-1` rather than forwarding it verbatim, because `preprocess_and_run` only treats a *negative* return as failure (`if (rc < 0)`) — forwarding the raw positive value would let an enclosing `source` line silently read a transitively-aborted nested source as success. No test yet exercises this specific nested-abort-propagation case (all 3 required scenarios pass either way) — worth a dedicated test in Validation phase (4.3 or nearby). (task 2.4)
|
|
- libedit (this project's pinned NetBSD libedit 3.1) provides GNU-readline-compatible `rl_attempted_completion_function`/`rl_completion_matches` verbatim — confirmed against the actual linked header, not assumed. An ambiguous match needs a SECOND consecutive Tab to list candidates (standard readline/bash double-Tab convention) — later black-box tests using Tab must account for this. Also: `test_test_cli.c`'s `clitest_spawn()` disabling kernel ECHO also silently suppressed libedit's own completion feedback in this libedit build (bug-1792) — use the new `clitest_spawn_ex(sess, path, argv, disable_echo)` with `disable_echo=false` for any future test that needs to observe libedit UI behavior (completion, etc.), not just raw command output. (task 2.3)
|
|
- `listfmt_json.c`'s tests only exercise power-of-two double values (Unity's `TEST_ASSERT_EQUAL_DOUBLE` is disabled project-wide, matching `test_host_runusage.c`/`test_e2e_capnp.c`); cJSON's real print path uses shortest-round-trippable-double formatting so fractional values are fine, but task 3.4 should add a real fractional-value (e.g. a resource value like 12.34) round-trip test once actual command fields are wired in, since nothing here proves that end-to-end yet. (task 2.2)
|
|
- `confirm_destructive()`'s EINTR-as-decline (9.9) works because it's a single, non-looping `fgets()` call — a naive "robust I/O" retry-on-EINTR wrapper would silently defeat this requirement. `g_interactive` is exposed via a new `xcetest_is_interactive()` accessor (declared in `confirm.h`, mirroring `xcetest_interrupted()`'s pattern) for task 3.3 to reuse. Task 3.3 must short-circuit around `confirm_destructive()` entirely (not pass a parameter into it) when a command's own `--yes` flag was given — the function has no bypass-flag parameter by design. (task 2.1)
|
|
- The `test_test_cli.c` black-box harness (`clitest_*` functions) is file-static and single-file by design (matching design.md's File Structure Plan and this directory's own no-split norm) — tasks 2.3, 3.5, 4.1-4.4 should add their black-box test functions directly into this same file rather than a new one. `clitest_wait_for_text()` is generic (arbitrary needle/stream) and safe to call repeatedly mid-session; `clitest_send_signal()` targets the child's real pid via `kill()`. (task 1.7)
|
|
- `argp` has no built-in `-h` alias (its default short help flag is `-?`) — Requirement 4.1 literally names `-h`, so it needed an explicit `OPTION_HIDDEN` option added, not just verification of existing defaults. Task 1.6's new `test_xcetest_process_entry.c` already covers "`-h`/`--help`/`--version` exit without entering the REPL" end-to-end against the real binary; task 1.7's `test_test_cli.c` should either skip re-adding this exact scenario or consolidate the two rather than duplicating coverage. (task 1.6)
|
|
- The per-connection lock is a POSIX semaphore (`sem_t`), not a `pthread_mutex_t` — a mutex was tried first but rejected on review (POSIX classifies a non-owning-thread `pthread_mutex_unlock()` as UB for every mutex type, not just error-checking ones, and this design needs the worker thread to release a lock the caller thread acquired). `sem_t` has no ownership concept and is fully POSIX-defined for this pattern. See bug-1791. (task 1.5)
|
|
- `interruptcall.c`'s forced-abandonment test proves `-EINTR` via return-value disambiguation against a slow `fn`, not a direct wall-clock elapsed-time assertion — sound today (poll interval is 50ms, matching design.md) but a future regression widening the poll interval wouldn't be caught unless the test also asserts an elapsed-time bound. Non-blocking; noted for task 4.1's end-to-end Ctrl-C validation to consider. (task 1.4)
|
|
- After task 1.3's SIGINT/EINTR fix alone (before task 3.2 wires the REPL loop to check `xcetest_interrupted()`), a real Ctrl-C at `xce_test`'s top-level interactive prompt now causes the process to EXIT (readline's EINTR-driven NULL return is currently treated the same as EOF) rather than gracefully redisplaying the prompt. This is expected/deferred, not a regression — no current test exercises that path — but task 3.2's implementer must fix it as part of wiring the REPL loop, not just the RPC call sites. (task 1.3)
|