feat(xce-test-e2e-primitives): session verdict + exit code + abort/continue + comment handling (task 1.1)
This commit is contained in:
parent
64ab210cbf
commit
481e197928
2 changed files with 164 additions and 7 deletions
90
.kiro/specs/xce-test-e2e-primitives/tasks.md
Normal file
90
.kiro/specs/xce-test-e2e-primitives/tasks.md
Normal file
|
|
@ -0,0 +1,90 @@
|
|||
# Implementation Plan
|
||||
|
||||
> TDD is mandatory (project rule): add the failing test/scenario first, then the change (red → green → refactor).
|
||||
> Almost all work is in the single test client TU `src/controller/testsuite/test.c`, so tasks are sequential by design; only the soak conversion touches a different file. Behavioral tests are driven through `xce_test` itself (piped scripts), gating on its exit code.
|
||||
|
||||
- [ ] 1. Foundation: verdict, loop wrapper, ergonomics
|
||||
- [x] 1.1 Session failure tracking + process exit code + abort/continue + comment/blank handling
|
||||
- Add file-scope state (`g_session_failed`, `g_continue_on_error`, `g_last_status`) and a `preprocess_and_run(line)` wrapper used by the main read loop: ignore blank and `#`-comment lines; run normal lines via `execute_line`; update `g_last_status`/`g_session_failed` from the result; in default mode stop on first failure, under `continue-on-error` keep going. `main` returns `g_session_failed ? 1 : 0`. Extend `cmd_set` with a `continue-on-error` toggle.
|
||||
- Observable: a piped script with a bogus/failing command makes `xce_test` exit non-zero; an all-valid script exits 0; with `set continue-on-error` a failing line does not stop later lines but the final exit is still non-zero; `#`/blank lines are ignored.
|
||||
- _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 6.1, 6.2_
|
||||
- _Boundary: VerdictLoopWrapper_
|
||||
- [ ] 1.2 `echo` command
|
||||
- Add `cmd_echo` (table-registered) that prints its argument message and returns success.
|
||||
- Observable: `echo hello` emits `hello` and the command succeeds (exit unaffected).
|
||||
- _Requirements: 6.3_
|
||||
- _Boundary: VerdictLoopWrapper_
|
||||
- _Depends: 1.1_
|
||||
|
||||
- [ ] 2. Shell-escape
|
||||
- [ ] 2.1 `!<cmd>` runs a shell command and feeds `$?`
|
||||
- In `preprocess_and_run`, when the first non-whitespace char is `!`, run the remainder via `system()`; set `g_last_status = WEXITSTATUS(rc)` (non-zero on `system` failure); a non-zero status is a command failure (subject to 1.1's abort/continue + exit code).
|
||||
- Observable: `!true` succeeds and sets `$?`=0; `!false` fails the session and sets `$?`≠0; a script can `!kill ...` / `!bash start.sh ...` to drive node lifecycle.
|
||||
- _Requirements: 2.1, 2.2, 2.3, 2.4_
|
||||
- _Boundary: ShellEscape_
|
||||
- _Depends: 1.1_
|
||||
|
||||
- [ ] 3. Auto-variable substitution
|
||||
- [ ] 3.1 Substitute the fixed auto-vars `$?`, `$jobid`, `$jobcount`
|
||||
- Add `substitute_autovars(line)` (run before tokenizing in `preprocess_and_run`): replace whole-token `$?`→`g_last_status`, `$jobid`→`g_last_jobid`, `$jobcount`→`g_last_jobcount`. A reference to an unset `$jobid`/`$jobcount` fails the command (no garbage substitution). No other variable syntax is recognized.
|
||||
- Observable: after a command, `echo $?` prints the prior status; referencing `$jobid` before any successful `jobsub` fails that command rather than substituting an empty/garbage value.
|
||||
- _Requirements: 3.1, 3.3, 3.5, 3.6_
|
||||
- _Boundary: AutoVars_
|
||||
- _Depends: 1.1_
|
||||
|
||||
- [ ] 4. Correct job-submission reporting (bug-786) + `$jobid` capture
|
||||
- [ ] 4.1 `cmd_jobsub` reports success only on a clean ack with a valid JobId
|
||||
- Gate the `jobsubmit ok: JobId: <id>` report on a successful reply status AND a well-formed JobId; on success set `g_last_jobid` (feeds `$jobid`, 3.2). Otherwise print a typed error (redirect / no-master / not-ready / timeout) and return `<0`. Never print "ok" with an uninitialized/garbage id (`63-15--1-...`).
|
||||
- Observable: a submit that does not get a clean ack returns failure and does NOT set `$jobid`; `jobsub` never prints success with a malformed id; a successful submit sets `$jobid` to the acked JobId.
|
||||
- _Requirements: 7.1, 7.2, 7.3, 3.2_
|
||||
- _Boundary: JobsubFix_
|
||||
- _Depends: 1.1, 3.1_
|
||||
|
||||
- [ ] 5. Job-count capture and assertions
|
||||
- [ ] 5.1 `cmd_joblist` sets `$jobcount`; extract a shared count/contains helper
|
||||
- Refactor `cmd_joblist` to expose an internal helper that returns the current job count and tests membership of a JobId; set `g_last_jobcount` after a `joblist`.
|
||||
- Observable: after `joblist`, `echo $jobcount` prints the number of jobs returned; the helper is reusable by the assertions.
|
||||
- _Requirements: 3.4_
|
||||
- _Boundary: JoblistCount_
|
||||
- _Depends: 1.1, 3.1_
|
||||
- [ ] 5.2 `expect-jobs N` assertion
|
||||
- Add `cmd_expect_jobs`: pass when the current job count equals `N`; otherwise print expected vs actual and return `<0`.
|
||||
- Observable: `expect-jobs <correct>` succeeds; `expect-jobs <wrong>` fails the session and prints expected/actual.
|
||||
- _Requirements: 5.1, 5.2_
|
||||
- _Boundary: Assertions_
|
||||
- _Depends: 5.1_
|
||||
- [ ] 5.3 `expect-job <jobid>` assertion
|
||||
- Add `cmd_expect_job`: pass when the JobId is present in the job list; otherwise print the missing id and return `<0`. Works with `$jobid` substitution.
|
||||
- Observable: `expect-job $jobid` passes for a submitted job; `expect-job <absent>` fails the session and prints the missing id.
|
||||
- _Requirements: 5.3, 5.4_
|
||||
- _Boundary: Assertions_
|
||||
- _Depends: 5.1, 3.1, 4.1_
|
||||
|
||||
- [ ] 6. Wait for elected primary
|
||||
- [ ] 6.1 `wait-primary [timeout_ms]`
|
||||
- Add `cmd_wait_primary`: loop a cheap, failover-aware client op (the existing `joblist`/probe path via `active_ctx`) until it succeeds or the timeout elapses (monotonic clock); default timeout when the arg is omitted; return 0 on success, `<0` on timeout.
|
||||
- Observable: against a healthy cluster `wait-primary` returns immediately; after a primary kill it blocks until a new primary serves, then succeeds; with no primary within the timeout it fails.
|
||||
- _Requirements: 4.1, 4.2, 4.3, 4.4_
|
||||
- _Boundary: WaitPrimary_
|
||||
- _Depends: 1.1_
|
||||
|
||||
- [ ] 7. Reference E2E driver
|
||||
- [ ] 7.1 Convert `test-commit-durability.sh` to use the primitives
|
||||
- Replace the prose-grep + sanitizer-filter logic with piped `xce_test` scripts using `wait-primary`, `jobsub`/`$jobid`, `expect-jobs`/`expect-job`, and `!`-shell-escape for the primary kill/restart; gate each phase on the `xce_test` exit code. Preserve the rolling-kill + zero-loss coverage; add a `!`-driven cold-restart-all + `expect-jobs` step.
|
||||
- Observable: the converted soak runs end-to-end and reports pass/fail via `xce_test` exit codes (no joblist prose parsing); rolling-kill zero-loss coverage is retained; the cold-restart step asserts all committed jobs recover.
|
||||
- _Requirements: 8.1, 8.2, 8.3_
|
||||
- _Boundary: ReferenceDriver_
|
||||
- _Depends: 1.1, 2.1, 4.1, 5.2, 5.3_
|
||||
- [ ] 7.2 Run the converted soak on a live `/xce-deploy-ha` cluster
|
||||
- Execute the converted soak against a real 3-node cluster (real fsync-able fs) and confirm pass.
|
||||
- Observable: soak passes all rounds with the exit-code verdict; manual environment prerequisite (live cluster), same status as other live-E2E tasks.
|
||||
- _Requirements: 8.2, 8.3_
|
||||
- _Boundary: ReferenceDriver_
|
||||
- _Depends: 7.1_
|
||||
|
||||
## Implementation Notes
|
||||
- All of tasks 1–6 modify the single TU `src/controller/testsuite/test.c`; sequential by design (no `(P)`). Task 7 touches `src/tools/test-commit-durability.sh` but depends on the primitives, so it is also ordered last.
|
||||
- Exit-code verdict is only trustworthy with a clean process exit; run the soak with `ASAN_OPTIONS=detect_leaks=0` (existing convention). Client leak cleanup is out of scope (separate item).
|
||||
- Build dir is `src/xce.build` (not `src/build`). Fast incremental build: `cd src && nix develop --command bash -c "meson compile -C xce.build xce_test"`. Binary: `src/xce.build/controller/xce_test`. Behavioral tests = piped scripts gating on `$?` with `ASAN_OPTIONS=detect_leaks=0`; no controller needed (an unknown command is a convenient failing command). `xce_test` is NOT a meson-registered `test()`, so `meson test --suite controller` does not exercise it.
|
||||
- Pre-existing unrelated failure: `meson test --suite controller` reports `test_demotion_replay_yields_committed_only` FAIL (in `controller_unit_test` / `test_commit_barrier_reply.c`) — from the in-flight `ha-commit-durability` feature, in a binary that does NOT compile `test.c`. Not caused by this spec; ignore it for verdicts here.
|
||||
- 1.1: `preprocess_and_run()` is the single chokepoint where the leading-`!` shell-escape (2.1) and `$`-autovar substitution (3.1) plug in. File-scope `g_session_failed`/`g_continue_on_error`/`g_last_status` added; `main` returns `g_session_failed ? 1 : 0`; `set continue-on-error` is a process-global argc==2 special-case in `cmd_set` (not a per-context `settings[]` entry).
|
||||
|
|
@ -117,6 +117,21 @@ extern int cmd_setpend(int, char*[]);
|
|||
extern int cmd_jobinfo(int, char*[]);
|
||||
static int worker_house_keeping(void*);
|
||||
|
||||
/* Session verdict / loop state (xce-test-e2e-primitives task 1.1).
|
||||
*
|
||||
* g_session_failed set to 1 the first time any command in the session
|
||||
* fails; drives the process exit code in main().
|
||||
* g_continue_on_error when set (via `set continue-on-error`), a failing
|
||||
* command does not stop the read loop; the session still
|
||||
* exits non-zero at the end.
|
||||
* g_last_status status of the most recently executed command
|
||||
* (0 == success, non-zero == failure). Future tasks
|
||||
* expose this as the `$?` auto-variable.
|
||||
*/
|
||||
static int g_session_failed = 0;
|
||||
static int g_continue_on_error = 0;
|
||||
static int g_last_status = 0;
|
||||
|
||||
static idgen_ctx_t idctx;
|
||||
delaymgr_t delaymgr;
|
||||
|
||||
|
|
@ -385,6 +400,51 @@ int execute_line(char* line)
|
|||
return rc;
|
||||
}
|
||||
|
||||
/* Main-loop wrapper around execute_line that implements the session verdict
|
||||
* (xce-test-e2e-primitives task 1.1):
|
||||
* - blank / whitespace-only lines and `#`-comment lines are ignored;
|
||||
* - normal lines run via execute_line; g_last_status / g_session_failed are
|
||||
* updated from the result;
|
||||
* - in default (abort) mode a failure stops the loop; under
|
||||
* g_continue_on_error the loop keeps running but the failure is recorded.
|
||||
*
|
||||
* Returns 0 to keep the read loop running, non-zero to request that the loop
|
||||
* stop (abort-on-failure). The leading `!` shell-escape and `$`-substitution
|
||||
* are intentionally handled in later tasks (2.1 / 3.1); this function is the
|
||||
* single chokepoint where they will be added.
|
||||
*/
|
||||
int preprocess_and_run(char* line)
|
||||
{
|
||||
char* p = line;
|
||||
int rc;
|
||||
|
||||
/* Skip leading whitespace to classify the line. */
|
||||
while ((*p == ' ') || (*p == '\t')) {
|
||||
p++;
|
||||
}
|
||||
|
||||
/* Ignore blank / whitespace-only lines and `#` comments. */
|
||||
if ((*p == '\0') || (*p == '#')) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
rc = execute_line(line);
|
||||
|
||||
g_last_status = (rc < 0) ? (rc) : 0;
|
||||
|
||||
if (rc < 0) {
|
||||
SET_ERROR(-rc);
|
||||
g_session_failed = 1;
|
||||
if (!g_continue_on_error) {
|
||||
return 1; /* abort: stop the read loop */
|
||||
}
|
||||
} else {
|
||||
add_history(line);
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
int cmd_help(int argc, char* argv[])
|
||||
{
|
||||
int i;
|
||||
|
|
@ -1791,6 +1851,15 @@ int cmd_set(int argc, char* argv[])
|
|||
return 0;
|
||||
}
|
||||
|
||||
/* Session verdict toggle (xce-test-e2e-primitives task 1.1):
|
||||
* `set continue-on-error` keeps the read loop running after a failing
|
||||
* command (the session still exits non-zero). It is not a per-context
|
||||
* setting, so it is handled here ahead of the key/value path. */
|
||||
if ((argc == 2) && (strcmp(argv[1], "continue-on-error") == 0)) {
|
||||
g_continue_on_error = 1;
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (argc != 3) {
|
||||
printf("set <key> <value>\n");
|
||||
return -EINVAL;
|
||||
|
|
@ -2520,14 +2589,12 @@ int main(int argc, char* argv[])
|
|||
|
||||
s = buf;
|
||||
|
||||
rc = execute_line(s);
|
||||
if (rc < 0) {
|
||||
SET_ERROR(-rc);
|
||||
} else {
|
||||
add_history(s);
|
||||
}
|
||||
rc = preprocess_and_run(s);
|
||||
free(buf);
|
||||
if (rc) {
|
||||
break; /* abort-on-failure requested loop stop */
|
||||
}
|
||||
}
|
||||
|
||||
return 0;
|
||||
return g_session_failed ? 1 : 0;
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue