56 lines
3.4 KiB
Markdown
56 lines
3.4 KiB
Markdown
---
|
|
id: "bug-1902"
|
|
status: "fixed"
|
|
status_updated: "2026-10-02"
|
|
status_note: ""
|
|
component: "harness"
|
|
severity: "noise"
|
|
timestamp: "2026-10-01T11:22:43Z"
|
|
last_seen: "2026-10-02T04:14:22Z"
|
|
occurrences: 2
|
|
files: ["src/harness/testsuite/test_test_cli.c"]
|
|
tags: ["harness", "test", "flaky", "pty"]
|
|
related: []
|
|
fix_commits: ["daeb9041"]
|
|
repro_attempts: "0/5 reruns, 2026-10-01 (4 targeted meson reruns + reviewer full run)"
|
|
---
|
|
|
|
# bug-1902: harness pty test test_source_missing_file_rejected_with_error timed out once after quit
|
|
|
|
## Description
|
|
**Observed:** during task 4.3 of spec job-array-one-based-index, one run of the harness suite under ASAN failed in the pty CLI test:
|
|
```
|
|
test_test_cli.c: test_source_missing_file_rejected_with_error: FAIL: xce_test did not exit within the harness's safety timeout after 'quit'
|
|
```
|
|
**Expected:** after `quit`, `xce_test` exits within the test's safety timeout and the test passes, as it does in every other run.
|
|
**Impact:** noise: an intermittent false failure of the harness suite; no product impact observed.
|
|
**Found by:** test (harness suite run during task 4.3, 2026-10-01)
|
|
|
|
## Environment
|
|
- OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
|
|
- libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2)
|
|
- Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
|
|
- Build: asan (src/build.jai)
|
|
- Code: scheduler-framework @ 1812a80d-dirty (task 4.3 in progress)
|
|
- Deployment: none (harness_unit_test, pty-driven xce_test, no controller)
|
|
|
|
## Reproduction
|
|
1. Build the harness (`bash src/tools/xce-build.sh harness asan build.jai --jobs 8 --incremental`).
|
|
2. Run the pty test binary 16 at a time to load the machine (32 cores here): `for i in $(seq 16); do ASAN_OPTIONS=detect_leaks=0 src/build.jai/harness/harness_unit_test > run$i.log 2>&1 & done; wait`, three rounds.
|
|
3. Some runs fail a different `test_test_cli.c` test each time with `xce_test did not exit within the harness's safety timeout after 'quit'`.
|
|
- Frequency: 12/48 runs under 16-way parallel load, 2026-10-01 (1/6 and 1/127-test full suite runs earlier, unloaded reruns pass)
|
|
- Evidence: not kept (reproduces on demand with the steps above)
|
|
|
|
## Root cause
|
|
Not an `xce_test` hang. `clitest_wait_for_exit()` and `clitest_wait_for_text()` (`src/harness/testsuite/test_test_cli.c`) did not measure time: they added `CLITEST_POLL_SLICE_MS` (100 ms) per loop iteration. After `quit`, `xce_test` closes its pty before it becomes reapable, so `poll()` returns at once on every call (hang-up, nothing to read) and the loop spun: the 5000 ms budget ran out within milliseconds while the ASAN child was still tearing down, which takes longer under load. Measured with a temporary wall-clock probe over 2287 exits under load: real time from `quit` to reap at most 6 ms, while the counter reached up to 58800 ms (9 exits would have tripped the 5 s limit).
|
|
|
|
## Fix
|
|
Both waits use a `CLOCK_MONOTONIC` deadline (`clitest_now_ms()`), and `clitest_pump_once()` naps 10 ms when `poll()` wakes with nothing to read, so a hung-up pty no longer spins. Test-harness only; `xce_test` is unchanged.
|
|
|
|
## Verification
|
|
- Test: the load reproduction above is the RED (12/48 runs failing); GREEN after the fix: 0/48 and 0/48 in two further rounds of the same 16-way parallel load
|
|
- Suites: harness meson suite 7/7 OK (asan, src/build.jai)
|
|
- Live: n/a (test-harness timing; no product code changed)
|
|
|
|
## References
|
|
- Related: none
|