docs(eliminate-frontend-makefile): task 3.5 - verified regression suite parity

Pure verification, no code changes. Deployed a live single-node
cluster and ran tests/regression/run.sh against a Meson-built xcecli
binary, then again via the still-present Makefile path -- both
produced identical 28 passed/1 failed/1 skipped results. Reviewer
independently reproduced this three times and traced the one
pre-existing failure (jobs submit's job-ID format regex) through
source and git history to a 2026-05-07 commit, four months before
this spec existed -- logged as bug-1765 (not fixed, out of scope).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
This commit is contained in:
Rongsong Shen 2026-09-03 14:53:24 +00:00
parent 0da4165524
commit 87707ff18b
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
2 changed files with 18 additions and 1 deletions

View file

@ -53,7 +53,7 @@
- _Boundary: test_ha_bootstrap.c, test_ha_ctrlhello.c, apiserver/cmd/main.go, xcecli/cmd/main.go_
- _Depends: 1.1, 2_
- [ ] 3.5 (P) Verify the regression suite runs against a Meson-built binary with parity to the pre-migration baseline
- [x] 3.5 (P) Verify the regression suite runs against a Meson-built binary with parity to the pre-migration baseline
- Run `tests/regression/run.sh` directly (`XCE=<meson-built xcecli path> URL=<live apiserver>`) against a live apiserver built via `/xce-build apiserver`
- Observable: the run reports the same PASS/FAIL/SKIP counts as the documented pre-migration `make test-integration` baseline, using no Makefile-built artifact
- _Requirements: 4.1, 4.2, 4.3_

View file

@ -20826,6 +20826,23 @@
],
"occurrences": 1,
"last_seen": "2026-09-03T03:01:26Z"
},
{
"id": "bug-1765",
"timestamp": "2026-09-03T14:51:54Z",
"error_message": "tests/regression/run.sh's 'jobs submit' test FAILs deterministically (28 passed / 1 failed / 1 skipped instead of 29/0/0), with the cascading 'jobs kill' test SKIPped as a result -- reproduced identically regardless of which build system (Makefile vs Meson) produced the xcecli binary under test, confirming it predates and is unrelated to the eliminate-frontend-makefile migration.",
"root_cause": "CONFIRMED via source + git history. src/frontend/apiserver/controllers/jobs.go:1518 formats the submit-response message with a plain-digit '%d' placeholder ('Job <%d> is submitted...'), but src/frontend/xcecli/cmd/jobs/submit.go:500 post-processes that message client-side, replacing the '<%d>' substring via format.FormatJobID() (src/frontend/xcecli/common/format/jobid.go:40-57), which renders the packed 64-bit job ID as the dashed '{cluster}-{apiserver}-{timestamp}-{sequence}' form (e.g. '1-0-1788446837-0'). tests/regression/run.sh:143's assertion regex ('Job <[0-9]+> is submitted') was never updated for this dashed format. The CLI-side reformatting was introduced by commit d6008ead ('improve frontend jobid display', 2026-05-07) -- nearly four months before this discovery -- so the test has been silently broken (as a false FAIL, not a false PASS) since then, not a new regression.",
"fix": "NOT FIXED -- out of scope for the eliminate-frontend-makefile spec's task 3.5 (pure verification task, no file changes permitted). Fix would be updating run.sh:143's regex to accept the dashed job-ID format (e.g. 'Job <[0-9-]+> is submitted' or delegating to the same format.FormatJobID() pattern). A second, independent, lower-severity issue was found in the same file during this investigation: run.sh:155's assert_not_contains 'jobs list submit_time not dash' calls grep -qF \"- \" without a '--' separator, so GNU grep treats the leading-dash argument as an invalid option and errors out -- this coincidentally still yields the intended PASS (grep's error-exit is indistinguishable here from 'no match'), so it doesn't affect reported counts, but isn't actually testing what it appears to test.",
"tags": [
"regression-suite",
"test-infra",
"jobid-format",
"false-fail",
"pre-existing"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-09-03T14:51:54Z"
}
]
}