55 lines
4.1 KiB
Markdown
55 lines
4.1 KiB
Markdown
---
|
|
id: "bug-1960"
|
|
status: "fixed"
|
|
status_updated: "2026-10-10"
|
|
status_note: ""
|
|
component: "controller"
|
|
severity: "wrong-result"
|
|
timestamp: "2026-10-10T10:16:02Z"
|
|
last_seen: "2026-10-10T10:16:02Z"
|
|
occurrences: 1
|
|
files: ["src/controller/job.c", "src/frontend/apiserver/controllers/jobs.go"]
|
|
tags: ["controller", "apiserver", "queue", "admin-control"]
|
|
related: ["bug-1938"]
|
|
fix_commits: ["b239db5f", "1ff068f6", "4f579041"]
|
|
gitea_issue: 391
|
|
gitea_synced_status: "fixed"
|
|
gitea_synced_hash: "1715dc1d9fd5"
|
|
---
|
|
|
|
# bug-1960: closed queues accept submissions: controller never checks, apiserver skips the default queue
|
|
|
|
## Description
|
|
**Observed:** (source reading; follow-up of the 2026-10-10 P0 fixes (bug-1936..1957))
|
|
Reported by the bug-1938 fix: `handle_jobsubmit()` (`src/controller/job.c`) never checks `QUEUE_STAT_CLOSED`; the apiserver's check in `preCheckJobRequest()` runs only when a queue name is given, so a submission without one goes to `normal` unchecked, and non-REST submit paths (libxce, harness) are not checked at all.
|
|
|
|
**Expected:** A submission to a closed queue (explicit or default) is rejected with a clear error on every submit path.
|
|
**Impact:** Admin queue close silently ignored for default-queue and non-REST submissions.
|
|
**Found by:** follow-up of the 2026-10-10 P0 fixes (bug-1936..1957)
|
|
## Environment
|
|
- OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
|
|
- libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2)
|
|
- Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
|
|
- Build: asan (xce.build)
|
|
- Code: scheduler-framework @ 19be145d-dirty
|
|
- Deployment: any (found by source reading; confirmed by the RED test of the fix)
|
|
|
|
## Reproduction
|
|
1. Close queue `normal`.
|
|
2. Submit a job without a queue (REST) or through xce_test: it is accepted.
|
|
- Frequency: deterministic (by code reading; confirmed by the RED test of the fix)
|
|
- Evidence: the code sites quoted in Description; no logs kept.
|
|
## Root cause
|
|
`handle_jobsubmit()` (`src/controller/job.c`) only checked that the queue field was non-empty, never the queue's existence or `QUEUE_STAT_CLOSED`, so single and array submissions from libxce/harness (and REST without a queue name) reached closed or unknown queues; `preCheckJobRequest()` (`controllers/jobs.go`) checked only a named queue. Pre-existing.
|
|
|
|
## Fix
|
|
Controller (`b239db5f`): new `jobsubmit_check_queue()` (in-memory read under the qmgr/queue read locks) rejects an unknown queue or `lost_and_found` with `-E_XCE_BAD_QUEUE` and a closed queue with new `-E_XCE_QUEUE_CLOSED` (10016, `include/xce/errnox.h`, string in `lib/utils.c`) before the hook, the in-flight registry and any WAL append; replay/apply paths unchanged, so committed submits still apply. Apiserver (`1ff068f6`): the default queue gets the same existence/closed checks. Apiserver (`4f579041`): `SubmitJob` maps the two codes to `backend.ErrQueueClosed`/`ErrQueueInvalid` (checked against errnox.h at compile time) and the submit/restart handlers answer 400 naming the queue (covers a close racing the apiserver check). Behaviour change: a submission without a queue is rejected when the cluster has no `normal` queue.
|
|
|
|
## Verification
|
|
- Test: Controller: `test_jobsubmit_closed_queue_rejected`, `test_jobsubmit_unknown_queue_rejected`, `test_jobsubmit_lost_and_found_rejected` (`src/controller/testsuite/test_jobsubmit_validate.c`): RED `test_jobsubmit_validate.c:370: FAIL: Expected -10016 Was -5` (and -10008), GREEN after; `test_ace_job_submit_closed_queue_still_applies` guards replay. Apiserver: `jobs_test.go:137: submission to the closed default queue: err = <nil>` RED, GREEN after; `TestSubmitRCError`, `TestJobHandler_SubmitQueueRejectedByController` failed to build before (no typed errors), GREEN after.
|
|
- Suites: `xce-build.sh all asan xce.build --incremental` on the merged branch, 2026-10-10: C suites Ok 157 / Fail 0; frontend Go tests ok (the build now also runs apiserver cmd, middleware, models and router)
|
|
- Live: n/a: unit/fixture-level verification; no live deployment was run
|
|
|
|
## References
|
|
- Fix commits: `b239db5f`, `1ff068f6`, `4f579041` (on `scheduler-framework`)
|
|
- Found as a follow-up of the 2026-10-10 P0 fixes (bug-1936..1957)
|