54 lines
2.9 KiB
Markdown
54 lines
2.9 KiB
Markdown
---
|
|
id: "bug-1963"
|
|
status: "fixed"
|
|
status_updated: "2026-10-10"
|
|
status_note: ""
|
|
component: "scheduler"
|
|
severity: "wrong-result"
|
|
timestamp: "2026-10-10T10:16:09Z"
|
|
last_seen: "2026-10-10T10:16:09Z"
|
|
occurrences: 1
|
|
files: ["src/scheduler/src/host_iter.c"]
|
|
tags: ["scheduler", "host"]
|
|
related: []
|
|
fix_commits: ["2cd02cdc"]
|
|
gitea_issue: 394
|
|
gitea_synced_status: "fixed"
|
|
gitea_synced_hash: "36afb617d440"
|
|
---
|
|
|
|
# bug-1963: xce_scheduler host iteration offers unavailable or reconciling hosts as placement candidates
|
|
|
|
## Description
|
|
**Observed:** (source reading; follow-up of the 2026-10-10 P0 fixes (bug-1936..1957))
|
|
Suspected (reported by the bug-1939 fix, not verified): `sched_host_iter_next()` (`src/scheduler/src/host_iter.c`) skips admin-closed hosts since bug-1939 but not `XCE_WORKER_UNAVAIL` or `XCE_WORKER_RECONCILING`; other checks (`resource.c:704,887`, `core_session.c:136`) are accounting skips, not placement filters.
|
|
|
|
**Expected:** Unavailable and reconciling hosts get no new placements.
|
|
**Impact:** Possible dispatch to a dead or reconciling host (dispatch then fails and is retried).
|
|
**Found by:** follow-up of the 2026-10-10 P0 fixes (bug-1936..1957)
|
|
## Environment
|
|
- OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
|
|
- libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2)
|
|
- Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
|
|
- Build: asan (xce.build)
|
|
- Code: scheduler-framework @ 19be145d-dirty
|
|
- Deployment: any (found by source reading; confirmed by the RED test of the fix)
|
|
|
|
## Reproduction
|
|
1. Mark a host UNAVAIL in a scheduler fixture and check whether the iterator offers it.
|
|
- Frequency: deterministic (by code reading; confirmed by the RED test of the fix)
|
|
- Evidence: the code sites quoted in Description; no logs kept.
|
|
## Root cause
|
|
`sched_host_iter_next()` (`src/scheduler/src/host_iter.c`) skipped only admin-closed hosts; the UNAVAIL/RECONCILING checks elsewhere (`resource.c`, `core_session.c`) only affect accounting, and such a host keeps its last usage, so the resource check does not reject it. Pre-existing.
|
|
|
|
## Fix
|
|
`host_iter.c` skips `IS_WORKER_UNAVAIL` and `IS_WORKER_RECONCILING` hosts right after the closed-host check, for both drivers.
|
|
|
|
## Verification
|
|
- Test: `test_unavail_and_reconciling_hosts_get_no_new_jobs` (`src/scheduler/testsuite/test_host_unavail_reconciling.c`): RED `test_host_unavail_reconciling.c:200: FAIL: Expected 1 Was 2 (normal mode, unavailable host: serial job B must pend)`, GREEN after (both drivers, both status bits).
|
|
- Suites: `xce-build.sh all asan xce.build --incremental` on the merged branch, 2026-10-10: C suites Ok 157 / Fail 0; frontend Go tests ok (the build now also runs apiserver cmd, middleware, models and router)
|
|
- Live: n/a: unit/fixture-level verification; no live deployment was run
|
|
|
|
## References
|
|
- Fix commits: `2cd02cdc` (on `scheduler-framework`)
|
|
- Found as a follow-up of the 2026-10-10 P0 fixes (bug-1936..1957)
|