2.9 KiB
| id | status | status_updated | status_note | component | severity | timestamp | last_seen | occurrences | files | tags | related | fix_commits | gitea_issue | gitea_synced_status | gitea_synced_hash | ||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| bug-1963 | fixed | 2026-10-10 | scheduler | wrong-result | 2026-10-10T10:16:09Z | 2026-10-10T10:16:09Z | 1 |
|
|
|
394 | fixed | 36afb617d440 |
bug-1963: xce_scheduler host iteration offers unavailable or reconciling hosts as placement candidates
Description
Observed: (source reading; follow-up of the 2026-10-10 P0 fixes (bug-1936..1957))
Suspected (reported by the bug-1939 fix, not verified): sched_host_iter_next() (src/scheduler/src/host_iter.c) skips admin-closed hosts since bug-1939 but not XCE_WORKER_UNAVAIL or XCE_WORKER_RECONCILING; other checks (resource.c:704,887, core_session.c:136) are accounting skips, not placement filters.
Expected: Unavailable and reconciling hosts get no new placements. Impact: Possible dispatch to a dead or reconciling host (dispatch then fails and is retried). Found by: follow-up of the 2026-10-10 P0 fixes (bug-1936..1957)
Environment
- OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
- libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2)
- Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
- Build: asan (xce.build)
- Code: scheduler-framework @ 19be145d-dirty
- Deployment: any (found by source reading; confirmed by the RED test of the fix)
Reproduction
- Mark a host UNAVAIL in a scheduler fixture and check whether the iterator offers it.
- Frequency: deterministic (by code reading; confirmed by the RED test of the fix)
- Evidence: the code sites quoted in Description; no logs kept.
Root cause
sched_host_iter_next() (src/scheduler/src/host_iter.c) skipped only admin-closed hosts; the UNAVAIL/RECONCILING checks elsewhere (resource.c, core_session.c) only affect accounting, and such a host keeps its last usage, so the resource check does not reject it. Pre-existing.
Fix
host_iter.c skips IS_WORKER_UNAVAIL and IS_WORKER_RECONCILING hosts right after the closed-host check, for both drivers.
Verification
- Test:
test_unavail_and_reconciling_hosts_get_no_new_jobs(src/scheduler/testsuite/test_host_unavail_reconciling.c): REDtest_host_unavail_reconciling.c:200: FAIL: Expected 1 Was 2 (normal mode, unavailable host: serial job B must pend), GREEN after (both drivers, both status bits). - Suites:
xce-build.sh all asan xce.build --incrementalon the merged branch, 2026-10-10: C suites Ok 157 / Fail 0; frontend Go tests ok (the build now also runs apiserver cmd, middleware, models and router) - Live: n/a: unit/fixture-level verification; no live deployment was run
References
- Fix commits:
2cd02cdc(onscheduler-framework) - Found as a follow-up of the 2026-10-10 P0 fixes (bug-1936..1957)