myxce/.wolf/bugs/bug-1963.md
Rongsong Shen f45b85c9e5
docs(buglog): record Gitea sync for bug-1958..bug-1967
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:51:33 +00:00

2.9 KiB

id status status_updated status_note component severity timestamp last_seen occurrences files tags related fix_commits gitea_issue gitea_synced_status gitea_synced_hash
bug-1963 fixed 2026-10-10 scheduler wrong-result 2026-10-10T10:16:09Z 2026-10-10T10:16:09Z 1
src/scheduler/src/host_iter.c
scheduler
host
2cd02cdc
394 fixed 36afb617d440

bug-1963: xce_scheduler host iteration offers unavailable or reconciling hosts as placement candidates

Description

Observed: (source reading; follow-up of the 2026-10-10 P0 fixes (bug-1936..1957)) Suspected (reported by the bug-1939 fix, not verified): sched_host_iter_next() (src/scheduler/src/host_iter.c) skips admin-closed hosts since bug-1939 but not XCE_WORKER_UNAVAIL or XCE_WORKER_RECONCILING; other checks (resource.c:704,887, core_session.c:136) are accounting skips, not placement filters.

Expected: Unavailable and reconciling hosts get no new placements. Impact: Possible dispatch to a dead or reconciling host (dispatch then fails and is retried). Found by: follow-up of the 2026-10-10 P0 fixes (bug-1936..1957)

Environment

  • OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
  • libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2)
  • Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
  • Build: asan (xce.build)
  • Code: scheduler-framework @ 19be145d-dirty
  • Deployment: any (found by source reading; confirmed by the RED test of the fix)

Reproduction

  1. Mark a host UNAVAIL in a scheduler fixture and check whether the iterator offers it.
  • Frequency: deterministic (by code reading; confirmed by the RED test of the fix)
  • Evidence: the code sites quoted in Description; no logs kept.

Root cause

sched_host_iter_next() (src/scheduler/src/host_iter.c) skipped only admin-closed hosts; the UNAVAIL/RECONCILING checks elsewhere (resource.c, core_session.c) only affect accounting, and such a host keeps its last usage, so the resource check does not reject it. Pre-existing.

Fix

host_iter.c skips IS_WORKER_UNAVAIL and IS_WORKER_RECONCILING hosts right after the closed-host check, for both drivers.

Verification

  • Test: test_unavail_and_reconciling_hosts_get_no_new_jobs (src/scheduler/testsuite/test_host_unavail_reconciling.c): RED test_host_unavail_reconciling.c:200: FAIL: Expected 1 Was 2 (normal mode, unavailable host: serial job B must pend), GREEN after (both drivers, both status bits).
  • Suites: xce-build.sh all asan xce.build --incremental on the merged branch, 2026-10-10: C suites Ok 157 / Fail 0; frontend Go tests ok (the build now also runs apiserver cmd, middleware, models and router)
  • Live: n/a: unit/fixture-level verification; no live deployment was run

References

  • Fix commits: 2cd02cdc (on scheduler-framework)
  • Found as a follow-up of the 2026-10-10 P0 fixes (bug-1936..1957)