From 4b2e053c3e9c57797e57d717d513fbbe486c9e21 Mon Sep 17 00:00:00 2001 From: Rongsong Shen Date: Tue, 8 Sep 2026 03:45:44 +0000 Subject: [PATCH] chore(wolf): mark bug-1743 Part 1 resolved, cross-link bug-1784-1790 bug-1743 (gitea#46's original tracking entry) still said Part 1 (dispatch-stops-at-8) was "UNRESOLVED/NOT ACTIONABLE, pending reporter info" -- stale since the 6-bug limit-plugin cascade (bug-1784 through bug-1790) fully resolved it without needing that info, and gitea#46 was just validated live and closed. Updates the entry with the full resolution chain and the closing live-verification evidence. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01GLfB5uWRdmx2WiZSFC7Fm7 --- .wolf/buglog.json | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/.wolf/buglog.json b/.wolf/buglog.json index f61e4f44..94f4234a 100644 --- a/.wolf/buglog.json +++ b/.wolf/buglog.json @@ -20277,12 +20277,19 @@ "gitea_issue": 46, "related_bugs": [ "bug-1673", - "bug-1556" + "bug-1556", + "bug-1784", + "bug-1785", + "bug-1786", + "bug-1787", + "bug-1788", + "bug-1789", + "bug-1790" ], "error_message": "Gitea issue #46 (\u8fd8\u9700\u8981\u4e00\u4e2aslots\u7684\u6982\u5ff5): host test01 shows NCPUS=16, MAX_SLOT=16 via `xcecli hosts list`, but of 9 submitted single-slot `sleep` jobs on queue `normal`, only 8 reach RUN and the 9th is stuck PEND -- dispatch stops at 8/16 host capacity. Secondary ask: `hosts list` should show slots used/free, not just NCPUS/MAX_SLOT.", "file": "src/scheduler/src/resource.c (sched_check_host_satisfy_:328-377), src/scheduler/src/plugins/limit.c (build_vres_name/per-user-queue-project virtual resource accounting:183-227,312-376), src/configs/resources.yaml (slots resource:4, consumable:true), src/configs/limits.yaml (ug1_normal_slots example, value:8, lines 41-49), src/frontend/apiserver/models/hosts.go (HostInfo.NumRun:87)", "root_cause": "Two independent findings. (1) Dispatch-stops-at-8 is most likely NOT a slot-counting bug in the scheduler -- MAX_SLOT genuinely gates per-host admission in sched_check_host_satisfy_() the same way `mem` does (per .kiro/steering/resource-accounting.md, reservation==usage for virtual resources like slots), but there is a SEPARATE, independent admission gate: cluster-wide usergroup+queue slot limits enforced by the scheduler's limit plugin (limit.c), configured in limits.yaml. This repo's example limits.yaml defines `ug1_normal_slots`: users in group ug1 on queue `normal` share a pool capped at exactly 8 (limits.yaml:41-49) -- an exact match to the reporter's queue (`normal`) and the observed 8/16 cutoff. This exact scenario (a queue/usergroup slot limit as an alternate explanation to a host-capacity bug) was explicitly predicted-but-unconfirmed in bug-1673 (filed against the related earlier report Gitea #40) and appears to be that prediction now reproduced. CAVEAT: limits.yaml/users.yaml in this repo look like sample/demo config (users.yaml's stock ug1 group only contains user1/user2, not `admin`) -- confirming this requires checking the reporter's actual deployed limits.yaml/users.yaml or their job's pend-reason output, neither of which was available this session (static analysis only). (2) The feature-request half (show slot usage in hosts list) is NOT starting from scratch: models.HostInfo.NumRun (hosts.go:87) already exists in the schema and is already an output column in `xcecli hosts list` (matching the reporter's own output), but `grep -rn \"\\.NumRun\\s*=\" apiserver --include=*.go` returns zero hits anywhere in the apiserver -- the field is never assigned and is permanently stuck at Go's zero value, which is exactly why the reporter's own output shows NUM_RUN=0 while 8 jobs are actually running on that host. Same for NumSSusp/NumUSusp.", - "fix": "PARTIALLY FIXED (2026-09-01). Part 2 (hosts-list slot visibility) is FIXED: added computeHostSlotUsage() (controllers/hosts.go) which scans store.Jobs' live ExecHosts JSONB (the same {\"host_slots\":[{\"host_name\",\"num_slots\"}],...} shape backend/convert_pure.go populates) and JobStatus bits (JOBSTAT[\"RUN\"]/[\"SSUSP\"]/[\"USUSP\"]) to count occupied slots per host, per sub-state -- counting SLOTS (not just job count) so the numbers are directly comparable to max_slot, matching the reporter's actual ask ('host\u4e0a\u9700\u8981\u4f53\u73b0\u7528\u4e86\u591a\u5c11slots'). processHostData() now overwrites row.Data[\"num_run\"/\"num_ssusp\"/\"num_ususp\"] with these live counts instead of leaving HostInfo's always-zero struct fields untouched. TDD: added TestProcessHostData_ComputesSlotUsageFromJobs (host_control_test.go) covering plain RUN, USUSP, a 2-slot SSUSP job (verifies slot-counting, not job-counting), a PEND job with no exec host (must not count), and a job on a different host (must not bleed across hosts). Confirmed red (num_run=0, want 6) before the fix, green after. Full apiserver suite passes with zero regressions.\n\nPart 1 (dispatch-stops-at-8 root cause) remains UNRESOLVED/NOT ACTIONABLE this session: confirming whether a real usergroup/queue limits.yaml entry is gating the reporter's deployment requires their actual live config or pend-reason output, neither available statically. No scheduler/limit.c code change was made pending that confirmation -- see root_cause for the open question to ask the reporter.", + "fix": "PARTIALLY FIXED (2026-09-01). Part 2 (hosts-list slot visibility) is FIXED: added computeHostSlotUsage() (controllers/hosts.go) which scans store.Jobs' live ExecHosts JSONB (the same {\"host_slots\":[{\"host_name\",\"num_slots\"}],...} shape backend/convert_pure.go populates) and JobStatus bits (JOBSTAT[\"RUN\"]/[\"SSUSP\"]/[\"USUSP\"]) to count occupied slots per host, per sub-state -- counting SLOTS (not just job count) so the numbers are directly comparable to max_slot, matching the reporter's actual ask ('host\u4e0a\u9700\u8981\u4f53\u73b0\u7528\u4e86\u591a\u5c11slots'). processHostData() now overwrites row.Data[\"num_run\"/\"num_ssusp\"/\"num_ususp\"] with these live counts instead of leaving HostInfo's always-zero struct fields untouched. TDD: added TestProcessHostData_ComputesSlotUsageFromJobs (host_control_test.go) covering plain RUN, USUSP, a 2-slot SSUSP job (verifies slot-counting, not job-counting), a PEND job with no exec host (must not count), and a job on a different host (must not bleed across hosts). Confirmed red (num_run=0, want 6) before the fix, green after. Full apiserver suite passes with zero regressions.\n\nPart 1 (dispatch-stops-at-8 root cause): FULLY RESOLVED (2026-09-07/08, follow-up session). Confirmed live (no reporter config needed after all -- the leading hypothesis was right, just broken at a much deeper layer than 'is a limit configured'): the entire usergroup/queue limit-matching machinery in the limit plugin was silently non-functional, a 6-bug cascade -- see bug-1784 (named usergroups never registered into the scheduler), bug-1785 (unconfigured scope matched no one instead of falling back to all), bug-1786 (election loop aborted after the first matched limit), bug-1787 (extra pointer indirection broke a resmgr handle lookup), bug-1788 (instantiate_limit() conflated not-found with a real error), and bug-1789/bug-1790 (even once matching worked, the limit's own consumption computation read the wrong request vector and always saw a 0 'wanted' value, so the numeric cap itself never bound). All six fixed and live-verified. Final confirmation (2026-09-08): fresh single-node deploy at current HEAD, 9 single-slot jobs submitted as user1 (a genuine ug1 member) on queue normal -- the exact repro shape from this issue -- scheduler trace shows limit1/ug1_slots/ug1_normal_slots all correctly matching and, for the first time, a real nonzero wanted value ('unsatisfy_ resource 35 total 3.000000 inuse 3.000000 uncomit 0.000000 wanted 1.000000' / 'unsatisfy_ unmet resource 35 (limit=limit1) shortfall 1.000000'), correctly PENDing jobs past the cap. Posted as a comment on gitea#46 and closed the issue (2026-09-08).", "tags": [ "scheduler", "slots", @@ -20301,10 +20308,11 @@ "related-to-bug-1673", "related-to-bug-1556", "partially-fixed", - "needs-reporter-confirmation" + "needs-reporter-confirmation", + "fixed" ], "occurrences": 1, - "last_seen": "2026-09-01T05:10:00Z" + "last_seen": "2026-09-08T00:00:00Z" }, { "id": "bug-1744",