lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(scheduler): record each pend reason once per job (bug-1927)

Browse source 
The scheduler appended one NO_RESOURCE/MATCH_FAIL entry per host per
placement plan, so a pending parallel job carried plans x hosts copies
(~93k entries in the 1000-host bounds test). libxce gains
add_job_preason_once() with a per-context hash index keyed on
(host, resource, code) that keeps first-occurrence order; the
scheduler's three builders use it. add_job_preason() is unchanged for
other callers. The parjob equivalence test now expects each short host
once; the bounds test runs in ~2 s instead of ~19 s.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-09 06:30:58 +00:00
parent 6c3393aa86
commit 9f9b63dd24
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
10 changed files with 517 additions and 17 deletions
Download patch file Download diff file Expand all files Collapse all files

39
src/include/preason.h
View file

@ -12,6 +12,45 @@ extern void erase_job_preason(xce_job_preason_t*);
extern int add_job_preason(xce_job_preason_t*, int, char*, char*);
extern xce_pendreason_t* make_pendreason(uint64_t);
/*
* De-duplicating pend-reason builder (bug-1927).
*
* add_job_preason() always appends, which is right for callers that build a
* reason list entry by entry (tests, the harness setpend command). A caller
* that re-evaluates the same hosts many times while building one job's
* reasons -- the scheduler's allocator, once per placement plan -- uses
* add_job_preason_once() instead, so each distinct (host, code) host reason
* and each distinct (host, resource, code) resource reason appears once.
* Entries keep the order of their first add. A NULL resource-reason host is
* stored as "" (as add_job_preason() does) and is the same key as "".
*
* xce_job_preason_t is the wire-mapped record (controller.capnp
* JobPendReason), so the lookup index lives beside it, owned by the
* builder: zero-initialize it, pass the same index with the same reasons
* for every add, and release it with erase_preason_index(). The index only
* points at list positions and checks the entry there on every hit, so an
* erase of the lists (a main reason, erase_job_preason()) or plain
* add_job_preason() calls in between leave it correct, never stale: an
* erased entry is simply added again. Expected O(1) per add.
*
* host == NULL && res == NULL sets the main reason exactly like
* add_job_preason() (erasing both lists) and returns its result.
*
* Returns the list's new entry count (> 0) when an entry was appended, 0
* when an identical entry is already present (or a main reason was set),
* -EINVAL for a NULL reasons or index, or -ENOMEM (nothing appended).
*/
typedef struct {
int nslots; /* power of two; 0 until the first add */
int nused;
uint32_t* hashes;
int* refs; /* 0 empty, else 1 + (list position << 1 | is_res) */
} xce_preason_index_t;
extern int add_job_preason_once(xce_job_preason_t*, xce_preason_index_t*, int,
char*, char*);
extern void erase_preason_index(xce_preason_index_t*);
/*
* Pending-reason equality: "are these two pending-reason records identical".
*