lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

refactor(scheduler): defer job-array expansion off the vtask thread pool

Browse source 
sched_on_job_array() expanded a compact array-job push (up to
hundreds of elements: a per-element dedup lookup plus a full deep
clone) inline, synchronously, on whatever vtaskmgr thread handled the
incoming request -- the same shared pool the scheduler's own
controller connection traffic depends on, violating ha-async-io.md's
"bounded handler work only" contract for that pool (a project-wide
design invariant). A prior finding already showed a burst of
6x50-element array pushes (300 job events landing within ~2ms of each
other) causing the scheduler to lose its controller connection within
245ms and enter a reconnect/failover loop -- directly consistent with
this mechanism, and the same reconnect storm whose resync races
motivated the companion JOBS_PUSH fix in this branch.

The request handler now does O(1) work only: validate the descriptor
and enqueue it unexpanded (XCE_SCHED_OP_UPDATE_JOB_ARRAY, mirroring
the existing XCE_SCHED_OP_ADMIN_RECORD deferred-dispatch pattern).
The actual per-element expansion and dedup check moved to
translate_one_pending_update(), which runs on the round-processing
thread (main.c's xce_sched_driver_run_normal_mode_round(), before
xce_core_begin_session() each round) -- a thread with no connection-
liveness responsibility. As a side effect this also closes the
dedup check's own pre-existing cross-thread TOCTOU race (previously
documented as bug-1714 follow-up "Bug B"): only one thread ever runs
it now.

test_job_array_expand.c and test_job_array_wire_e2e.c, which asserted
the old inline-expansion behavior, are updated to assert the new
deferred-enqueue behavior instead; expansion, dedup, and full
drain-push-release ownership coverage moved to new tests in
test_driver_translate.c.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RLSNJhNoULvVcUhXEFafQC
This commit is contained in:
Rongsong Shen 2026-09-15 09:01:47 +00:00
parent 6c201bb3e8
commit d5ec1eb3ae
No known key found for this signature in database
GPG key ID: 5BCDAAE0123E9FE2
10 changed files with 838 additions and 517 deletions
Download patch file Download diff file Expand all files Collapse all files

15
.wolf/anatomy.md
View file

@ -1,7 +1,7 @@
# anatomy.md
> Auto-maintained by OpenWolf. Last scanned: 2026-09-15T03:56:28.973Z
> Files: 567 tracked | Anatomy hits: 0 | Misses: 0
> Auto-maintained by OpenWolf. Last scanned: 2026-09-15T08:47:36.542Z
> Files: 572 tracked | Anatomy hits: 0 | Misses: 0
## ./
@ -906,12 +906,14 @@
## src/scheduler/include/
- `scheduler.h` — if !defined(_SCHEDULER_H_) (~4716 tok)
- `scheduler.h` — if !defined(_SCHEDULER_H_) (~5360 tok)
- `sim.h` — if !defined(_SIM_H_) (~4213 tok)
## src/scheduler/src/
- `core_events.c` — include <errno.h> (~16287 tok)
- `core_events.c` — include <errno.h> (~16984 tok)
- `driver.c` — include <errno.h> (~11968 tok)
- `handler.c` — include <errno.h> (~5162 tok)
- `plugin.c` — include <errno.h> (~4554 tok)
- `repl_break.c` — include <stdio.h> (~9921 tok)
- `repl_info.c` — include <stdio.h> (~16226 tok)
@ -927,7 +929,10 @@
## src/scheduler/testsuite/
- `test_core_events.c` — Declares and (~18252 tok)
- `test_core_events.c` — Declares and (~19786 tok)
- `test_driver_translate.c` — Declares field (~11757 tok)
- `test_job_array_expand.c` (~3393 tok)
- `test_job_array_wire_e2e.c` — Declares vtask_entry (~4026 tok)
- `test_parjob_combinatorial.c` — Declares of (~12038 tok)
- `test_repl_breakpoints_inspection.c` — Declares and (~23533 tok)
- `test_repl_help.c` — Declares error (~6047 tok)