docs(specs): rename parjob-capacity-partitioning to parjob-placement, init pend-reason-push-dedup
parjob-placement now covers full LSF-style parallel placement: task range (-n min,max), capacity-driven partitioning, span[ptile=n], span[hosts=1], span[block=n] and a host-count limit. bug-1931 points to it for the full task-range behaviour. pend-reason-push-dedup: skip pushing a job's pend reason to the controller when it equals the last one the controller accepted. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
c7d82cac92
commit
75ca663726
5 changed files with 65 additions and 3 deletions
|
|
@ -1,6 +1,19 @@
|
|||
# Requirements Document
|
||||
|
||||
## Project Description (Input)
|
||||
**Scope (renamed 2026-10-09 from `parjob-capacity-partitioning`):** complete placement of parallel jobs, LSF-style. Covers:
|
||||
1. **Task range (`-n min,max`)** — place as many tasks as fit, up to `maxTasks`; if `minTasks` cannot be placed, pend after one search (feasibility is monotone in the task count for nested per-task requests). bug-1931 delivers a first, min-first + upward-search version on today's balanced plans; this spec owns the final behaviour.
|
||||
2. **Capacity-driven partitioning** — plan shapes from the hosts' free capacity (below).
|
||||
3. **`span[ptile=n]`** — exactly n tasks per host (the last host may get fewer, as in LSF).
|
||||
4. **`span[hosts=1]`** — all tasks on one host.
|
||||
5. **`span[block=n]`** — tasks per host a multiple of n.
|
||||
6. **Host-count limit (nhosts)** — bounds on how many hosts a job may use (`span[hosts=n]` style; exact syntax and min/max form to settle in requirements).
|
||||
7. **Cross-cutting** — parse `span[...]` into `xce_span_t` (today `section.peg:55` only captures it as raw `spanString`, and `xce_span_t` is an empty struct, `resreq.h:233`); validation and conflicting options (with each other and with `-n min,max`); decision-trace and pend-reason wording (e.g. "ptile=4 but no host has 4 free slots"); submission and display in the apiserver, `xcecli` and job detail.
|
||||
|
||||
Out of scope for now: `same[...]` and `affinity[...]` (also reserved, unused).
|
||||
|
||||
Each span option becomes a constraint on the capacity-driven heuristic (ptile fixes per-host counts, hosts=1 forces k=1, block rounds per-host counts down to multiples of n, nhosts bounds k), which is why they are designed together.
|
||||
|
||||
**Who has the problem:** users submitting parallel (multi-task, `maxTasks > 1`) jobs, and the operators who must explain why such jobs pend.
|
||||
|
||||
**Current situation:** `parjob_plan_iter_next()` (`src/scheduler/src/plugins/parjob_allocator.c`) generates exactly one plan per host count k = 1..maxTasks, always the balanced ceil/floor partition (e.g. N=5, k=2 → only `[3,2]`). Plan shape is a pure function of `maxTasks` and never looks at worker state (`scheduler-allocation` requirements.md, Req 4 AC 1 and "Data Structure Changes Summary", v1 balanced default). Unbalanced splits such as `[4,1]` or `[3,1,1]` are never generated, so a job pends even when the cluster's free capacity could hold it (e.g. two hosts with 4 and 1 free slots for a 5-task job). The per-position first-fit host claim in `sched_claim_bucket_()` (`alloc_runner.c`) is already complete for balanced plans (nested per-bucket requests, largest bucket first); the coverage gap is only the partition shape. Exhaustive partition enumeration is infeasible (p(100) ≈ 1.9×10⁸).
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
{
|
||||
"feature_name": "parjob-capacity-partitioning",
|
||||
"feature_name": "parjob-placement",
|
||||
"created_at": "2026-10-09T07:06:43Z",
|
||||
"updated_at": "2026-10-09T07:08:41Z",
|
||||
"updated_at": "2026-10-09T08:03:23Z",
|
||||
"language": "en",
|
||||
"phase": "initialized",
|
||||
"approvals": {
|
||||
25
.kiro/specs/pend-reason-push-dedup/requirements.md
Normal file
25
.kiro/specs/pend-reason-push-dedup/requirements.md
Normal file
|
|
@ -0,0 +1,25 @@
|
|||
# Requirements Document
|
||||
|
||||
## Project Description (Input)
|
||||
**Who has the problem:** operators of an XCE cluster, through the load on the controller and its HA standbys. Every pend-reason push becomes a durable, Raft-replicated log entry.
|
||||
|
||||
**Current situation:** the scheduler publishes every PEND decision in every scheduling session. `xce_sched_driver_on_decision()` (`src/scheduler/src/driver.c:914-919`) wraps the decision and calls `sched_publish_pend_reason()` (`scheduler.c:1913`) without comparing it with what was pushed before. On the controller each push goes through `set_preason()` (`src/controller/job.c`) and `persist_job_hook_fun`'s `HOOK_DETAIL_SETPREASON` arm, which appends a pend-reason WAL record (`src/include/wal_types.h` ~235-265, bug-1692) that is replicated to the standbys. A job that stays pending for hours with the same reason therefore costs one network push plus one durable, replicated WAL write per session, multiplied by every pending job.
|
||||
|
||||
**What should change:** when a job's pend reason in the current session is identical to the one last successfully pushed for it (full structural match, libxce `job_preason_eq()`, `preason.h`), the scheduler skips the push. Only real changes reach the controller.
|
||||
|
||||
Questions to settle in requirements:
|
||||
- **Baseline:** the last reason the controller *accepted* for the job, not merely the last one computed. A failed or retried push must not update it.
|
||||
- **Losing the baseline:** after a scheduler restart or failover, or a controller failover or resync, the baseline starts empty, so every pending job is pushed once.
|
||||
- **Controller-side clears:** `do_clr_preason()` (dispatch, requeue, other state changes) must invalidate the baseline so the next PEND is pushed even if it matches the old reason.
|
||||
- **Memory:** cost per pending job — keep a deep copy (exact compare) or a hash (cheaper, tiny collision risk).
|
||||
- **Where the baseline lives:** on the scheduler's job object, or in the driver.
|
||||
- **Simulator and replay:** the decision stream, the decision trace and `--sim-verify` must still see every PEND decision; only the publish to the controller is skipped.
|
||||
- **Observability:** count skipped pushes (log or stats).
|
||||
- **Defense in depth (optional):** a controller-side guard that skips the WAL append when the incoming reason equals the stored one.
|
||||
|
||||
Related: bug-1932 (session-level unsatisfiable-request cache, which skips repeated computation within one session), bug-1927 (pend reasons recorded once per job), spec `libxce-alloc-preason-equality` (`job_preason_eq`).
|
||||
|
||||
Status: initialized only; requirements to be discussed with the user before `/kiro-spec-requirements`.
|
||||
|
||||
## Requirements
|
||||
<!-- Will be generated in /kiro-spec-requirements phase -->
|
||||
24
.kiro/specs/pend-reason-push-dedup/spec.json
Normal file
24
.kiro/specs/pend-reason-push-dedup/spec.json
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
{
|
||||
"feature_name": "pend-reason-push-dedup",
|
||||
"created_at": "2026-10-09T08:19:27Z",
|
||||
"updated_at": "2026-10-09T08:19:27Z",
|
||||
"language": "en",
|
||||
"phase": "initialized",
|
||||
"approvals": {
|
||||
"requirements": {
|
||||
"generated": false,
|
||||
"approved": false
|
||||
},
|
||||
"design": {
|
||||
"generated": false,
|
||||
"approved": false
|
||||
},
|
||||
"tasks": {
|
||||
"generated": false,
|
||||
"approved": false
|
||||
}
|
||||
},
|
||||
"ready_for_implementation": false
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -64,4 +64,4 @@ TODO: required for fixed/partial
|
|||
|
||||
## References
|
||||
- Related: bug-1770 (wired real minTasks/maxTasks into the primary resreq; the allocator never consumed minTasks), bug-1932 (same trace; unsatisfiable cache not used in sim)
|
||||
- Docs: .kiro/specs/scheduler-allocation/requirements.md (Req 4 AC 1), .kiro/specs/resreq-maxtasks-wiring/, .kiro/specs/parjob-capacity-partitioning/
|
||||
- Docs: .kiro/specs/scheduler-allocation/requirements.md (Req 4 AC 1), .kiro/specs/resreq-maxtasks-wiring/, .kiro/specs/parjob-placement/ (owns the full task-range behaviour together with capacity-driven partitioning and span options; this bug fix covers only the min-first + upward search on today's balanced plans)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue