jobsub -a parses its value strictly (digits only, leading zeros ok):
1 submits an ordinary job; 2..32767 allocate with idgen_array, whose
result is checked; missing, non-digit and 0 values are refused with the
1-32767 range and values above 32767 with the limit, submitting nothing.
The success line and $jobid name the array identity; the random
sequence seed is bounded so the first array cannot overflow; help text
updated. Fixes bug-1896 for the harness.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The worker sets XCE_JOBINDEX to the element number for jobs carrying the
array marking (a one-element array's element gets 1) and to 0 otherwise.
A submission-supplied XCE_JOBINDEX is not applied and a WARN names the
job and the ignored variable; XCE_JOBID handling is unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The scheduler expands an array push only when the first ID carries the
array marking and the size fits in its block (size <= 32768 - idx, no
addition); otherwise it logs an ERROR naming the array identity and
expands nothing. Pre-upgrade arrays starting at index 0 still expand.
The INFO log names the array identity (array_jobid=...). This also
closes a crash: a malformed push with a huge size overflowed calloc.
Expansion tests move to tagged block|1 first IDs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
handle_jobsubmit now registers the submission first and checks the job
table second; a hit is answered -EEXIST and released through the atomic
reply-and-release step. For array submissions, array_identity_exists()
does one read-locked scan of the 32768 tagged element keys of the
array's block (element 0 included for pre-upgrade arrays), covering
both the element range and the array identity; unmarked submissions
keep jobid_range_exists. An existing X|3 now blocks a new X|5.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
In-flight registry entries carry is_array and array_id; two array
entries with the same array identity overlap. inflight_reply_and_remove()
sends the reply and unlinks the entry under the registry lock (xce_reply
is non-blocking) and is used on every post-registration path, inside the
async one-shot reply guard; jobsubmit_async_ctx_release no longer touches
the registry, so a retry registered after the reply cannot be deleted by
the original's release. Test-only seams under XCE_TESTING make the
atomicity observable deterministically.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
handle_jobsubmit classifies (first_jobid, array_size) before the field
checks, the before-event hook and any registry access: a tagged array
starting at index 0 is answered with the new E_XCE_ARRAY_START0 and an
ERROR log asking for the client to be upgraded (7.1, wins per 7.9);
size 0, tags 2/3, untagged multi-element submissions and arrays whose
last element exceeds 32767 (checked without addition) get -EINVAL.
Nothing is stored for a rejected submission. This also closes a crash:
a size-0 submission used to reach payload building and overflow calloc.
Existing array tests move to tagged block|1 first IDs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
idgen_array() allocates the next whole 32768 block and returns its
first element (array tag, index 1), refusing n outside 1..32767 and a
block past the 20-bit sequence (-EOVERFLOW, context unchanged). The
reset-on-new-second step is shared with idgen() via idgen_tick_locked();
idgen() itself is unchanged until task 9.1.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
parse_jobid() keeps its signature but is now bounded and NULL-safe: a
bare identity returns the exact ID, [k] the element ID, and a range, any
rejected selection (5.7, 5.8, 5.14, 5.15) or a NULL argument returns
-EINVAL with *jobid = 0. Fixes the strlen-before-NULL-check crash and
the atoi overflow; regex parsing is gone.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
parse_jobsel() parses c-a-ts-seq with an optional [k] or [a-b] under a
fixed grammar and check order (syntax, field widths, array-identity
alignment, element bound, reversed range), saturating digits so an
oversized number cannot wrap; plus is_array_tag(), jobsel_element_id(),
jobsel_strerror() and JOBID_FIELD_MAX_* constants. The selection-cases
conformance table runs in the libxce suite.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Spec for numbering job-array elements from 1 (bug-1896): approved
requirements, a design reviewed over 20 rounds until clean (log in
research.md), and a 30-task plan. The design review found two live
apiserver defects, logged here: bracketed job_id/job_name filters are
never evaluated (bug-1898) and ControlJobByID re-dispatches earlier
selections (bug-1899).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Feature validation GO (ASAN scheduler suite 96/96, smoke replay clean on
the allocation path). Roadmap entry marked implemented with follow-ups.
bug-1897 (found by the validation smoke run, not caused by this feature):
xce_scheduler --sim segfaults in limit_job_election on a job whose
snapshot resreq failed to parse (pre-bug-1888-fix replays).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Clean ASAN scheduler suite 96/96 with no expectation changes in existing
tests beyond wiring; scheduler invariant audit of the allocation path
clean (driver state-equivalence, core I/O isolation, picker/allocator
rendezvous, snapshot parity). issues/new_requirements.md item 4 marked
resolved.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Comments in scheduler sources and tests that presented the deleted
my_alloc()/parjob claim helpers as current code now name
sched_allocator_run() and its runner helpers; test helpers and tests
named after my_alloc are renamed. No logic or expectation changes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
seqjob registers sched_allocator_run with seqjob_alloc_iface (no accepts
guard, plus its one-bucket plan iterator); its private my_alloc() is
gone. New fixtures pin the intended changes: a reason for every short
host, the bug-1893 shared reason, the host-stage OOM debit release, and
-EINVAL for missing inputs; seqjob's plan is asserted to be one bucket of
one slot. No other test expectation changed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
parjob registers sched_allocator_run with parjob_alloc_iface (accepts
maxTasks > 1, bug-1767, plus its plan iterator); its private my_alloc()
and claim helpers are gone. Characterization now pins the intended
changes: a reason for every short host (2/6/5) and the bug-1893 shared
reason. No other test expectation changed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
sched_finalize_job_alloc_() turns a satisfied plan's claims into the job
allocation exactly as parjob's commit tail does (host records from the
claim-time snapshots in claim order, shared records, jobid 0, no
duration) and publishes ctx->alloc only on success. Every out-of-memory
step frees what it built and releases the debits of claims not yet
turned into host allocations; each claim snapshot is freed exactly once.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The runner tries the plugin's plans in order and claims one distinct host
per bucket, with parjob's rollback and claim snapshots, nslots/nhosts per
candidate, a growable claims buffer and malformed-plan skipping. It
clears alloc_req.host before every candidate so each short host gets its
pend reasons (Req 3.6/3.7). Result assembly follows in task 3.3; until
then a satisfied plan's claims are released and the attempt pends.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
New alloc_runner.c: sched_allocator_run() validates its inputs (-EINVAL
without touching the pend reason), applies the plugin's optional accepts
guard, creates the pend-reason record, then checks shared resources: one
host-less NO_RESOURCE reason per unmet entry (skip only on vget < 0,
fixing bug-1893) and the met amounts counted once as uncommitted. Plan
search follows in task 3.2. Runner tests use a fake allocator contract;
the pend-reason factory audit now scans the runner.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
15 tests drive the real parjob plugin through job_alloc_fn: success,
short shared, pending on hosts, mid-plan rollback, task counts 0/1,
missing inputs, six OOM steps with their debits, COMPOUND. Values the
runner changes on purpose sit in *_today tests.
Spec: the task showed that parjob records host reasons only for a bucket
search's first rejected candidate (the rejected host's request stays in
alloc_req.host and the host iterator pre-filters later hosts against
it). Decision: the runner clears alloc_req.host before every candidate,
for both allocators, so every short host gets its reasons (Req 3.6/3.7);
publishing reasons only when they change is a follow-up.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
12 tests drive the real seqjob plugin through job_alloc_fn and record
today's results, reasons and debits, with switch-stable OOM injection
(sched_make_allocs stub, --wrap of make_host_alloc/make_job_alloc).
Assertions the runner changes on purpose sit in *_today tests.
The characterization showed bug-1893's real effect: vget() returns 0 on
success, so the shared-reason loop (rc <= 0 skip) never records a shared
NO_RESOURCE reason. Bug record and spec wording corrected; the runner
skips only on rc < 0.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
xce_sched_allocator_iface_t (optional accepts guard + plan_init/next/fini)
and the sched_allocator_run() entry point in sched/alloc_iter.h; the
plugin's const allocator_iface reference, forward-declared in plugin.h
with the picker_iface ownership rule. No plugin uses them yet.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
xce_alloc_plan_iter_t.init takes xce_sched_core_t* instead of
xce_scheduler_t*; both built-in generators, both allocators' call sites
and the contract-test stub follow. The parjob purity test now shows an
identical plan sequence for a NULL and a poisoned core.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Requirements reviewed and revised (seqjob per-bucket reset, nslots/nhosts,
bug-1893 fix in the runner), design and 11-task plan generated and reviewed.
bug-1892: unsatisfy_() reads index-keyed uncommitted amounts by position.
bug-1893: allocators' shared pend-reason loop reads past the unmet list.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
New specs from /kiro-discovery (2026-09-29), based on the LSF 10.1 Admin
Guide section "Job dependency and job priority". Dependency only; job priority
is deferred to a later discovery. Both are added to roadmap.md.
- job-dependency: -w expressions by job ID (done, ended, exit(id[,op code]),
started, post_done, post_err, bare id; && || ! and parentheses). One libxce
grammar, used from Go through cgo. The apiserver validates early (syntax,
parent exists, job:view). The controller re-validates, stores the
expression, and keeps an in-memory reverse index rebuilt on replay. The
scheduler evaluates in sched_job_is_ready(); unsatisfiable dependencies stay
PEND. Also closes the scheduler exit-code field-completeness gap. Adds
xcecli -w / modify -w -wn and harness jobsub -w / jobmod.
- job-dependency-insight: bjdepinfo-style query, dependency info in job
details, and optional orphan termination (-ti or a cluster grace period).
- controller-job-cleanup brief: job-dependency will replace the
job_has_dependents() stub.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
All required tasks are done and feature-level GO is recorded (d19e9f27);
only optional task 8.5 (observing transient catching-up live) remains.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Its false-ERROR-on-reconnect symptom was already fixed by bug-1854 and
bug-1886; today's HA soak shows zero scheduler ERRORs on reconnect.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
scheduler-sim-diagnostics, scheduler-sim-live-allocation, xce-test-cli-ux
and eliminate-frontend-makefile had every sub-task checked but their
parent boxes and spec.json phase still read tasks-generated.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
New spec from /kiro-discovery (2026-09-29), added to roadmap.md:
- Begin time: the scheduler won't start a job before it (gate in
sched_job_is_ready() using the core clock). New pend reason
BEGIN_TIME_NOT_REACHED.
- End time: the worker enforces it for running jobs (SIGTERM, grace, SIGKILL)
and records a persisted END_TIME_REACHED exit reason. The primary is not
involved. A pending job whose end time can't be met stays PEND with
END_TIME_UNMEETABLE until the user kills or modifies it.
- Runtime estimate (-We): a scheduling hint only. It sets the rule
now + (estimate, else run limit, else 0) > end; an estimate larger than the
run limit is rejected.
- Times and estimate can be changed only while the job is pending, through a
new `xce jobs modify` in xcecli and a new `jobmod` in the harness. Harness
jobsub gains all four options.
- Fixes the silent drop of xcecli -b/-t/-W and REST run_time_estimation,
which the apiserver's modelJobToSubmit never passes to the controller.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
New specs from /kiro-discovery (2026-09-28/29), added to roadmap.md:
- job-status-detail-display: show pending reasons and reserved-vs-used
resources in REST and xcecli. Requirements generated (7 requirements).
- scheduler-alter-pend-reasons: keep a pending reason for every alternative
resreq (per-alternative JobPendReason) and fix the obj->reason leak.
- controller-job-cleanup: primary-only sweep that removes finished jobs past
a configurable retention, via a replicated WAL removal fact.
- application-profile: admin-defined application profiles (execution mode,
launch type, env, resource defaults) chosen per job, with app:use/app:view
RBAC. Queues and profiles stay orthogonal.
- job-sandbox-execution / job-sandbox-multinode: run jobs in a systemd
DynamicUser sandbox behind a pluggable execution backend.
- rbac-persistence: make RBAC changes made through the API survive an
apiserver restart (closes the security.md gap).
roadmap.md also records the scheduler-sim-diagnostics info job update and a
security item: remote-task units appear to run as root (found by code
reading).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
scheduler-allocator-runner: brief and requirements for moving the duplicated
seqjob/parjob my_alloc() body into one framework-owned allocator runner.
scheduler-sim-decision-trace: brief and requirements for a per-job decision
trace in the simulator REPL (converged after 9 review rounds). The
decision-reasoning seed capability in scheduler-sim-diagnostics now points to
this spec.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Add test_sim_replay_reproduces_recorded_alloc_from: a recorded DISPATCH at
an epoch-scale beginRound with a sub-second part replays at the default
simulation settings with no --sim-verify mismatch, and the live allocation's
from equals beginRound in whole seconds. Share the dispatch fixture setup
of the existing tests through two helpers.
Live: 2-round HA soak recordings (11 DISPATCH decisions) replayed with
--sim-verify and no --sim-speed report 0 mismatches.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
