Commit graph

1395 commits

Author SHA1 Message Date
bcd357c531
docs(job-array-one-based-index): GenerateJobID allocates ordinary IDs only
The comment still described arraySize > 1 as the array path; since task
9.1 idgen() rejects any size other than 1 and arrays use
GenerateArrayJobID.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 13:18:15 +00:00
63eaea215a
feat(job-array-one-based-index): idgen allocates ordinary IDs only (task 9.1)
idgen() now accepts only a size of 1 (anything else: -EINVAL, context
unchanged) and refuses an exhausted 20-bit sequence with -EOVERFLOW
instead of spilling into the timestamp (also after idgen_array has
taken the last block). Arrays come only from idgen_array; every caller
of idgen/GenerateJobID passes 1. The old multi-element test loop moves
to idgen_array with 1-based assertions.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 13:17:59 +00:00
d3b12159a4
feat(job-array-one-based-index): xcecli submit shows the array (task 8.3)
Submitting an array prints the server's message with the array ID in
written form (no [k]) and the element count, e.g.
'Job <0-0-...> is submitted to queue <normal> (3 elements).'; ordinary
submits print exactly as before. The client model mirrors array_size.
Submit help documents name[1-N] and name[1], numbering from 1 and the
API-server limit (default 1000, at most 32767).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 12:58:36 +00:00
58ca7abb36
feat(job-array-one-based-index): xcecli list and control use the selection parser (task 8.2)
ID lists split only at commas outside brackets and every selection is
validated before any request (control allows identity ranges, listing
refuses them). A raw -J value with brackets is checked before the
wildcard mapping, and the search rule and * stripping apply to the
name part only (-J 'foo*[1]' lists element 1 of arrays whose name
starts with foo); values without brackets map exactly as before.
Control results print each item's own job ID when the server sends it,
so a pre-upgrade element 0 shows as <array-id>[0]. Control and list
help show <array-id>[k] and <array-id>[a-b]; list help now says -J.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 12:53:50 +00:00
bc616e8c4a
feat(job-array-one-based-index): client identity and selection parser (task 8.1)
xcecli's format package gains ParseSelection (written or decimal key
with optional [k]/[a-b], the server's grammar and check order; 5.8 and
5.7 also on decimal keys with [...], so an upgraded client protects
itself against an older server; query ranges refused after the other
checks), the bracket-aware SplitSelections, the raw -J
ParseNameSelection and Selection.APIKey. ParseJobID is bounded;
ToAPIJobID(s, allowRange) no longer passes unrecognised text through;
peek keeps today's behaviour via ToAPIJobIDLegacy.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 12:47:39 +00:00
40917a01f0
feat(job-array-one-based-index): control by filter with element selections (task 7.5)
Filter-based ControlJob validates a bracketed -J value before fetching
(400 with the parser's message; unbracketed names, including commands
with spaces, unchanged). FetchJobList evaluates bracketed job_name
entries through the selection predicates instead of unevaluated
compound conditions, so -J 'foo[1]' acts on element 1 of every array
named foo and never on ordinary jobs of that name. The most-recent-job
rule applies only without an element number and without all-jobs, and
treats the element of a one-element array like an ordinary job
(isSingleElementArray, a documented approximation). The old
index-section parser is deleted. Completes bug-1898 on the owned paths.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 12:40:09 +00:00
4020740208
feat(job-array-one-based-index): control by identity validates first and dispatches once (task 7.4)
ControlJobByID parses every selection with ParseIdentitySelection before
any store access (first rejection: 400, nothing dispatched), collects
the jobs to act on de-duplicated by unique_job_id plus one not-found
result per missing number per selection (carrying the would-be element
ID), validates each job once after the loop and dispatches each
accepted job once. foundJob gains an explicit-index flag so
<array-id>[0] matches only element 0 and never falls back to the whole
array or an ordinary job. Fixes bug-1899 (earlier selections were
re-dispatched for every later one).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 12:29:19 +00:00
3457d892b6
feat(job-array-one-based-index): query handlers evaluate element selections (task 7.3)
GetJobsInfo validates every filter entry in request order before any
store access (including the exec_hosts lookup) and GetJobByIDs every key
before querying; the first rejection returns 400 with the parser's
message. Bracketed job_id/job_name entries are evaluated as row
predicates before sorting and pagination, so job_id:eq:X[2] lists only
element 2, GET /jobs/X[1-3] lists elements 1-3 and <ordinary-id>[0]
lists nothing. The unevaluated compound conditions and the old index
parsing are gone from both handlers. Fixes bug-1898 on the query paths.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 12:21:11 +00:00
675d7d7f6c
feat(job-array-one-based-index): selection predicate builder (task 7.2)
BuildSelectionPredicates converts each job_id/job_name filter entry
containing a bracketed value into one row predicate (values OR'ed,
entries AND'ed with each other and the plain conditions), removing only
that entry. Operators are honoured instead of dropped (bug-1898):
eq/in/between select, notin excludes, name pattern operators reuse
today's condition semantics on the name part, and ne / comparison /
null operators keep today's rejection. Element selections match only
array elements in range; bare identities match the whole array and any
ordinary job; an empty name under exact operators matches nothing.
Bare written keys are normalised to decimal on every path. Not yet
wired into the handlers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 12:09:47 +00:00
2a8490bc92
feat(job-array-one-based-index): REST identity and name selection parsers (task 7.1)
ParseIdentitySelection (strict written form or base-10 decimal key,
optional [k] or [a-b]) and ParseNameSelection (name part may be empty,
no whitespace or brackets) apply the shared selection grammar and check
order (5.16 syntax, 5.14 width, 5.17 marked decimal without [k], 5.8
alignment only with [...], 5.7 element bound, 5.5 reversed), reporting
only the first failure with a rule tag and a message quoting the
selection. Runs the shared selection-cases table plus REST-only rows.
Not yet wired into the handlers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 12:01:22 +00:00
c30e5f1d82
feat(job-array-one-based-index): REST submit creates arrays numbered from 1 (task 6.2)
preCheckJobRequest returns (n, isArray, err): a bracketed job name is
parsed by ParseSubmitIndexExpr first, so name[1-N] and name[1] create
arrays (one-element arrays included) allocated with
generateArrayJobID, stored under the bare name; expression errors are
answered 400 with their specific message and create nothing; names
without brackets take the unchanged createArrayName path. The response
names the array (jobid = array ID, new array_size, "(n elements)"),
submit-time %I/%J use the first element, and the submit and dispatch
logs identify arrays by array ID. Fixes bug-1896 for REST and xcecli.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 11:55:39 +00:00
0318ac5f44
feat(job-array-one-based-index): parse submit index expressions (task 6.1)
ParseSubmitIndexExpr accepts only name[1-N] and name[1] (leading zeros
allowed) and otherwise reports the lowest-numbered broken rule with its
specific message: interactive arrays (2.1), invalid name or malformed
expression (2.2), lists/steps (2.3), %N (2.4), start 0 (2.5), reversed
(2.6), start above 1 (2.7), above the limit (2.8). Oversized numbers
saturate for the bound checks while reversal compares the digit
strings, so the rule order holds for any input. Not yet wired into
CreateJob (task 6.2).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 11:47:23 +00:00
3a7e3890e2
test(job-array-one-based-index): apiserver handler test seams and fixture (task 5.4)
CreateJob's ID generation and submission and both control dispatch
sites now go through package variables (generateJobID,
generateArrayJobID, submitJob, controlJob) that default to the real
backend/services functions, so production behaviour is unchanged. A new
non-cgo handler test file provides a reusable fixture: it saves and
restores every global it touches, sets up auth/RBAC, seeds jobs and
their queues, and records each dispatched job.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 11:41:45 +00:00
46f57095c1
feat(job-array-one-based-index): row predicates in the apiserver query pipeline (task 5.3)
QueryOptions gains Predicates; store.ExecuteQuery keeps a row only if
every predicate returns true, after the plain conditions and before
sort, pagination and field selection, so page boundaries and tokens
count only matching rows. Nil/empty predicates leave every existing
query unchanged. Foundation for evaluating element selections
(bug-1898).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 11:35:05 +00:00
7f417262c5
feat(job-array-one-based-index): clamp the apiserver array size limit (task 5.2)
job.max_array_size is resolved by a pure resolveMaxArraySize: unset or 0
gives 1000, 1..32767 is used as is, values above 32767 are clamped to
32767 with a startup warning naming the configured and effective value,
and a negative value refuses startup. The old >2147483646 rejection is
gone; the shipped config documents the bounds.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 11:32:25 +00:00
b88e352616
feat(job-array-one-based-index): apiserver backend ID helpers and array ID wrapper (task 5.1)
GenerateArrayJobID wraps libxce idgen_array (first element, index 1;
one-element arrays included) with a non-cgo stub; pure helpers
ParseHumanJobIDStrict (written identity with exact field-width checks,
uint64 overflow reported as a width error), IsArrayCapableKey and
MakeElementID (bit-identical to libxce jobsel_element_id).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 11:29:11 +00:00
0e7cfdcdf3
feat(job-array-one-based-index): strict single-job harness arguments (task 4.3)
Every harness command that acts on one job (jobinfo per -j, jobrun,
jobkill, jobckpt, taskrun, setpend, jstart, jfinish, jfail) and
expect-job parse their job argument with cmdutil_job_arg on top of the
libxce selection parser: element ranges (reversed too) are refused with
the single-element message, invalid identities are rejected with the
parser's reason, and no command sends job 0 for a rejected argument.
Not-found results (controller -ENOENT, or jobinfo's empty reply) name
the job and, for bare identities, hint at <array-id>[k]. expect-job
matches job numbers instead of formatted text.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 11:22:39 +00:00
1812a80d19
feat(job-array-one-based-index): harness $jobid[...] substitution (task 4.2)
Auto-variable substitution moves out of test.c into its own unit-tested
module (autovars.c) with an explicit context for $jobid, $? and
$jobcount, keeping today's token rule, messages and buffer handling.
New: a token $jobid[...] expands $jobid and keeps the rest of the token,
so $jobid[3] names element 3 of the array just submitted; it fails like
an unset $jobid when none is set.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 10:56:15 +00:00
d2c87f185a
feat(job-array-one-based-index): harness jobsub -a numbers arrays from 1 (task 4.1)
jobsub -a parses its value strictly (digits only, leading zeros ok):
1 submits an ordinary job; 2..32767 allocate with idgen_array, whose
result is checked; missing, non-digit and 0 values are refused with the
1-32767 range and values above 32767 with the limit, submitting nothing.
The success line and $jobid name the array identity; the random
sequence seed is bounded so the first array cannot overflow; help text
updated. Fixes bug-1896 for the harness.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 10:47:22 +00:00
7aa00f0d9d
feat(job-array-one-based-index): XCE_JOBINDEX is the element number (task 3.2)
The worker sets XCE_JOBINDEX to the element number for jobs carrying the
array marking (a one-element array's element gets 1) and to 0 otherwise.
A submission-supplied XCE_JOBINDEX is not applied and a WARN names the
job and the ignored variable; XCE_JOBID handling is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 10:34:59 +00:00
4f44a68973
feat(job-array-one-based-index): guard scheduler array expansion (task 3.1)
The scheduler expands an array push only when the first ID carries the
array marking and the size fits in its block (size <= 32768 - idx, no
addition); otherwise it logs an ERROR naming the array identity and
expands nothing. Pre-upgrade arrays starting at index 0 still expand.
The INFO log names the array identity (array_jobid=...). This also
closes a crash: a malformed push with a huge size overflowed calloc.
Expansion tests move to tagged block|1 first IDs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 10:24:42 +00:00
fe5ddc9854
feat(job-array-one-based-index): collision checks after registration (task 2.3)
handle_jobsubmit now registers the submission first and checks the job
table second; a hit is answered -EEXIST and released through the atomic
reply-and-release step. For array submissions, array_identity_exists()
does one read-locked scan of the 32768 tagged element keys of the
array's block (element 0 included for pre-upgrade arrays), covering
both the element range and the array identity; unmarked submissions
keep jobid_range_exists. An existing X|3 now blocks a new X|5.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 10:14:47 +00:00
81bfaf1f50
feat(job-array-one-based-index): atomic reply-and-release of in-flight submissions (task 2.2)
In-flight registry entries carry is_array and array_id; two array
entries with the same array identity overlap. inflight_reply_and_remove()
sends the reply and unlinks the entry under the registry lock (xce_reply
is non-blocking) and is used on every post-registration path, inside the
async one-shot reply guard; jobsubmit_async_ctx_release no longer touches
the registry, so a retry registered after the reply cannot be deleted by
the original's release. Test-only seams under XCE_TESTING make the
atomicity observable deterministically.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 09:59:45 +00:00
e6c484a0e7
feat(job-array-one-based-index): classify every job submission first (task 2.1)
handle_jobsubmit classifies (first_jobid, array_size) before the field
checks, the before-event hook and any registry access: a tagged array
starting at index 0 is answered with the new E_XCE_ARRAY_START0 and an
ERROR log asking for the client to be upgraded (7.1, wins per 7.9);
size 0, tags 2/3, untagged multi-element submissions and arrays whose
last element exceeds 32767 (checked without addition) get -EINVAL.
Nothing is stored for a rejected submission. This also closes a crash:
a size-0 submission used to reach payload building and overflow calloc.
Existing array tests move to tagged block|1 first IDs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 09:44:12 +00:00
ef728435d4
feat(job-array-one-based-index): array ID allocation numbered from 1 (task 1.3)
idgen_array() allocates the next whole 32768 block and returns its
first element (array tag, index 1), refusing n outside 1..32767 and a
block past the 20-bit sequence (-EOVERFLOW, context unchanged). The
reset-on-new-second step is shared with idgen() via idgen_tick_locked();
idgen() itself is unchanged until task 9.1.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 09:24:00 +00:00
d027410304
feat(job-array-one-based-index): parse_jobid on top of parse_jobsel (task 1.2)
parse_jobid() keeps its signature but is now bounded and NULL-safe: a
bare identity returns the exact ID, [k] the element ID, and a range, any
rejected selection (5.7, 5.8, 5.14, 5.15) or a NULL argument returns
-EINVAL with *jobid = 0. Fixes the strlen-before-NULL-check crash and
the atoi overflow; regex parsing is gone.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 09:15:48 +00:00
6e6c599d27
feat(job-array-one-based-index): bounded job-identity and element-selection parser in libxce (task 1.1)
parse_jobsel() parses c-a-ts-seq with an optional [k] or [a-b] under a
fixed grammar and check order (syntax, field widths, array-identity
alignment, element bound, reversed range), saturating digits so an
oversized number cannot wrap; plus is_array_tag(), jobsel_element_id(),
jobsel_strerror() and JOBID_FIELD_MAX_* constants. The selection-cases
conformance table runs in the libxce suite.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 09:01:57 +00:00
e822b88886
fix(scheduler): admission check reads uncommitted amounts by resource index (bug-1892)
unsatisfy_() compared positional total/inuse/wanted arrays (entry i is
resource wanted->idxs[i]) with uncomitted[i], but the allocation
context's uncommitted vectors are dense and keyed by resource index.
From the second primary of a COMPOUND/ALTER job on, the check ignored
earlier primaries' claim on the same resource (over-admit) or charged
another resource's (wrong pend), with the matching wrong shortfall in
the pend reason.

unsatisfy_() now takes the vector and reads entry wanted->idxs[i]; an
index past its end (sim_loop.c's short all-zero vectors) or a NULL
vector counts as nothing uncommitted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 04:00:42 +00:00
0fe27d4be3
fix(scheduler): never elect a job without a schedobj (bug-1897)
A job whose snapshot resreq fails to parse (any replay captured before
the bug-1888 fix) is kept by xce_sched_core_load_snapshot() with no
schedobj, but was still made a picker candidate -- by the loader and, in
simulation mode, by sim_bootstrap_picker()'s picker rebuild. The
allocator thread then dereferenced the NULL schedobj in
limit_job_election() and xce_scheduler --sim crashed.

Both push sites now skip such a job (it stays in the core so a restored
allocation is still charged), and sched_allocator() skips any picked job
without a schedobj as a backstop.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 03:45:51 +00:00
d006de3853
refactor(scheduler-allocator-runner): describe the framework runner, not the removed my_alloc() (task 5.1)
Comments in scheduler sources and tests that presented the deleted
my_alloc()/parjob claim helpers as current code now name
sched_allocator_run() and its runner helpers; test helpers and tests
named after my_alloc are renamed. No logic or expectation changes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 10:02:46 +00:00
65679285ba
feat(scheduler-allocator-runner): seqjob allocates through the framework runner (task 4.2)
seqjob registers sched_allocator_run with seqjob_alloc_iface (no accepts
guard, plus its one-bucket plan iterator); its private my_alloc() is
gone. New fixtures pin the intended changes: a reason for every short
host, the bug-1893 shared reason, the host-stage OOM debit release, and
-EINVAL for missing inputs; seqjob's plan is asserted to be one bucket of
one slot. No other test expectation changed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 09:53:14 +00:00
239bcf95cd
feat(scheduler-allocator-runner): parjob allocates through the framework runner (task 4.1)
parjob registers sched_allocator_run with parjob_alloc_iface (accepts
maxTasks > 1, bug-1767, plus its plan iterator); its private my_alloc()
and claim helpers are gone. Characterization now pins the intended
changes: a reason for every short host (2/6/5) and the bug-1893 shared
reason. No other test expectation changed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 09:40:47 +00:00
0e68ed991a
feat(scheduler-allocator-runner): runner result assembly (task 3.3)
sched_finalize_job_alloc_() turns a satisfied plan's claims into the job
allocation exactly as parjob's commit tail does (host records from the
claim-time snapshots in claim order, shared records, jobid 0, no
duration) and publishes ctx->alloc only on success. Every out-of-memory
step frees what it built and releases the debits of claims not yet
turned into host allocations; each claim snapshot is freed exactly once.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 09:27:32 +00:00
0c011f784e
feat(scheduler-allocator-runner): runner plan and claim search (task 3.2)
The runner tries the plugin's plans in order and claims one distinct host
per bucket, with parjob's rollback and claim snapshots, nslots/nhosts per
candidate, a growable claims buffer and malformed-plan skipping. It
clears alloc_req.host before every candidate so each short host gets its
pend reasons (Req 3.6/3.7). Result assembly follows in task 3.3; until
then a satisfied plan's claims are released and the attempt pends.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 09:12:01 +00:00
d9ef983014
feat(scheduler-allocator-runner): runner guards and shared-resource precheck (task 3.1)
New alloc_runner.c: sched_allocator_run() validates its inputs (-EINVAL
without touching the pend reason), applies the plugin's optional accepts
guard, creates the pend-reason record, then checks shared resources: one
host-less NO_RESOURCE reason per unmet entry (skip only on vget < 0,
fixing bug-1893) and the met amounts counted once as uncommitted. Plan
search follows in task 3.2. Runner tests use a fake allocator contract;
the pend-reason factory audit now scans the runner.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 08:58:07 +00:00
b4470b2b94
test(scheduler-allocator-runner): characterize the parjob allocator (task 2.2)
15 tests drive the real parjob plugin through job_alloc_fn: success,
short shared, pending on hosts, mid-plan rollback, task counts 0/1,
missing inputs, six OOM steps with their debits, COMPOUND. Values the
runner changes on purpose sit in *_today tests.

Spec: the task showed that parjob records host reasons only for a bucket
search's first rejected candidate (the rejected host's request stays in
alloc_req.host and the host iterator pre-filters later hosts against
it). Decision: the runner clears alloc_req.host before every candidate,
for both allocators, so every short host gets its reasons (Req 3.6/3.7);
publishing reasons only when they change is a follow-up.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 08:40:35 +00:00
91a8580948
test(scheduler-allocator-runner): characterize the seqjob allocator (task 2.1)
12 tests drive the real seqjob plugin through job_alloc_fn and record
today's results, reasons and debits, with switch-stable OOM injection
(sched_make_allocs stub, --wrap of make_host_alloc/make_job_alloc).
Assertions the runner changes on purpose sit in *_today tests.

The characterization showed bug-1893's real effect: vget() returns 0 on
success, so the shared-reason loop (rc <= 0 skip) never records a shared
NO_RESOURCE reason. Bug record and spec wording corrected; the runner
skips only on rc < 0.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 08:11:26 +00:00
31ad79d325
feat(scheduler-allocator-runner): declare the allocator plugin contract (task 1.2)
xce_sched_allocator_iface_t (optional accepts guard + plan_init/next/fini)
and the sched_allocator_run() entry point in sched/alloc_iter.h; the
plugin's const allocator_iface reference, forward-declared in plugin.h
with the picker_iface ownership rule. No plugin uses them yet.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 07:49:25 +00:00
86ee3491dd
feat(scheduler-allocator-runner): plan generators initialise from the scheduling core (task 1.1)
xce_alloc_plan_iter_t.init takes xce_sched_core_t* instead of
xce_scheduler_t*; both built-in generators, both allocators' call sites
and the contract-test stub follow. The parjob purity test now shows an
identical plan sequence for a NULL and a poisoned core.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 07:44:27 +00:00
4e48462968
fix(tools): a failing apiserver Go test fails xce-build.sh (bug-1895)
The apiserver and all components ran the frontend Go tests as one list,
'cd frontend/apiserver && go test ... && cd ../xcecli && go test ...'.
set -e ignores a failure of any command in an && list except the last,
so a failing apiserver go test only short-circuited the list: the build
reported success and the xcecli tests never ran. Run each suite as its
own command in a subshell (which also keeps the cd from leaking).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 04:31:25 +00:00
141ac22c54
fix(apiserver): an accepted job is in the job store at once (bug-1894)
The apiserver's job store learned of a new job only from the
controller's asynchronous OnJobCreated / array push, so a 'jobs kill'
(or query) straight after 'jobs submit' could answer 'job not found'
for the job just accepted (2 of 4 immediate kills).

services.SubmitJob now records the accepted job right after the
controller's reply (which comes only after the job is committed and
applied): one record per element, base+i for arrays, stored with the new
DataStore.PutIfAbsent and marked JobInfo.Provisional (json:"-", not a
column). An OnJobCreated that already arrived is never overwritten; the
push that arrives later replaces the provisional record -- including in
dispatchJobArray, whose duplicate check no longer treats a provisional
element as already known.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 04:23:10 +00:00
0f4d23e3cd
fix(frontend-test): regression suite accepts dashed job IDs and no longer hangs on stdin (bug-1765)
xcecli prints a submitted job's ID in the dashed
{cluster}-{apiserver}-{timestamp}-{sequence} form (format.FormatJobID,
since d6008ead), but run.sh still matched 'Job <[0-9]+>' -- so 'jobs
submit' always failed and 'jobs kill' was skipped (28/1/1). It also
extracted the ID with [0-9]{10,}, which would have picked out only the
timestamp. Match the dashed form and take the whole ID from between the
angle brackets; 'jobs kill' accepts it back.

'jobs submit' reads a job script from stdin whenever stdin is a pipe,
so the suite hung forever when run from a script or CI with an open
pipe; the submit now reads /dev/null.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 04:13:26 +00:00
f5c887dc50
fix(controller-test): clean_failover_preservation.sh runs again (bug-1721)
The script required a pre-generated <repo>/ha/data (cluster.bin + TLS
certs) that ee8473bd removed, and called deploy-ha.sh with that dir as an
extra argument; deploy-ha.sh now generates both into the deploy dir and
takes <binary> <top-dir> <peers>. It failed at its first precondition in
every checkout.

Behind that, both committed-LSN boundaries still assumed bug-845's doubled
submits (1 + 2*N_JOBS): the commit wait never reached it and walcompare
compared an LSN past every member's WAL. Since that fix each job is one
WAL entry, so the boundary is 1 + N_JOBS.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 03:51:20 +00:00
980a0054d1
fix(tools): one xce-build.sh run per build dir at a time (bug-1604)
Two xce-build.sh runs against the same build dir raced: one run's
setup_build rm -rf hit files the other was writing ('Directory not
empty'), or deleted a binary the other had just built and tested.

xce-build.sh now takes an exclusive per-build-dir lock right after cd-ing
into src/ and holds it until it exits; a second run against the same dir
waits, runs against different dirs are unaffected. The lock
(src/tools/build-lock.sh) is an mkdir lock holding the owner PID, outside
the build dir, keyed by a hash of its absolute path; it needs only
coreutils (the nix dev shell has no flock) and takes over a lock whose
owner died. src/tools/test-build-lock.sh tests it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 03:43:48 +00:00
16b7aa4b60
fix(lib,scheduler): zarray_get rejects negative indexes; info resource bounds by capacity (bug-1824)
zarray_get_ex() checked only idx >= caps, so a negative index read
array->data[idx] out of bounds -- reachable through a resource whose
registration failed and kept idx == -1 (xcelimit.c add_vres()). It now
returns NULL for idx < 0.

repl_info_resource_one() bounded a shared resource's slot by
sharedInst.count / sharedInUse.count, the number of populated slots, not
the capacity: a populated slot above that count showed as value=(none).
It now relies on zarray_get()'s own bounds check.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 03:34:57 +00:00
cfa0a94166
fix(scheduler): the simulator's threads stop when its run returns (bug-1632, bug-1763)
sched_sim_driver_run() started the picker and allocator threads on
&sim->core and returned with them still running; only
sched_sim_driver_fini() stopped them, and 24 test files (35 call sites)
run the driver on a stack-local sim without calling it. The leftover
threads kept a pointer into the dead frame, and ASAN occasionally caught
one reading it (stack-use-after-return in cbuf_get) in
sched_sim_verify_test and sched_sim_record_output_test.

sched_sim_driver_run() now wraps the loop and stops both threads on
every return (new sched_sim_driver_stop_threads(), shared with fini). If
the allocator fails to start, the running picker is stopped too. The
stop no longer pushes a jobid 0 to wake the allocator: cbuf_get/put wait
in pthread_cond_wait, a cancellation point, and the fake end-of-session
was visible to the allocator.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 02:47:24 +00:00
4922f6c844
fix(controller): record standby state under its absolute peer index (bug-1891)
A promoted vote endpoint carries a peer index relative to the votes
array (which leaves out the primary); a late-joiner's heartbeat endpoint
carries the absolute index. repl_ack_decode_and_dispatch recorded state
reports under the raw index, so with ctrl-1 primary, ctrl-3's vote
endpoint (relative 1) and a rejoined ctrl-2's heartbeat endpoint
(absolute 1) shared standby_reported_*[1]. Each handler then acted on
the other's position over its own endpoint: the snapshot meant for the
empty, rejoining ctrl-2 went to the caught-up ctrl-3, and ctrl-2 never
caught up.

Resolve the absolute index with ha_repl_ack_abs_peer_idx (as the
WAL-ack branch already did) for the state report and for
ha_walack_on_ack, and size catchup_inflight by MAX_HA_CANDIDATES so
every absolute index fits.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 15:12:53 +00:00
8b561547dd
fix(lib): netio TCP connections run with TCP_NODELAY (bug-1890)
With Nagle on, the small TLS record that ends a burst -- a packet header,
the last short chunk of an HA snapshot, a reply -- waited for the peer's
delayed ACK, ~40 ms on Linux: seen on 5 of 40 snapshot transfers. Set
TCP_NODELAY on every netio TCP socket, outbound (socket_create) and
accepted (socket_accept).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 14:38:57 +00:00
754d5bcf3e
fix(lib): a stale OpenSSL error no longer kills a healthy connection (bug-1871)
SSL_get_error() reports SSL_ERROR_SSL whenever the calling thread's
OpenSSL error queue is non-empty. io_worker serves every endpoint from
one thread, so an error left by another connection -- a ctrlstatus
client's EOF queues 'unexpected eof while reading' in OpenSSL 3 -- turned
the next plain would-block SSL_read on a healthy connection into a fatal
error. handle_error() then tore that connection down; mid snapshot
transfer this left a rejoining standby with a truncated snap.part until
its liveness timeout forced a reconnect and a fresh transfer.

Clear the queue before each SSL_read/SSL_write/SSL_do_handshake so the
classification is about that call only.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 13:59:44 +00:00
9c51de88c6
fix(lib): percent limits render as configured; xcelib_test counts every failure again
test_fmt_limit_units_percent_limit_ignores_units_mode failed on every
run: fmt_limit_units() widened the float limit value before scaling it,
so 0.8 printed as "80.000001%". The product is now taken in float and
widened after, which gives the configured 80.

The failure never counted: test_libxce_retry(), one group of test.c's
single runner, ran its own UNITY_BEGIN()/UNITY_END(), which reset Unity's
counters -- every failure in a group before it was dropped and
xcelib_test exited 0. With the pair removed the runner reports 695 tests;
this was the only hidden failure.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 10:31:54 +00:00