The loader now stores the effective interval (10 s when unset or 0, LSF's
JOB_TERMINATE_INTERVAL default) and refuses a negative one; app_schema.json
describes the setting; src/docs/xce.md documents kill/stop behaviour and
the interval.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A plain kill sent one SIGTERM and never escalated, and a plain stop sent
one uncatchable SIGSTOP. Following LSF's TERMINATE and SUSPEND actions, a
plain kill now sends SIGINT, then SIGTERM, then SIGKILL, and a plain stop
sends SIGTSTP (the job can catch it, prepare and stop itself), then SIGSTOP
if it has not stopped; each next signal follows after the grace period and
only while the job is alive (or not yet stopped). A resume cancels a
pending stop; the job's exit ends any sequence. An explicit signal is
delivered once; a force kill is SIGKILL at once.
- JobSignal gains grace (capnp @2, xce_signal_t.grace): > 0 asks for the
sequence, in seconds between signals.
- apiserver: job.terminate_interval (default 10 s, LSF's
JOB_TERMINATE_INTERVAL); jobCtrlParams sends kill = SIGKILL and stop =
SIGSTOP with that grace when no signal is given; the handler no longer
pre-fills a default signal.
- worker: grace travels through the job-control task, the kept-signal table
(pending_signal_*_ex) and the IPC message; the job process runs the
sequence (job_signal_seq_*, uv timer) and checks stop state in /proc.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
worker_signal_callback() sent SIGTERM for a KILL request without force and
SIGSTOP for any TSTP request. job_signal_to_deliver() (new job_signal.c) now
delivers the requested signal; only a forced terminate escalates to SIGKILL.
The apiserver's default signal for a plain stop becomes SIGSTOP (it was
TSTP and relied on the worker's substitution), so a plain stop still
suspends jobs that ignore SIGTSTP.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
preCheckJobRequest() returned from inside its job-name branch, so every
named or array job skipped the description, project and resreq checks,
resreq validation and the shell-limit conversion that follow. The name now
only sets the job count and array flag, and every submission runs the rest.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Cause: AddRolesForUser and DeleteRolesForUser let an rbac:control grant
narrower than all (e.g. scope self, an accepted domain type for rbac
permissions) change roles whenever the target user was the caller, so a
user holding only rbac:control/self could grant itself cluster_admin
(confirmed by TestAddRolesForUser_SelfScopeCannotGrantItselfRoles: HTTP
200 and cluster_admin assigned).
Fix: canChangeUserRoles requires an rbac:control grant with scope all
(checked over every grant, before deduplication prefers a narrower scope)
to grant or revoke roles for any user, the caller included; anything else
gets 403. Chosen over a "cannot grant permissions you do not hold" rule
because a scoped role change has no safe meaning here and the subset
check across domains would be complex and easy to get wrong.
Tests: TestAddRolesForUser_SelfScopeCannotGrantItselfRoles,
TestDeleteRolesForUser_SelfScopeRejected, TestUserRoleChanges_AllScopeAllowed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Cause: Logout only deleted tokens/<user>/token, a file that token
validation (ValidateHandler / GetUsernameFromToken) never reads, so a
logged-out token stayed valid until its expiry (24 h by default).
Fix: an in-memory revocation set (controllers/token_revocation.go) keyed
by the token's jti (generateJWT now sets a random uuid jti, so two logons
in the same second give different tokens) or, for tokens without a jti,
by SHA-256 of the token; each entry is kept until the token's expiry and
expired entries are purged on every revocation. Logout revokes the token;
the shared ParseTokenClaims (used by ValidateHandler, RequirePermission
and handlers via GetUsernameFromToken) rejects revoked tokens. A missing
token file no longer fails logout. Scope: per process, not persisted and
not shared between apiservers.
Tests: TestLogout_RevokesToken (middleware), TestTokenRevocations_*,
TestIntegration_PostLogoutTokenRejected (replaces the design-gap test).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Cause: config/app.yaml shipped secretKey "xce-api-key" and
resolveJWTSecret used it whenever XCE_JWT_SECRET was unset, so anyone
could forge tokens for any user on such a deployment.
Fix: resolveJWTSecret uses XCE_JWT_SECRET, else the config value, but
treats an empty value or the old public default as unset; then it
generates a random 256-bit per-process secret and InitConf logs a warning
(secret never logged): tokens do not survive a restart and apiservers
sharing one endpoint must set the same XCE_JWT_SECRET. The shipped
app.yaml no longer carries a secret. The controllers test secret is no
longer the old default.
Tests: TestResolveJWTSecret_NeverUsesKnownDefaultOrEmpty,
TestShippedAppYAML_NoJWTSecret, TestInitConf_NoJWTSecretGeneratesAndWarns,
TestInitConf_JWTSecretFromEnvNoWarning.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Cause: the router used gin.Default() without SetTrustedProxies, and gin
then trusts forwarding headers from every peer, so ClientIP() returned
whatever X-Forwarded-For a client sent: the IP whitelist and the IP bound
into JWTs could be spoofed.
Fix: new app.yaml option security.trusted_proxies (IP or CIDR list,
default empty = trust no proxy, validated at startup) passed to
SetTrustedProxies in Router(); an invalid list falls back to trusting
none. The nginx samples note that the proxy must be listed there.
Tests: TestRouter_IgnoresForwardedForByDefault,
TestRouter_HonoursForwardedForFromTrustedProxy, TestResolveTrustedProxies.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Cause: ReturnError only writes the response and never aborts the gin
chain; RequirePermission (token error) and WhitelistIPMiddleware (invalid
client IP) returned after ReturnError without ctx.Abort(), so the route
handler still ran after the rejection.
Fix: ctx.Abort() after ReturnError at both sites (and, for clarity, on the
ValidateHandler body-check path, which validateRequestBody already
aborted). RequirePermission now answers a bad token with 401 instead of
500. ReturnError stays non-aborting (documented): handlers such as
ControlHost call it from helpers and then test ctx.IsAborted(), so a
global abort would change handler flows, and it adds nothing in handlers.
Tests: TestRequirePermission_AbortsOnInvalidToken,
TestWhitelistIPMiddleware_AbortsOnFailure.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Cause: the ValidateHandler parse callback (middleware isValidToken)
returned the secret without looking at token.Method, and
GetUsernameFromToken accepted any HMAC variant, so tokens signed with
HS384/HS512 were accepted although the server only signs HS256.
Fix: one JWTSigningMethod (HS256) used for signing, plus a shared
JWTKeyFunc (rejects any other alg) and JWTValidMethods parser option used
by both parse paths.
Tests: TestIsValidToken_PinsSigningAlgorithm (middleware),
TestGetUsernameFromToken_PinsSigningAlgorithm (controllers).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Cause: ConfigFileProvider.Authenticate compared the stored plaintext
password with != (timing side channel, passwords readable from the file),
and config/auth.yaml shipped admin/admin, user1/user1 etc. as plaintext
with nothing reporting that a default credential was active.
Fix: auth.yaml passwords may be bcrypt hashes (golang.org/x/crypto/bcrypt,
v0.23.0 as already pinned in go.mod, now vendored); plaintext entries are
still accepted but compared in constant time (SHA-256 + subtle), and an
unknown user costs the same bcrypt work. The provider reports startup
warnings (SecurityWarner, logged by main) for every plaintext entry and for
every shipped default credential still active. The shipped auth.yaml and
auth.yaml.example now store bcrypt hashes of the same dev passwords, so
tests/regression/run.sh (admin/admin) keeps working. tools/encrypt gains
-action=hash to generate entries.
Tests: TestConfigFileProvider_BcryptPassword,
TestConfigFileProvider_SecurityWarnings, TestShippedAuthConfig_HashedDefaults.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Cause: NewConfigFileProvider read cfg.ConfigFile.Users without checking
that the optional config_file block was present, so provider: config_file
with no block panicked with a nil-pointer dereference at startup.
Fix: return a clear configuration error when the block is missing.
Test: TestNewConfigFileProvider_Errors/missing_config_file_block.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Cause: ControlJob (backend/clib.go) looked its content argument up as a
signal name, but the job control handlers pass a JSON JobControlEvent, so
the lookup always missed and every kill sent TERM (stop TSTP, resume
CONT); sig.force and ckpt.kill were hard-coded to 0 and the checkpoint
method was never set.
Fix: new pure jobCtrlParams (backend/convert_pure.go) decodes the event
(or a bare signal name/number), validates the signal against
JobSignalMapTable and returns an error instead of falling back, and maps
force_kill to xce_signal_t.force (kill) or xce_ckpt_options_t.kill
(checkpoint) plus the checkpoint method. ControlJob fills the C structs
from it. Tests: backend/jobctrl_params_test.go.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Cause: modelJobToSubmit() filled only cmdline, execuser, launch_type and
pre/post-exec of job_startinfo_t, so the extended data the submit handler
builds (env, *_limit incl. -W run limit, file transfers, interactive mode),
the cwd and the exec home never reached the controller or the worker.
Fix: map the start info in a pure submitStartInfo() (exdata from
ExtendData, execcwd from ExecCWD or CWD, exechome from ExecHome) that
modelJobToSubmit copies into the C struct; decode exdata back in
cJobToModel/cJobArrayToModels so the store keeps ExtendData and CWD.
Reject extend_data the worker's parse_exdata() cannot take (not a JSON
object, over EXDATA_LENGTH-1 bytes) and drop an empty transfer list
instead of sending a null.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
bug-1934: every job-permission check read a single scope and compared it
only with "self", so a job:view or job:control grant scoped to self_ug acted
like "all" (list, get by ID, control), and self + self_ug collapsed to self.
userJobScope() now collects all of a user's grants of a job permission
before deduplication and adds them up (all; self; self_ug = the user and
everyone sharing a user group). The job list appends a submit_user filter
for narrower scopes, get-by-ID keeps covered rows in a new slice (removing
from the ranged slice skipped rows) and answers 403, both control paths
and job output check the owner against the scope. User groups now reach
the apiserver: the config push converts cluster_config_t.ugroups and
OnUserGroupsUpdated() replaces the new store (nested groups, all,
exclusions). The default basic_user now also sees its user-group mates'
jobs, as its configuration says.
bug-1935: project_name and license_project are accepted job filters, so
xcecli jobs list -P works.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The controller computed run usage (CPU time, memory, CPU peak and
efficiency, LSF bjobs -l definitions) but the capnp Job struct had no
field for it, so the job pushed to the apiserver never carried it, and
snapshots (the same struct) dropped it: a job restored from a checkpoint
lost its accumulated usage.
- controller.capnp: Job.runUsage @13 (JobRunusage); old entries decode
with no run usage.
- apiserver: map run_usage into JobInfo; max_mem, avg_mem, used_mem,
cpu_time, peak_cpu, peak_cpu_time, peak_cpu_duration, mem_efficiency,
avg_cpu_efficiency, peak_cpu_efficiency become queryable columns.
- xcecli: -o fields MAX_MEM, AVG_MEM, MEM_EFFICIENCY, CPU_PEAK,
CPU_PEAK_DURATION, CPU_EFFICIENCY, CPU_PEAK_EFFICIENCY (CPU_TIME now
renders too), and a MEMORY USAGE / CPU USAGE block in jobs list -l.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- apiserver peek (parseJobIDParam): <array-id>[k] now resolves to the
element; it used to drop [k] and look up the array identity itself
(and panicked in parseArrayName on some inputs). A job number still
names that job exactly (an element by its own XCE_JOBID); ranges and
invalid selections are rejected.
- apiserver restart: the decimal-only pre-check (isValidJobId, now
unused and removed) rejected the written form c-a-ts-seq[k];
ParseIdentitySelection, already used for the lookup, validates it.
- xcecli jobs restart converts its argument with ToAPIJobID like the
control commands, so written IDs work and invalid ones fail locally.
- docs: the peek/restart caveat in the Job arrays section.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
It logged in to a hard-coded http://localhost:8080 with fixed
credentials, so 'go test ./...' failed with connection refused. It now
skips unless XCE_LOAD_TEST_URL is set, with the credentials in
XCE_LOAD_TEST_USER/XCE_LOAD_TEST_PASSWORD; 'go test ./...' is green.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Formatting only (gofmt -w on the 29 non-vendored, non-generated files
gofmt -l reported); swag-generated apiserver/docs/docs.go left as is.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
RestartJobByID looked the job up with a job_id CompCondition, which the
store never evaluates, and restarted jqueryReply[0] -- the first stored
job -- for any ID, including unknown ones. It now parses the ID with
ParseIdentitySelection, fetches the rows by key and matches them with
foundJob, as ControlJobByID does: an ordinary job or <array-id>[k] is
restarted, an unknown job or element is not found, and a bare array
identity (several elements) is refused. Restart goes through the
generateJobID/submitJob seams so the handler fixture can test it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Follows page_token with limit=3 over the handler fixture. Before
efff6b65 the token's float-rounded array-element ID matched no row and
every page restarted at row 0 (the original repeat-the-first-page
report); it now pages through every row exactly once.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
findPageTokenIndex decoded the token's unique_job_id into a float64;
array element IDs set bit 62, where float64 spacing is 1024, so paging
skipped rows or restarted at row 0 -- an endless loop for callers that
page until the token is empty (control/query of arrays over 1000
elements). Decode with UseNumber and keep integers as int64.
Found by the job-array-one-based-index feature validation.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
bug-1904: both scheduler JOB_UPDATE paths parsed "" as-is:
parse_resreq() returns success with a PRIMARY container whose body is
NULL, and apply_job_task_count() wrote through it. Use ingest's default
("type == any") instead.
bug-1906: xcecli jobs list sent local selection-parse errors through
ExtractErrorMessage, which strips brackets and the quoted selection;
report them as written.
Also logs bug-1905 (xcecd_virtual ignores kill of a running job). All
three found by job-array-one-based-index task 10.2's live REST flow.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The comment still described arraySize > 1 as the array path; since task
9.1 idgen() rejects any size other than 1 and arrays use
GenerateArrayJobID.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Submitting an array prints the server's message with the array ID in
written form (no [k]) and the element count, e.g.
'Job <0-0-...> is submitted to queue <normal> (3 elements).'; ordinary
submits print exactly as before. The client model mirrors array_size.
Submit help documents name[1-N] and name[1], numbering from 1 and the
API-server limit (default 1000, at most 32767).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ID lists split only at commas outside brackets and every selection is
validated before any request (control allows identity ranges, listing
refuses them). A raw -J value with brackets is checked before the
wildcard mapping, and the search rule and * stripping apply to the
name part only (-J 'foo*[1]' lists element 1 of arrays whose name
starts with foo); values without brackets map exactly as before.
Control results print each item's own job ID when the server sends it,
so a pre-upgrade element 0 shows as <array-id>[0]. Control and list
help show <array-id>[k] and <array-id>[a-b]; list help now says -J.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
xcecli's format package gains ParseSelection (written or decimal key
with optional [k]/[a-b], the server's grammar and check order; 5.8 and
5.7 also on decimal keys with [...], so an upgraded client protects
itself against an older server; query ranges refused after the other
checks), the bracket-aware SplitSelections, the raw -J
ParseNameSelection and Selection.APIKey. ParseJobID is bounded;
ToAPIJobID(s, allowRange) no longer passes unrecognised text through;
peek keeps today's behaviour via ToAPIJobIDLegacy.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Filter-based ControlJob validates a bracketed -J value before fetching
(400 with the parser's message; unbracketed names, including commands
with spaces, unchanged). FetchJobList evaluates bracketed job_name
entries through the selection predicates instead of unevaluated
compound conditions, so -J 'foo[1]' acts on element 1 of every array
named foo and never on ordinary jobs of that name. The most-recent-job
rule applies only without an element number and without all-jobs, and
treats the element of a one-element array like an ordinary job
(isSingleElementArray, a documented approximation). The old
index-section parser is deleted. Completes bug-1898 on the owned paths.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ControlJobByID parses every selection with ParseIdentitySelection before
any store access (first rejection: 400, nothing dispatched), collects
the jobs to act on de-duplicated by unique_job_id plus one not-found
result per missing number per selection (carrying the would-be element
ID), validates each job once after the loop and dispatches each
accepted job once. foundJob gains an explicit-index flag so
<array-id>[0] matches only element 0 and never falls back to the whole
array or an ordinary job. Fixes bug-1899 (earlier selections were
re-dispatched for every later one).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
GetJobsInfo validates every filter entry in request order before any
store access (including the exec_hosts lookup) and GetJobByIDs every key
before querying; the first rejection returns 400 with the parser's
message. Bracketed job_id/job_name entries are evaluated as row
predicates before sorting and pagination, so job_id:eq:X[2] lists only
element 2, GET /jobs/X[1-3] lists elements 1-3 and <ordinary-id>[0]
lists nothing. The unevaluated compound conditions and the old index
parsing are gone from both handlers. Fixes bug-1898 on the query paths.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
BuildSelectionPredicates converts each job_id/job_name filter entry
containing a bracketed value into one row predicate (values OR'ed,
entries AND'ed with each other and the plain conditions), removing only
that entry. Operators are honoured instead of dropped (bug-1898):
eq/in/between select, notin excludes, name pattern operators reuse
today's condition semantics on the name part, and ne / comparison /
null operators keep today's rejection. Element selections match only
array elements in range; bare identities match the whole array and any
ordinary job; an empty name under exact operators matches nothing.
Bare written keys are normalised to decimal on every path. Not yet
wired into the handlers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ParseIdentitySelection (strict written form or base-10 decimal key,
optional [k] or [a-b]) and ParseNameSelection (name part may be empty,
no whitespace or brackets) apply the shared selection grammar and check
order (5.16 syntax, 5.14 width, 5.17 marked decimal without [k], 5.8
alignment only with [...], 5.7 element bound, 5.5 reversed), reporting
only the first failure with a rule tag and a message quoting the
selection. Runs the shared selection-cases table plus REST-only rows.
Not yet wired into the handlers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
preCheckJobRequest returns (n, isArray, err): a bracketed job name is
parsed by ParseSubmitIndexExpr first, so name[1-N] and name[1] create
arrays (one-element arrays included) allocated with
generateArrayJobID, stored under the bare name; expression errors are
answered 400 with their specific message and create nothing; names
without brackets take the unchanged createArrayName path. The response
names the array (jobid = array ID, new array_size, "(n elements)"),
submit-time %I/%J use the first element, and the submit and dispatch
logs identify arrays by array ID. Fixes bug-1896 for REST and xcecli.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ParseSubmitIndexExpr accepts only name[1-N] and name[1] (leading zeros
allowed) and otherwise reports the lowest-numbered broken rule with its
specific message: interactive arrays (2.1), invalid name or malformed
expression (2.2), lists/steps (2.3), %N (2.4), start 0 (2.5), reversed
(2.6), start above 1 (2.7), above the limit (2.8). Oversized numbers
saturate for the bound checks while reversal compares the digit
strings, so the rule order holds for any input. Not yet wired into
CreateJob (task 6.2).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
CreateJob's ID generation and submission and both control dispatch
sites now go through package variables (generateJobID,
generateArrayJobID, submitJob, controlJob) that default to the real
backend/services functions, so production behaviour is unchanged. A new
non-cgo handler test file provides a reusable fixture: it saves and
restores every global it touches, sets up auth/RBAC, seeds jobs and
their queues, and records each dispatched job.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
QueryOptions gains Predicates; store.ExecuteQuery keeps a row only if
every predicate returns true, after the plain conditions and before
sort, pagination and field selection, so page boundaries and tokens
count only matching rows. Nil/empty predicates leave every existing
query unchanged. Foundation for evaluating element selections
(bug-1898).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
job.max_array_size is resolved by a pure resolveMaxArraySize: unset or 0
gives 1000, 1..32767 is used as is, values above 32767 are clamped to
32767 with a startup warning naming the configured and effective value,
and a negative value refuses startup. The old >2147483646 rejection is
gone; the shipped config documents the bounds.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
GenerateArrayJobID wraps libxce idgen_array (first element, index 1;
one-element arrays included) with a non-cgo stub; pure helpers
ParseHumanJobIDStrict (written identity with exact field-width checks,
uint64 overflow reported as a width error), IsArrayCapableKey and
MakeElementID (bit-identical to libxce jobsel_element_id).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The apiserver's job store learned of a new job only from the
controller's asynchronous OnJobCreated / array push, so a 'jobs kill'
(or query) straight after 'jobs submit' could answer 'job not found'
for the job just accepted (2 of 4 immediate kills).
services.SubmitJob now records the accepted job right after the
controller's reply (which comes only after the job is committed and
applied): one record per element, base+i for arrays, stored with the new
DataStore.PutIfAbsent and marked JobInfo.Provisional (json:"-", not a
column). An OnJobCreated that already arrived is never overwritten; the
push that arrives later replaces the provisional record -- including in
dispatchJobArray, whose duplicate check no longer treats a provisional
element as already known.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
xcecli prints a submitted job's ID in the dashed
{cluster}-{apiserver}-{timestamp}-{sequence} form (format.FormatJobID,
since d6008ead), but run.sh still matched 'Job <[0-9]+>' -- so 'jobs
submit' always failed and 'jobs kill' was skipped (28/1/1). It also
extracted the ID with [0-9]{10,}, which would have picked out only the
timestamp. Match the dashed form and take the whole ID from between the
angle brackets; 'jobs kill' accepts it back.
'jobs submit' reads a job script from stdin whenever stdin is a pipe,
so the suite hung forever when run from a script or CI with an open
pipe; the submit now reads /dev/null.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- bug-1872 (worker): "Failed to find any disk statistics in
/proc/diskstats" ERROR every 15 s on an NVMe-only host. The whole-disk
filter guessed from the name and skipped anything ending in a digit or
containing 'p' -- every nvmeXnY, mmcblkN, even sdp -- so io was never
reported. diskstats.h now counts a device only when
/sys/block/<name>/device exists (real disks; not partitions, loop, dm,
md, zram), and a host with no disk warns once.
- bug-1873 (worker): "Failed to get worker pid" WARN on job start. MainPID
reads 0 until systemd has forked the new transient unit; xcecd.c already
retries, so the miss is DEBUG and the final failure names the unit.
- bug-1874 (scheduler): deprecation WARN for the legacy xce_sched_fcfs on
every start, though builtin_fcfs is the picker. Registration still wraps
a job_elect_fn plugin but no longer warns; selecting one warns once.
- bug-1875 (apiserver): "Not running as root -- PAM authentication will not
work" with the default provider config_file; now only for provider pam.
- bug-1876 (controller): standbys logged "cluster_init_phase2 ... failed
(rc=-22)" WARN every 3 s while waiting for the primary's config push,
a state the function itself calls normal. It returns -EAGAIN for that
wait; callers log it at INFO, other failures still WARN.
- bug-1877 (controller): "WAL/snapshot stored unencrypted" warned without
looking. persist_path_encrypted() checks fscrypt (statx) and dm-crypt
(sysfs dm/uuid CRYPT-, through LVM slaves); the warning stays when the
storage is not shown to be encrypted (e.g. a plain /tmp).
Tests: worker diskstats_test (2 of 4 RED on the old filter with pc13v's
/proc/diskstats), controller test_phase2_cold_boot and
test_persist_encryption, scheduler picker tests, apiserver
TestRootWarning. Full ASAN build: all 113 suites pass (controller 319/0,
harness 141/0).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Gitea #47: `xcecli jobs submit -R "rusage[mem=100000000a0]"` was accepted
with a normal "Job <...> is submitted" reply. The apiserver only
length-checked resreq; the real PEG grammar ran later, asynchronously, in
the scheduler's core_build_job(), which logged a server-side WARN and
dropped the job -- it then sat PEND forever with no sign of why.
- libxce: xce_check_rusage_syntax(resreq, &errmsg) runs parse_rusage() on
every rusage[...] clause (extracted the way section.peg does: up to the
first ']'), so primary, compound and alternative requests are all
covered. No resource manager needed, so select[]/order[] resource names
are still only checked by the scheduler (the documented scope).
- apiserver/backend: ValidateResreqSyntax() cgo wrapper (resreq_check.go);
the !cgo stub returns nil (no job can be submitted without CGO anyway).
- controllers/jobs.go preCheckJobRequest(): calls it after the length
check via a package var, so a malformed request is refused up front.
Tests: lib test_check_rusage_syntax (RED before: the reported input was
accepted) and backend TestValidateResreqSyntax (RED before) through the
real grammar; controllers TestPreCheckJobRequest_RejectsMalformedResreq
checks the call site with an injected validator (written with the call
site, so no separate RED). libxce, apiserver backend (cgo) and controllers
tests pass. apiserver/middleware TestLoadJobs fails here and on the
baseline alike: it needs a live apiserver on :8080.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ControlJob() set xce_jobctrl_t.action to the job_ctrl_type_t action code
(kill/stop/resume/checkpoint), but that field is actually the capnp
JobControl_u_which union discriminant -- a different enum entirely. This
made kill/resume/checkpoint hit request.c's -EINVAL default case, and made
stop collide with the checkpoint discriminant and read through the wrong
union member. Live-verified against a real deployed cluster: kill/stop/
resume/checkpoint now behave correctly with no apiserver crash.
Also additively accept a human-readable job ID ("{cluster}-{apiserver}-
{timestamp}-{seq}") on the job control endpoint, alongside the existing
numeric form xcecli already translates to -- lets a caller hit the REST
API directly without reimplementing xcecli's ID packing. Live-verified via
a direct curl call with the human-readable ID.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hb6Edtj4KbS8CrgEWYdA2E
modelJobToSubmit() never set job_startinfo_t.execuser when converting
a submitted job to the C struct sent to the controller/worker, so
every job ran as the worker's 'nobody' default and failed to exec
(exit 126). Default execuser to the submitting user via a new
resolveExecUser() pure helper, and free the newly allocated C string.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hb6Edtj4KbS8CrgEWYdA2E
NJOBS/RUN/SSUSP/USUSP (xcecli hosts list -l) and EXEC_HOST (xcecli jobs
list) always showed a hardcoded 1 slot per host regardless of a job's
actual per-host allocation -- rawJobToModel()'s ExecHosts JSONB builder
never read the real per-host "slots" resource from the C job allocation,
so the already-correct downstream slot-counting logic (computeHostSlotUsage,
bug-1743) and EXEC_HOST's "N*host" formatter never had real data to work
with. convert.go now extracts each host's real slots allocation via a new
CGO indexing helper, mirroring the scheduler's own get_nslots_in_alloc()
convention.
Also wires hosts list -l's "CURRENT LOAD USED FOR SCHEDULING" table (tmp/
swap/mem/slots and friends) to the host's real live resource values --
previously entirely hardcoded placeholder strings, never connected to any
backend data.
The terminal step: src/frontend/Makefile (194 lines, the second,
independently-maintained build/test path this whole spec exists to
eliminate) is deleted. Re-ran the full task 4.1 validation suite with
the file genuinely absent (build, meson test --suite apiserver, and
-- going beyond citing task 3.5 -- a real live run-resource-verify.sh
end-to-end run: VERDICT result=PASS, closing design.md's Testing
Strategy "Discovery-path regression" check specifically). Full-repo
sweep: 181 hits, zero unaccounted, byte-identical to task 4.1's
non-Makefile hits (the only delta is the Makefile's own 8
self-referential lines, now gone since the file doesn't exist).
Independently re-verified by the task reviewer.
Meson is now the single build/test system for apiserver and xcecli.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Comment-only edits in 4 files: test_ha_bootstrap.c/test_ha_ctrlhello.c
rename their "make test-integration" pointers to reference the
regression suite (tests/regression/run.sh) directly; apiserver/xcecli
cmd/main.go drop the Makefile clause from their ldflags-injection
comments, keeping only src/tools/go_build. Zero production/test
logic changed -- verified via full controller unit-test suite rerun
(1261 tests, 0 failures) and xcecli build.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Removes every live current-fact reference to src/frontend/Makefile
targets or Makefile-specific fixed paths across 11 docs (steering,
frontend/CLAUDE.md, root CLAUDE.md, deployment docs, and two skill
docs), replacing with the Meson-based equivalents established by
tasks 1-3.2: /xce-build apiserver, meson test --suite apiserver,
direct tests/regression/run.sh invocation.
Round-1 review rejected the first pass: xce-deploy-ha/SKILL.md had
over-corrected by deleting all mention of deploy-ha.sh's still-real
/tmp/apiserver legacy fallback (that script's discovery logic is
unchanged, unlike run-resource-verify.sh's, which genuinely lost its
fallback in task 3.1 -- xce-test-resources/SKILL.md's identical-
looking edit was correct for that reason). Fixed by restoring an
accurate, Makefile-free description of the real fallback at all four
touched sites. Re-reviewed and approved.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Rewords deploy-ha.sh's block comment and two fallback echo lines, and
run.sh's XCE-not-found error, to stop naming the now-deleted
Makefile's build-server/build-cli targets, pointing at /xce-build
apiserver instead. Discovery/control-flow logic in both files is
unchanged (verified line-by-line: no if/elif/else lines touched);
run.sh's XCE default placeholder is untouched by design.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
Adds a new `test('apiserver backend', ...)` in apiserver/meson.build
reusing the existing apiserver_cgo_deps-derived CGO flags (resolving
their @CURRENT_SOURCE_DIR@/@BUILD_ROOT@ placeholder tokens via
Meson's own .replace(), since this target invokes go_compiler
directly rather than through tools/go_build). Adds timeout: 300
(matching the existing controller test's convention) since a
cold-cache CGO+ASAN build exceeds Meson's 30s default.
Verified cold and warm cache; confirmed CGO_ENABLED=1 selects the
real backend/ CGO path, not the //go:build !cgo stub.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
