preCheckJobRequest() returned from inside its job-name branch, so every
named or array job skipped the description, project and resreq checks,
resreq validation and the shell-limit conversion that follow. The name now
only sets the job count and array flag, and every submission runs the rest.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Cause: AddRolesForUser and DeleteRolesForUser let an rbac:control grant
narrower than all (e.g. scope self, an accepted domain type for rbac
permissions) change roles whenever the target user was the caller, so a
user holding only rbac:control/self could grant itself cluster_admin
(confirmed by TestAddRolesForUser_SelfScopeCannotGrantItselfRoles: HTTP
200 and cluster_admin assigned).
Fix: canChangeUserRoles requires an rbac:control grant with scope all
(checked over every grant, before deduplication prefers a narrower scope)
to grant or revoke roles for any user, the caller included; anything else
gets 403. Chosen over a "cannot grant permissions you do not hold" rule
because a scoped role change has no safe meaning here and the subset
check across domains would be complex and easy to get wrong.
Tests: TestAddRolesForUser_SelfScopeCannotGrantItselfRoles,
TestDeleteRolesForUser_SelfScopeRejected, TestUserRoleChanges_AllScopeAllowed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Cause: Logout only deleted tokens/<user>/token, a file that token
validation (ValidateHandler / GetUsernameFromToken) never reads, so a
logged-out token stayed valid until its expiry (24 h by default).
Fix: an in-memory revocation set (controllers/token_revocation.go) keyed
by the token's jti (generateJWT now sets a random uuid jti, so two logons
in the same second give different tokens) or, for tokens without a jti,
by SHA-256 of the token; each entry is kept until the token's expiry and
expired entries are purged on every revocation. Logout revokes the token;
the shared ParseTokenClaims (used by ValidateHandler, RequirePermission
and handlers via GetUsernameFromToken) rejects revoked tokens. A missing
token file no longer fails logout. Scope: per process, not persisted and
not shared between apiservers.
Tests: TestLogout_RevokesToken (middleware), TestTokenRevocations_*,
TestIntegration_PostLogoutTokenRejected (replaces the design-gap test).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Cause: config/app.yaml shipped secretKey "xce-api-key" and
resolveJWTSecret used it whenever XCE_JWT_SECRET was unset, so anyone
could forge tokens for any user on such a deployment.
Fix: resolveJWTSecret uses XCE_JWT_SECRET, else the config value, but
treats an empty value or the old public default as unset; then it
generates a random 256-bit per-process secret and InitConf logs a warning
(secret never logged): tokens do not survive a restart and apiservers
sharing one endpoint must set the same XCE_JWT_SECRET. The shipped
app.yaml no longer carries a secret. The controllers test secret is no
longer the old default.
Tests: TestResolveJWTSecret_NeverUsesKnownDefaultOrEmpty,
TestShippedAppYAML_NoJWTSecret, TestInitConf_NoJWTSecretGeneratesAndWarns,
TestInitConf_JWTSecretFromEnvNoWarning.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Cause: ReturnError only writes the response and never aborts the gin
chain; RequirePermission (token error) and WhitelistIPMiddleware (invalid
client IP) returned after ReturnError without ctx.Abort(), so the route
handler still ran after the rejection.
Fix: ctx.Abort() after ReturnError at both sites (and, for clarity, on the
ValidateHandler body-check path, which validateRequestBody already
aborted). RequirePermission now answers a bad token with 401 instead of
500. ReturnError stays non-aborting (documented): handlers such as
ControlHost call it from helpers and then test ctx.IsAborted(), so a
global abort would change handler flows, and it adds nothing in handlers.
Tests: TestRequirePermission_AbortsOnInvalidToken,
TestWhitelistIPMiddleware_AbortsOnFailure.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Cause: the ValidateHandler parse callback (middleware isValidToken)
returned the secret without looking at token.Method, and
GetUsernameFromToken accepted any HMAC variant, so tokens signed with
HS384/HS512 were accepted although the server only signs HS256.
Fix: one JWTSigningMethod (HS256) used for signing, plus a shared
JWTKeyFunc (rejects any other alg) and JWTValidMethods parser option used
by both parse paths.
Tests: TestIsValidToken_PinsSigningAlgorithm (middleware),
TestGetUsernameFromToken_PinsSigningAlgorithm (controllers).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Cause: modelJobToSubmit() filled only cmdline, execuser, launch_type and
pre/post-exec of job_startinfo_t, so the extended data the submit handler
builds (env, *_limit incl. -W run limit, file transfers, interactive mode),
the cwd and the exec home never reached the controller or the worker.
Fix: map the start info in a pure submitStartInfo() (exdata from
ExtendData, execcwd from ExecCWD or CWD, exechome from ExecHome) that
modelJobToSubmit copies into the C struct; decode exdata back in
cJobToModel/cJobArrayToModels so the store keeps ExtendData and CWD.
Reject extend_data the worker's parse_exdata() cannot take (not a JSON
object, over EXDATA_LENGTH-1 bytes) and drop an empty transfer list
instead of sending a null.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
bug-1934: every job-permission check read a single scope and compared it
only with "self", so a job:view or job:control grant scoped to self_ug acted
like "all" (list, get by ID, control), and self + self_ug collapsed to self.
userJobScope() now collects all of a user's grants of a job permission
before deduplication and adds them up (all; self; self_ug = the user and
everyone sharing a user group). The job list appends a submit_user filter
for narrower scopes, get-by-ID keeps covered rows in a new slice (removing
from the ranged slice skipped rows) and answers 403, both control paths
and job output check the owner against the scope. User groups now reach
the apiserver: the config push converts cluster_config_t.ugroups and
OnUserGroupsUpdated() replaces the new store (nested groups, all,
exclusions). The default basic_user now also sees its user-group mates'
jobs, as its configuration says.
bug-1935: project_name and license_project are accepted job filters, so
xcecli jobs list -P works.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- apiserver peek (parseJobIDParam): <array-id>[k] now resolves to the
element; it used to drop [k] and look up the array identity itself
(and panicked in parseArrayName on some inputs). A job number still
names that job exactly (an element by its own XCE_JOBID); ranges and
invalid selections are rejected.
- apiserver restart: the decimal-only pre-check (isValidJobId, now
unused and removed) rejected the written form c-a-ts-seq[k];
ParseIdentitySelection, already used for the lookup, validates it.
- xcecli jobs restart converts its argument with ToAPIJobID like the
control commands, so written IDs work and invalid ones fail locally.
- docs: the peek/restart caveat in the Job arrays section.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Formatting only (gofmt -w on the 29 non-vendored, non-generated files
gofmt -l reported); swag-generated apiserver/docs/docs.go left as is.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
RestartJobByID looked the job up with a job_id CompCondition, which the
store never evaluates, and restarted jqueryReply[0] -- the first stored
job -- for any ID, including unknown ones. It now parses the ID with
ParseIdentitySelection, fetches the rows by key and matches them with
foundJob, as ControlJobByID does: an ordinary job or <array-id>[k] is
restarted, an unknown job or element is not found, and a bare array
identity (several elements) is refused. Restart goes through the
generateJobID/submitJob seams so the handler fixture can test it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Follows page_token with limit=3 over the handler fixture. Before
efff6b65 the token's float-rounded array-element ID matched no row and
every page restarted at row 0 (the original repeat-the-first-page
report); it now pages through every row exactly once.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Filter-based ControlJob validates a bracketed -J value before fetching
(400 with the parser's message; unbracketed names, including commands
with spaces, unchanged). FetchJobList evaluates bracketed job_name
entries through the selection predicates instead of unevaluated
compound conditions, so -J 'foo[1]' acts on element 1 of every array
named foo and never on ordinary jobs of that name. The most-recent-job
rule applies only without an element number and without all-jobs, and
treats the element of a one-element array like an ordinary job
(isSingleElementArray, a documented approximation). The old
index-section parser is deleted. Completes bug-1898 on the owned paths.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ControlJobByID parses every selection with ParseIdentitySelection before
any store access (first rejection: 400, nothing dispatched), collects
the jobs to act on de-duplicated by unique_job_id plus one not-found
result per missing number per selection (carrying the would-be element
ID), validates each job once after the loop and dispatches each
accepted job once. foundJob gains an explicit-index flag so
<array-id>[0] matches only element 0 and never falls back to the whole
array or an ordinary job. Fixes bug-1899 (earlier selections were
re-dispatched for every later one).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
GetJobsInfo validates every filter entry in request order before any
store access (including the exec_hosts lookup) and GetJobByIDs every key
before querying; the first rejection returns 400 with the parser's
message. Bracketed job_id/job_name entries are evaluated as row
predicates before sorting and pagination, so job_id:eq:X[2] lists only
element 2, GET /jobs/X[1-3] lists elements 1-3 and <ordinary-id>[0]
lists nothing. The unevaluated compound conditions and the old index
parsing are gone from both handlers. Fixes bug-1898 on the query paths.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
BuildSelectionPredicates converts each job_id/job_name filter entry
containing a bracketed value into one row predicate (values OR'ed,
entries AND'ed with each other and the plain conditions), removing only
that entry. Operators are honoured instead of dropped (bug-1898):
eq/in/between select, notin excludes, name pattern operators reuse
today's condition semantics on the name part, and ne / comparison /
null operators keep today's rejection. Element selections match only
array elements in range; bare identities match the whole array and any
ordinary job; an empty name under exact operators matches nothing.
Bare written keys are normalised to decimal on every path. Not yet
wired into the handlers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ParseIdentitySelection (strict written form or base-10 decimal key,
optional [k] or [a-b]) and ParseNameSelection (name part may be empty,
no whitespace or brackets) apply the shared selection grammar and check
order (5.16 syntax, 5.14 width, 5.17 marked decimal without [k], 5.8
alignment only with [...], 5.7 element bound, 5.5 reversed), reporting
only the first failure with a rule tag and a message quoting the
selection. Runs the shared selection-cases table plus REST-only rows.
Not yet wired into the handlers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
preCheckJobRequest returns (n, isArray, err): a bracketed job name is
parsed by ParseSubmitIndexExpr first, so name[1-N] and name[1] create
arrays (one-element arrays included) allocated with
generateArrayJobID, stored under the bare name; expression errors are
answered 400 with their specific message and create nothing; names
without brackets take the unchanged createArrayName path. The response
names the array (jobid = array ID, new array_size, "(n elements)"),
submit-time %I/%J use the first element, and the submit and dispatch
logs identify arrays by array ID. Fixes bug-1896 for REST and xcecli.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ParseSubmitIndexExpr accepts only name[1-N] and name[1] (leading zeros
allowed) and otherwise reports the lowest-numbered broken rule with its
specific message: interactive arrays (2.1), invalid name or malformed
expression (2.2), lists/steps (2.3), %N (2.4), start 0 (2.5), reversed
(2.6), start above 1 (2.7), above the limit (2.8). Oversized numbers
saturate for the bound checks while reversal compares the digit
strings, so the rule order holds for any input. Not yet wired into
CreateJob (task 6.2).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
CreateJob's ID generation and submission and both control dispatch
sites now go through package variables (generateJobID,
generateArrayJobID, submitJob, controlJob) that default to the real
backend/services functions, so production behaviour is unchanged. A new
non-cgo handler test file provides a reusable fixture: it saves and
restores every global it touches, sets up auth/RBAC, seeds jobs and
their queues, and records each dispatched job.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Gitea #47: `xcecli jobs submit -R "rusage[mem=100000000a0]"` was accepted
with a normal "Job <...> is submitted" reply. The apiserver only
length-checked resreq; the real PEG grammar ran later, asynchronously, in
the scheduler's core_build_job(), which logged a server-side WARN and
dropped the job -- it then sat PEND forever with no sign of why.
- libxce: xce_check_rusage_syntax(resreq, &errmsg) runs parse_rusage() on
every rusage[...] clause (extracted the way section.peg does: up to the
first ']'), so primary, compound and alternative requests are all
covered. No resource manager needed, so select[]/order[] resource names
are still only checked by the scheduler (the documented scope).
- apiserver/backend: ValidateResreqSyntax() cgo wrapper (resreq_check.go);
the !cgo stub returns nil (no job can be submitted without CGO anyway).
- controllers/jobs.go preCheckJobRequest(): calls it after the length
check via a package var, so a malformed request is refused up front.
Tests: lib test_check_rusage_syntax (RED before: the reported input was
accepted) and backend TestValidateResreqSyntax (RED before) through the
real grammar; controllers TestPreCheckJobRequest_RejectsMalformedResreq
checks the call site with an injected validator (written with the call
site, so no separate RED). libxce, apiserver backend (cgo) and controllers
tests pass. apiserver/middleware TestLoadJobs fails here and on the
baseline alike: it needs a live apiserver on :8080.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ControlJob() set xce_jobctrl_t.action to the job_ctrl_type_t action code
(kill/stop/resume/checkpoint), but that field is actually the capnp
JobControl_u_which union discriminant -- a different enum entirely. This
made kill/resume/checkpoint hit request.c's -EINVAL default case, and made
stop collide with the checkpoint discriminant and read through the wrong
union member. Live-verified against a real deployed cluster: kill/stop/
resume/checkpoint now behave correctly with no apiserver crash.
Also additively accept a human-readable job ID ("{cluster}-{apiserver}-
{timestamp}-{seq}") on the job control endpoint, alongside the existing
numeric form xcecli already translates to -- lets a caller hit the REST
API directly without reimplementing xcecli's ID packing. Live-verified via
a direct curl call with the human-readable ID.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hb6Edtj4KbS8CrgEWYdA2E
xcecli hosts list showed NUM_RUN/NUM_SSUSP/NUM_USUSP as always 0 --
the fields existed on HostInfo and were displayed, but nothing ever
assigned them. processHostData now derives per-host slot usage from
the live job store (exec_hosts + job_status), counting occupied slots
so the numbers are comparable to MAX_SLOT.
Partial fix for gitea#46 / bug-1743: the separate "dispatch stops at
8/16" question needs the reporter's live limits.yaml/pend-reason to
confirm a usergroup/queue slot limit, not addressed here.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y4M5MJM2zK6nU43da267Ku
xcecli jobs kill/stop/resume/checkpoint failed for every job with an
empty error body. models.JobInfo.JobStatus/.ArrayIdx are plain Go
`int`, but validateJobControl/foundJob asserted them as `int64` via a
bare type assertion, which always panicked; Gin's Recovery()
middleware swallowed the panic and returned an empty 500.
Fixes gitea#45 / bug-1742.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y4M5MJM2zK6nU43da267Ku
CreateJob now does FromHost: req.FromHost, a straight passthrough of
whatever xcecli (or any other client) sends in JobSubmitRequest.FromHost,
instead of deriving it server-side via ctx.ClientIP().
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AgYRU3zDTqNGV5Jk8aiKLg
GetJobsInfo/GetJobByIDs validated the -o/-l field list against
models.JobSubmitRequest{} -- the job-submission payload -- instead of
models.JobInfo{}, the query/response model (the already-correct
pattern used elsewhere in this file at line 1131). Response-only
fields like pending_reason don't exist on JobSubmitRequest, so any
such field made the whole listing fail outright with "unsupported
query field", not just omit that column.
core_limit/mem_limit exist on JobSubmitRequest but not on JobInfo --
switching wholesale would have newly broken those two previously-valid
fields. Added CoreLimit/MemLimit to JobAdditionalFields, the existing
merge-in bucket already used for pids/pgids/interactive_job_url.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AgYRU3zDTqNGV5Jk8aiKLg
When a worker re-joins and already exists in workermgr, makeWorker_()
now overrides the existing slot instead of returning EEXIST. Admin-set
flags (LOCKEDU, CLOSED, HIDDEN, CLOSED_BY_ADMIN) survive the override.
Add XCE_WORKER_CLOSED_BY_ADMIN flag to distinguish admin-closed hosts
from workload-closed ones. worker_ctrl sets/clears it alongside CLOSED.
Refactor worker_init/worker_destroy into copy_worker/kill_worker helpers
shared by both fresh creation and override paths.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Routes and RBAC entries already existed from prior P0-2a/2b work. This
adds three TDD tests that lock down the RBAC invariants:
- TestRBAC_HostControl_ClusterAdminOnly: cluster_admin has host:control
with domain=all; basic_user must NOT have host:control
- TestRBAC_HostView_BasicUserHasAccess: basic_user has host:view with
domain=all; queue_admin must NOT have host:view
- TestRBAC_HostControl_DomainTypeIsAll: cluster_admin host:control
domain type must be "all" (not hostgroup)
All apiserver tests passing.
1. Add //go:build !cgo to control_host_test.go so the stub-testing
tests only run in non-CGO builds, matching the stub.go build
constraint. Clean up dead nil check in SignatureCompiles test.
2. Replace two independent if-statements for actionType mapping in
ControlHost with a map lookup + explicit guard, consistent with
ControlQueue's ActionMap pattern. Prevents silent zero-value
actionType if SupportedHostActions expands later.
- backend/clib.go: Replace ControlHost stub with real implementation
using xcecli_hostctrl, following the same pattern as ControlQueue
- controllers/hosts.go: Remove package-level allFlag variable;
preCheckHostControlRequest now returns (bool, error) so the allFlag
is a local value in the caller, eliminating the data race
- controllers/queues.go: Same fix for preCheckQueueControlRequest
- Add unit tests for both preCheck functions verifying return values,
error cases, and concurrent safety
- Add compile-time verification test for ControlHost stub signature
