Commit graph

2299 commits

Author SHA1 Message Date
4f57904134
fix(apiserver): answer the controller's queue rejections as client errors (bug-1960)
Since the controller rejects submissions to closed or unknown queues
(E_XCE_QUEUE_CLOSED, E_XCE_BAD_QUEUE), the apiserver answered them as a 500
"xcecli_jobsubmit failed: -10016". SubmitJob now maps them to
backend.ErrQueueClosed / ErrQueueInvalid (numbers checked against errnox.h
at compile time), and the submit and restart handlers answer 400 naming
the queue, as the apiserver's own pre-check does. This covers a queue
closed between that check and the controller's.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:40:15 +00:00
b239db5fa3
fix(controller): reject submissions to closed or unknown queues (bug-1960)
Cause: handle_jobsubmit() never looked at the target queue, so a
submission that bypassed the apiserver's check (libxce, harness
xce_test, or a REST submit without a queue name) was accepted into a
closed queue, an unknown queue, or the reserved lost_and_found queue.

Fix: jobsubmit_check_queue() reads the queue's in-memory state under the
qmgr/queue read locks. handle_jobsubmit() calls it after the field
checks and before the in-flight jobid registry and any WAL append, for
single jobs and job arrays. It replies -E_XCE_QUEUE_CLOSED (new code,
"queue is closed") for a closed queue and -E_XCE_BAD_QUEUE for an
unknown or reserved one. Apply/replay (restore_job) is unchanged, so
committed submit facts for a now-closed queue still apply.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:38:42 +00:00
387cc99486
fix(build): run every apiserver Go test package in xce-build.sh (bug-1965)
Cause: the apiserver and all components ran go test on a fixed list
(./store/... ./utils/... ./controllers/... ./backend/...), so the
middleware, router, cmd and models tests (several security tests among
them) never ran in /xce-build; a deliberately broken middleware test left
the build green.

Fix: run ./... (vendor/ is excluded by the pattern) in both places.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:37:52 +00:00
f7fef278f6
docs(apiserver): fix swag annotations and regenerate the swagger spec (bug-1966)
Cause: the handler annotations referenced types that do not exist
(models.Error, ErrorResponse, JobsResponse, JobSignalRequest,
JobControlResponseSummary, unqualified JobSubmitRequest), had malformed
@Param/@Failure lines and /api/v1 paths, and there was no general API
info, so `swag init -g cmd/main.go` failed and docs/ stayed a stale stub.

Fix: use the real response envelopes (JsonStruct / JsonErrStruct) and
request models, the real routes under @BasePath /xce/v1, add general info
to cmd/main.go, and regenerate docs/ with `swag init -g cmd/main.go -o
docs`; the spec now has /jobs/{jobids}/control and /jobs/control.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:37:52 +00:00
1ff068f6e7
fix(apiserver): check the default queue exists and is open on submit (bug-1960)
Cause: preCheckJobRequest (controllers/jobs.go) ran the reserved-name,
existence and closed checks only when a queue name was given; an empty
name was set to "normal" and skipped them, so jobs were accepted into a
closed default queue.

Fix: default the name first, then run the same checks for every
submission. Two existing tests now seed the default queue. (The controller
side of bug-1960 is fixed separately.)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:37:52 +00:00
387a421c57
fix(apiserver): match JWT-excluded paths exactly, wildcard only for /* (bug-1962)
Cause: isExcludedPath (middleware/validatehandler.go) dropped the last
character of every excluded entry and prefix-matched, so /ping excused any
path starting with /pin (e.g. /pinX) from JWT validation, while
/swagger/*any (trimmed to /swagger/*an) never matched.

Fix: entries match exactly; an entry whose last segment is a gin catch-all
(/swagger/*any) matches any path under the part before the "*".

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:37:52 +00:00
d720132c8c
fix(apiserver): stop ControlHost after validateHost rejects the request (bug-1961)
Cause: validateHost (controllers/hosts.go) wrote a 403 via ReturnError
without aborting, so ControlHost's ctx.IsAborted() check never fired and it
went on to write a second (202) response; the permission was also checked
once per host and not at all for an empty host list.

Fix: validateHost writes no response; it checks host:control once up front
and returns an error, on which ControlHost writes a single 403 and returns.
For "all", the host list is resolved before the one validateHost call.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:37:52 +00:00
736ed81d65
fix(apiserver): require rbac:control scope all to create, extend or delete roles (bug-1958)
Cause: RoleController.CreateRole, AddPermissionsForRole and
DeleteRoleAndPermissions relied on RequirePermission("rbac:control"), which
accepts any scope, so a user with rbac:control scoped to self could add
cluster:control to a role it holds.

Fix: checkRoleChangePermission() in controllers/roles.go refuses with 403
unless one of the caller's grants (checked before deduplication) is
rbac:control with scope all, reusing canChangeUserRoles from bug-1952.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:37:52 +00:00
954c96cf5b
feat(xcecli): jobs kill --signal and --force
The control API already takes `signal` (kill only: that signal once,
instead of the graceful INT, TERM, KILL sequence) and `force_kill`
(SIGKILL at once); xcecli could not send either. `jobs kill` now has
-s/--signal <name|number> (names in any case, SIG prefix optional, sent
upper case without it) and --force, in both the job-ID and the filter
request; using both at once, or an empty signal, is refused before any
request. stop and resume get neither (the server rejects a signal there).
Kill help and docs/xce.md "Killing and stopping jobs" document the flags.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:25:23 +00:00
0379823098
fix(xcecli): call the logout API so the token is revoked (bug-1959)
xcecli logout only removed the token from the local config, so the token
stayed valid on the server until it expired. It now first sends
POST /xce/v1/auth/logout with the bearer token (APIClient.Logout), which
revokes it server side since bug-1946. Best effort: if the server is
unreachable or rejects the token, the local login info is still cleared
and a warning says the revocation could not be confirmed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:25:23 +00:00
c71049f28d
docs(buglog): record Gitea sync for bug-1958..bug-1966
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:20:18 +00:00
7b77ce9c1d
docs(buglog): log follow-ups of the P0 fixes (bug-1958..bug-1966)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:16:33 +00:00
19be145d6f
docs(buglog): record Gitea sync for bug-1936..bug-1957
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:14:39 +00:00
d84a793578
docs: document graceful job kill/stop and job.terminate_interval (bug-1957)
- src/docs/xce.md: explicit signal and force kill are REST-only (signal,
  force_kill); xcecli has no signal option yet.
- src/docs/deployment.md: app.yaml row lists the job block
  (max_array_size, terminate_interval) and trusted proxies.
- xcecli jobs kill/stop/resume help describes the signal sequences.
- REST: control endpoint annotations and the signal/force_kill model fields
  describe the behaviour (swagger output not regenerated: swag fails on an
  unrelated annotation).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:07:42 +00:00
ff1de90ae3
docs(buglog): bug-1957 config completed (97f7c800)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:04:47 +00:00
97f7c800f1
fix(apiserver): validate, default and document job.terminate_interval (bug-1957)
The loader now stores the effective interval (10 s when unset or 0, LSF's
JOB_TERMINATE_INTERVAL default) and refuses a negative one; app_schema.json
describes the setting; src/docs/xce.md documents kill/stop behaviour and
the interval.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:04:47 +00:00
73fe8f787c
docs(buglog,specs): bug-1957 fixed (367af11a); time-window and host-failure briefs reuse the graceful kill
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:01:02 +00:00
367af11af1
fix(worker): graceful job kill and catchable stop, as LSF (bug-1957)
A plain kill sent one SIGTERM and never escalated, and a plain stop sent
one uncatchable SIGSTOP. Following LSF's TERMINATE and SUSPEND actions, a
plain kill now sends SIGINT, then SIGTERM, then SIGKILL, and a plain stop
sends SIGTSTP (the job can catch it, prepare and stop itself), then SIGSTOP
if it has not stopped; each next signal follows after the grace period and
only while the job is alive (or not yet stopped). A resume cancels a
pending stop; the job's exit ends any sequence. An explicit signal is
delivered once; a force kill is SIGKILL at once.

- JobSignal gains grace (capnp @2, xce_signal_t.grace): > 0 asks for the
  sequence, in seconds between signals.
- apiserver: job.terminate_interval (default 10 s, LSF's
  JOB_TERMINATE_INTERVAL); jobCtrlParams sends kill = SIGKILL and stop =
  SIGSTOP with that grace when no signal is given; the handler no longer
  pre-fills a default signal.
- worker: grace travels through the job-control task, the kept-signal table
  (pending_signal_*_ex) and the IPC message; the job process runs the
  sequence (job_signal_seq_*, uv timer) and checks stop state in /proc.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:00:36 +00:00
e5a0e42b66
docs(buglog): P0 defects fixed (bug-1936..bug-1955; 1936 and 1955 partial), bug-1956 cannot-reproduce
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 09:15:15 +00:00
d3da314041
fix(worker): keep signals for remote tasks that are still starting (bug-1955)
Only a head-host job start opened the pending-signal window (bug-1908), so a
kill for a remote task of a multi-host job that this host had accepted
(runtasks) but not yet registered was dropped with "Failed to find worker".
handle_runtasks() now opens the same window before queuing the task and
closes it if queuing fails; the signal is delivered when the task runs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 09:10:13 +00:00
148557a5d5
fix(worker): deliver the job-control signal the user asked for (bug-1954)
worker_signal_callback() sent SIGTERM for a KILL request without force and
SIGSTOP for any TSTP request. job_signal_to_deliver() (new job_signal.c) now
delivers the requested signal; only a forced terminate escalates to SIGKILL.
The apiserver's default signal for a plain stop becomes SIGSTOP (it was
TSTP and relied on the worker's substitution), so a plain stop still
suspends jobs that ignore SIGTSTP.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 09:07:38 +00:00
e00526f9f5
fix(apiserver): validate named and array submissions like any other (bug-1953)
preCheckJobRequest() returned from inside its job-name branch, so every
named or array job skipped the description, project and resreq checks,
resreq validation and the shell-limit conversion that follow. The name now
only sets the job count and array flag, and every submission runs the rest.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:59:34 +00:00
27ef506058
docs(apiserver): document security defaults changed by bug-1945..1952
deployment.md: auth.yaml now holds bcrypt hashes of the (still public)
dev credentials with startup warnings; JWT secret from XCE_JWT_SECRET or a
random per-process secret; logout revocation is per process;
security.trusted_proxies for reverse proxies.
security.md steering: JWT parsing, revocation, secret resolution,
trusted proxies, middleware abort rule, bcrypt and role-change rule.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:58:20 +00:00
a80768f31d
fix(apiserver): only rbac:control scope all may change user roles (bug-1952)
Cause: AddRolesForUser and DeleteRolesForUser let an rbac:control grant
narrower than all (e.g. scope self, an accepted domain type for rbac
permissions) change roles whenever the target user was the caller, so a
user holding only rbac:control/self could grant itself cluster_admin
(confirmed by TestAddRolesForUser_SelfScopeCannotGrantItselfRoles: HTTP
200 and cluster_admin assigned).

Fix: canChangeUserRoles requires an rbac:control grant with scope all
(checked over every grant, before deduplication prefers a narrower scope)
to grant or revoke roles for any user, the caller included; anything else
gets 403. Chosen over a "cannot grant permissions you do not hold" rule
because a scoped role change has no safe meaning here and the subset
check across domains would be complex and easy to get wrong.

Tests: TestAddRolesForUser_SelfScopeCannotGrantItselfRoles,
TestDeleteRolesForUser_SelfScopeRejected, TestUserRoleChanges_AllScopeAllowed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:58:20 +00:00
4ce4f0b798
fix(apiserver): revoke the token on logout (bug-1946)
Cause: Logout only deleted tokens/<user>/token, a file that token
validation (ValidateHandler / GetUsernameFromToken) never reads, so a
logged-out token stayed valid until its expiry (24 h by default).

Fix: an in-memory revocation set (controllers/token_revocation.go) keyed
by the token's jti (generateJWT now sets a random uuid jti, so two logons
in the same second give different tokens) or, for tokens without a jti,
by SHA-256 of the token; each entry is kept until the token's expiry and
expired entries are purged on every revocation. Logout revokes the token;
the shared ParseTokenClaims (used by ValidateHandler, RequirePermission
and handlers via GetUsernameFromToken) rejects revoked tokens. A missing
token file no longer fails logout. Scope: per process, not persisted and
not shared between apiservers.

Tests: TestLogout_RevokesToken (middleware), TestTokenRevocations_*,
TestIntegration_PostLogoutTokenRejected (replaces the design-gap test).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:58:20 +00:00
19a1a6f023
fix(apiserver): never sign or accept JWTs with the shipped default secret (bug-1945)
Cause: config/app.yaml shipped secretKey "xce-api-key" and
resolveJWTSecret used it whenever XCE_JWT_SECRET was unset, so anyone
could forge tokens for any user on such a deployment.

Fix: resolveJWTSecret uses XCE_JWT_SECRET, else the config value, but
treats an empty value or the old public default as unset; then it
generates a random 256-bit per-process secret and InitConf logs a warning
(secret never logged): tokens do not survive a restart and apiservers
sharing one endpoint must set the same XCE_JWT_SECRET. The shipped
app.yaml no longer carries a secret. The controllers test secret is no
longer the old default.

Tests: TestResolveJWTSecret_NeverUsesKnownDefaultOrEmpty,
TestShippedAppYAML_NoJWTSecret, TestInitConf_NoJWTSecretGeneratesAndWarns,
TestInitConf_JWTSecretFromEnvNoWarning.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:58:20 +00:00
c4c63f11e7
fix(apiserver): trust X-Forwarded-For only from configured proxies (bug-1948)
Cause: the router used gin.Default() without SetTrustedProxies, and gin
then trusts forwarding headers from every peer, so ClientIP() returned
whatever X-Forwarded-For a client sent: the IP whitelist and the IP bound
into JWTs could be spoofed.

Fix: new app.yaml option security.trusted_proxies (IP or CIDR list,
default empty = trust no proxy, validated at startup) passed to
SetTrustedProxies in Router(); an invalid list falls back to trusting
none. The nginx samples note that the proxy must be listed there.

Tests: TestRouter_IgnoresForwardedForByDefault,
TestRouter_HonoursForwardedForFromTrustedProxy, TestResolveTrustedProxies.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:58:20 +00:00
42a402a5b4
fix(apiserver): abort the request on every middleware failure (bug-1949)
Cause: ReturnError only writes the response and never aborts the gin
chain; RequirePermission (token error) and WhitelistIPMiddleware (invalid
client IP) returned after ReturnError without ctx.Abort(), so the route
handler still ran after the rejection.

Fix: ctx.Abort() after ReturnError at both sites (and, for clarity, on the
ValidateHandler body-check path, which validateRequestBody already
aborted). RequirePermission now answers a bad token with 401 instead of
500. ReturnError stays non-aborting (documented): handlers such as
ControlHost call it from helpers and then test ctx.IsAborted(), so a
global abort would change handler flows, and it adds nothing in handlers.

Tests: TestRequirePermission_AbortsOnInvalidToken,
TestWhitelistIPMiddleware_AbortsOnFailure.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:58:19 +00:00
42dd1e7a5e
fix(apiserver): pin JWT parsing to the HS256 signing algorithm (bug-1950)
Cause: the ValidateHandler parse callback (middleware isValidToken)
returned the secret without looking at token.Method, and
GetUsernameFromToken accepted any HMAC variant, so tokens signed with
HS384/HS512 were accepted although the server only signs HS256.

Fix: one JWTSigningMethod (HS256) used for signing, plus a shared
JWTKeyFunc (rejects any other alg) and JWTValidMethods parser option used
by both parse paths.

Tests: TestIsValidToken_PinsSigningAlgorithm (middleware),
TestGetUsernameFromToken_PinsSigningAlgorithm (controllers).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:58:19 +00:00
f91642dbc0
fix(apiserver): bcrypt passwords and default-credential warnings in config-file auth (bug-1947)
Cause: ConfigFileProvider.Authenticate compared the stored plaintext
password with != (timing side channel, passwords readable from the file),
and config/auth.yaml shipped admin/admin, user1/user1 etc. as plaintext
with nothing reporting that a default credential was active.

Fix: auth.yaml passwords may be bcrypt hashes (golang.org/x/crypto/bcrypt,
v0.23.0 as already pinned in go.mod, now vendored); plaintext entries are
still accepted but compared in constant time (SHA-256 + subtle), and an
unknown user costs the same bcrypt work. The provider reports startup
warnings (SecurityWarner, logged by main) for every plaintext entry and for
every shipped default credential still active. The shipped auth.yaml and
auth.yaml.example now store bcrypt hashes of the same dev passwords, so
tests/regression/run.sh (admin/admin) keeps working. tools/encrypt gains
-action=hash to generate entries.

Tests: TestConfigFileProvider_BcryptPassword,
TestConfigFileProvider_SecurityWarnings, TestShippedAuthConfig_HashedDefaults.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:58:19 +00:00
d3ce298232
fix(apiserver): reject auth.yaml without a config_file block (bug-1951)
Cause: NewConfigFileProvider read cfg.ConfigFile.Users without checking
that the optional config_file block was present, so provider: config_file
with no block panicked with a nil-pointer dereference at startup.

Fix: return a clear configuration error when the block is missing.
Test: TestNewConfigFileProvider_Errors/missing_config_file_block.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:58:19 +00:00
6445367c47
docs(buglog): log worker signal and launch-type issues (bug-1954..bug-1956)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:46:24 +00:00
8568c8ab80
fix(controller): send job control to every host of a multi-host job (bug-1942)
Cause: job_ctrl (job.c) sent kill/stop/resume only to allocs[0]->host.
The tasks on the other hosts are started through runtasks (by the head
worker for a parallel job, or by xrun as the MPI launcher's rsh agent)
and run under their own worker entry on each host; the head worker
signals only its local process group and forwards nothing, so they kept
running.

Fix: job_ctrl sends the request to every allocated host through
job_ctrl_to_host. The head decides the result as before; an unconnected
remote host is logged and skipped (its tasks belong to host failure
handling). Sends are non-blocking enqueues. Test:
testsuite/test_job_ctrl_hosts.c.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:43:53 +00:00
f244661b14
fix(controller): allow control of running jobs on connected hosts of any status (bug-1941)
Cause: job_ctrl (job.c) sent kill/stop/resume of a running job only when
the host's status was exactly 0, so any status bit (CLOSED,
CLOSED_BY_ADMIN, BUSY, LOCKED...) refused the request, and the refusal
returned -errno, i.e. whatever errno happened to hold (often 0 or
-ENOENT).

Fix: new job_ctrl_to_host sends the request whenever the worker is
connected (endpoint present, no XCE_WORKER_UNAVAIL); an unknown or
unconnected worker fails with -E_XCE_BAD_WORKER and an ERROR naming the
host. Nothing else changes for jobs on dead hosts. Test:
testsuite/test_job_ctrl_hosts.c.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:43:53 +00:00
4d2e80c4bb
fix(apiserver): send the requested signal, force kill and checkpoint options (bug-1940)
Cause: ControlJob (backend/clib.go) looked its content argument up as a
signal name, but the job control handlers pass a JSON JobControlEvent, so
the lookup always missed and every kill sent TERM (stop TSTP, resume
CONT); sig.force and ckpt.kill were hard-coded to 0 and the checkpoint
method was never set.

Fix: new pure jobCtrlParams (backend/convert_pure.go) decodes the event
(or a bare signal name/number), validates the signal against
JobSignalMapTable and returns an error instead of falling back, and maps
force_kill to xce_signal_t.force (kill) or xce_ckpt_options_t.kill
(checkpoint) plus the checkpoint method. ControlJob fills the C structs
from it. Tests: backend/jobctrl_params_test.go.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:43:53 +00:00
8f164b8205
docs(buglog): log bug-1953 (named jobs skip submit validation)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:42:58 +00:00
b9747ad2ff
fix(worker): parse job file transfers under the apiserver's "transfer_files" key (bug-1937)
Cause: the apiserver writes file transfers to exdata as "transfer_files"
(controllers/jobs.go processJobRequest), but parse_exdata() only matched
"transfer_file", so the transfers were logged as an unknown key and
dropped.

Fix: parse_exdata() accepts "transfer_files" and still the older
"transfer_file"; if both are present the first list wins instead of
leaking it. New worker unit test test_exdata_parse.c parses the exdata
JSON the apiserver produces (transfers, env, interactive, limits).
Stage-in/out itself is still not implemented in the worker.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:38:26 +00:00
d01ced9bd0
fix(apiserver): send exdata, exec cwd and exec home with submitted jobs (bug-1936)
Cause: modelJobToSubmit() filled only cmdline, execuser, launch_type and
pre/post-exec of job_startinfo_t, so the extended data the submit handler
builds (env, *_limit incl. -W run limit, file transfers, interactive mode),
the cwd and the exec home never reached the controller or the worker.

Fix: map the start info in a pure submitStartInfo() (exdata from
ExtendData, execcwd from ExecCWD or CWD, exechome from ExecHome) that
modelJobToSubmit copies into the C struct; decode exdata back in
cJobToModel/cJobArrayToModels so the store keeps ExtendData and CWD.
Reject extend_data the worker's parse_exdata() cannot take (not a JSON
object, over EXDATA_LENGTH-1 bytes) and drop an empty transfer list
instead of sending a null.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:38:26 +00:00
7a9d75d661
fix(scheduler): keep a moved job's own priority (bug-1944)
Both job-fields update paths, the core JOB_UPDATE event (core_events.c,
used by the live and sim drivers) and sched_update_existing_job()
(schedobj.c), set job->priority to the new queue's priority when a job
changed queue, while ingest gives every job 0. builtin_fcfs orders a
queue's jobs by job priority first, so a moved job overtook every job
already pending in the target queue (and snapshot restore, which sets 0,
disagreed with live for moved jobs).

Moving now only re-points job->queue; queue priority still orders the
queues themselves. test_job_move_priority.c moves a job into a priority-10
queue and checks it keeps priority 0 and does not take the only slot from
the queue's first job; test_job_update.c's in-place update case asserted
the old behaviour and now expects the job's priority to be kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:37:11 +00:00
8b89a0ce6b
fix(scheduler): load user/host groups in the simulator too (bug-1943)
Only live's sched_init_config() (scheduler.c) registered the config's
named user and host groups into core.ugmgr/hgmgr. The simulator takes its
config from the replay snapshot (snapshot_load.c) and later pushes through
the core CONFIG_PUSH event (core_events.c); neither loaded groups, so a
limit whose users name a group ("ug1") never matched in sim, while the
same limit matched in live.

New xce_core_load_groups() (core.c) is the one loader, called by
sched_init_config(), the CONFIG_PUSH event and snapshot restore, each
before limit consumers are built. snapshot.c's field notes now say the
groups are rebuilt from the snapshot's config.
test_sim_group_limit_parity.c checks a ug1-scoped limit matches the same
job on both drivers and that CONFIG_PUSH alone registers ug1 and hg1.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:37:11 +00:00
1668a43998
fix(scheduler): exclude admin-closed hosts from new placements (bug-1939)
No scheduler code read XCE_WORKER_CLOSED / XCE_WORKER_CLOSED_BY_ADMIN:
sched_host_iter_next() (host_iter.c), the candidate-host source for both
the seqjob and the parjob allocator, filtered only on the select
expression and resource availability, so jobs kept landing on a host an
admin had closed (parjob's skip_bad_worker_ hook is empty and unused).

sched_host_iter_next() now skips a host with either closed bit before
opening its trace scope, which covers every new placement for serial and
parallel jobs on the live and simulation drivers. Accounting of jobs
already running on the host is unchanged. No new pend-reason code: a
closed host is simply not a candidate. test_host_admin_close.c covers a
serial and a parallel job on both fixture shapes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:37:11 +00:00
f51a19d3d5
fix(scheduler): hold jobs of inactive queues at dispatch (bug-1938)
sched_queue_is_ready() in core.c was a `return 1` stub, so the picker
(builtin_fcfs and the legacy fcfs) took jobs from every queue and admin
queue inactivation never reached dispatch.

It now returns false for a queue with QUEUE_STAT_INACT_ADMIN or
QUEUE_STAT_INACT_WINDOW set, so an inactive queue's jobs stay pending.
QUEUE_STAT_CLOSED is deliberately not a gate (LSF semantics: closing only
blocks new submissions; already-pending jobs still dispatch). The status
reaches the core through the same queue add/update paths in both drivers,
so live and sim behave identically. test_queue_admin_control.c covers
open, closed and inactive queues on both fixture shapes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:37:11 +00:00
2fed7a6e5f
docs(buglog): log P0 defects from the 2026-10-10 review (bug-1936..bug-1952)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:15:28 +00:00
94472ee3e7
docs(review): project and spec review: conflicts, code gaps, priorities
27 cross-spec conflicts among the 30 pending specs (5 high), code gaps
against the five product goals and the no-shared-storage use case (silent
failures such as unenforced queue/host close and REST submit dropping job
limits, cwd, env and file transfers; inter-component and apiserver
security; file staging; monitoring and accounting; scheduling policy;
operations), and a tiered priority list (spec hygiene, P0 defects, P1
foundation specs, P2 features, P3 initiatives).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:11:08 +00:00
a68aac1a2b
docs(specs): record xce-node-manager interactions with service and provisioning specs
xce-node-manager: a "Cross-Spec Interactions" section (pending a
requirements revision; approved criteria unchanged) with six points: a
supervision/probe module shared with service-supervision, node-manager
components vs service jobs, a lease-based provisioner role, role
declaration and worker stop on decommission for provisioned hosts, a
host-loss grace period longer than worker restart plus rejoin, and a future
service proxy under the reverse-proxy role. design.md Open Questions points
to it.

The service-supervision, host-failure-job-recovery, elastic-hosts,
resource-provisioner and service-endpoint briefs cross-reference the
matching points; the roadmap lists the xce-node-manager revision.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 07:46:24 +00:00
922188b5d7
docs(specs): discovery for auto resource provisioning
New briefs: provisioning-demand (the scheduler knows a provisionable
catalog, tries host templates as virtual hosts and license headroom after
real resources fail, finalizes demand and idle supply at session end;
simulator support is a hard requirement), resource-provisioner (new
daemon, several instances owning providers via controller leases,
self-describing providers, automatic policy, durable request/instance
state; src/autoscaler is prior art only), elastic-hosts (host templates,
expected hosts, drain and decommission), elastic-shared-resources
(borrowed license tokens change the shared total).

Roadmap: the new specs, plus updates to resource-distribution-mapping
(shared pools) and peer-authentication-hardening (provisioner role).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 07:37:25 +00:00
664ba565cc
docs(specs): discovery for long-running service jobs
New briefs: host-failure-job-recovery (jobs on a lost host are requeued or
end EXIT after a grace period; stale copies fenced on rejoin; manual
requeue), service-supervision (profile kind service: restart policy and
liveness/readiness probes run by each replica's own worker, in-place
restart), service-replica-set (controller-held service object owning
independent replica jobs; event-driven replacement, scaling and rolling
update; no head agent, no promotion), service-endpoint (per-replica ports,
endpoints of ready replicas), job-resize (backlog, elastic jobs).

application-profile brief gains the service section; roadmap lists the new
specs, the existing-spec updates and the backlog item.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 07:15:46 +00:00
253d92340f
docs(buglog,specs): record Gitea sync for bug-1933..1935 (#358-#360 closed) and job-config-merge/project-config milestones (#71, #72)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 07:00:11 +00:00
c670204e25
docs(buglog): bug-1934 and bug-1935 fixed (4a38bb3c)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 06:56:57 +00:00
4a38bb3ca9
fix(apiserver): enforce self_ug job scopes and accept project filters (bug-1934, bug-1935)
bug-1934: every job-permission check read a single scope and compared it
only with "self", so a job:view or job:control grant scoped to self_ug acted
like "all" (list, get by ID, control), and self + self_ug collapsed to self.
userJobScope() now collects all of a user's grants of a job permission
before deduplication and adds them up (all; self; self_ug = the user and
everyone sharing a user group). The job list appends a submit_user filter
for narrower scopes, get-by-ID keeps covered rows in a new slice (removing
from the ranged slice skipped rows) and answers 403, both control paths
and job output check the owner against the scope. User groups now reach
the apiserver: the config push converts cluster_config_t.ugroups and
OnUserGroupsUpdated() replaces the new store (nested groups, all,
exclusions). The default basic_user now also sees its user-group mates'
jobs, as its configuration says.

bug-1935: project_name and license_project are accepted job filters, so
xcecli jobs list -P works.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 06:56:57 +00:00