Commit graph

20 commits

Author SHA1 Message Date
0318ac5f44
feat(job-array-one-based-index): parse submit index expressions (task 6.1)
ParseSubmitIndexExpr accepts only name[1-N] and name[1] (leading zeros
allowed) and otherwise reports the lowest-numbered broken rule with its
specific message: interactive arrays (2.1), invalid name or malformed
expression (2.2), lists/steps (2.3), %N (2.4), start 0 (2.5), reversed
(2.6), start above 1 (2.7), above the limit (2.8). Oversized numbers
saturate for the bound checks while reversal compares the digit
strings, so the rule order holds for any input. Not yet wired into
CreateJob (task 6.2).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 11:47:23 +00:00
3a7e3890e2
test(job-array-one-based-index): apiserver handler test seams and fixture (task 5.4)
CreateJob's ID generation and submission and both control dispatch
sites now go through package variables (generateJobID,
generateArrayJobID, submitJob, controlJob) that default to the real
backend/services functions, so production behaviour is unchanged. A new
non-cgo handler test file provides a reusable fixture: it saves and
restores every global it touches, sets up auth/RBAC, seeds jobs and
their queues, and records each dispatched job.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 11:41:45 +00:00
b970ab75b1
fix(apiserver): reject a malformed rusage[] at job submit (bug-1744)
Gitea #47: `xcecli jobs submit -R "rusage[mem=100000000a0]"` was accepted
with a normal "Job <...> is submitted" reply. The apiserver only
length-checked resreq; the real PEG grammar ran later, asynchronously, in
the scheduler's core_build_job(), which logged a server-side WARN and
dropped the job -- it then sat PEND forever with no sign of why.

- libxce: xce_check_rusage_syntax(resreq, &errmsg) runs parse_rusage() on
  every rusage[...] clause (extracted the way section.peg does: up to the
  first ']'), so primary, compound and alternative requests are all
  covered. No resource manager needed, so select[]/order[] resource names
  are still only checked by the scheduler (the documented scope).
- apiserver/backend: ValidateResreqSyntax() cgo wrapper (resreq_check.go);
  the !cgo stub returns nil (no job can be submitted without CGO anyway).
- controllers/jobs.go preCheckJobRequest(): calls it after the length
  check via a package var, so a malformed request is refused up front.

Tests: lib test_check_rusage_syntax (RED before: the reported input was
accepted) and backend TestValidateResreqSyntax (RED before) through the
real grammar; controllers TestPreCheckJobRequest_RejectsMalformedResreq
checks the call site with an injected validator (written with the call
site, so no separate RED). libxce, apiserver backend (cgo) and controllers
tests pass. apiserver/middleware TestLoadJobs fails here and on the
baseline alike: it needs a live apiserver on :8080.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-27 03:27:32 +00:00
099dbdabdc
fix(apiserver): fix job control union discriminant, add human-readable job ID support (gitea#45)
ControlJob() set xce_jobctrl_t.action to the job_ctrl_type_t action code
(kill/stop/resume/checkpoint), but that field is actually the capnp
JobControl_u_which union discriminant -- a different enum entirely. This
made kill/resume/checkpoint hit request.c's -EINVAL default case, and made
stop collide with the checkpoint discriminant and read through the wrong
union member. Live-verified against a real deployed cluster: kill/stop/
resume/checkpoint now behave correctly with no apiserver crash.

Also additively accept a human-readable job ID ("{cluster}-{apiserver}-
{timestamp}-{seq}") on the job control endpoint, alongside the existing
numeric form xcecli already translates to -- lets a caller hit the REST
API directly without reimplementing xcecli's ID packing. Live-verified via
a direct curl call with the human-readable ID.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hb6Edtj4KbS8CrgEWYdA2E
2026-09-07 08:31:54 +00:00
728496bea0
fix(apiserver): populate hosts-list slot usage counts
xcecli hosts list showed NUM_RUN/NUM_SSUSP/NUM_USUSP as always 0 --
the fields existed on HostInfo and were displayed, but nothing ever
assigned them. processHostData now derives per-host slot usage from
the live job store (exec_hosts + job_status), counting occupied slots
so the numbers are comparable to MAX_SLOT.

Partial fix for gitea#46 / bug-1743: the separate "dispatch stops at
8/16" question needs the reporter's live limits.yaml/pend-reason to
confirm a usergroup/queue slot limit, not addressed here.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y4M5MJM2zK6nU43da267Ku
2026-09-01 06:37:37 +00:00
343639fc7e
fix(apiserver): stop job-control panic on int/int64 type mismatch
xcecli jobs kill/stop/resume/checkpoint failed for every job with an
empty error body. models.JobInfo.JobStatus/.ArrayIdx are plain Go
`int`, but validateJobControl/foundJob asserted them as `int64` via a
bare type assertion, which always panicked; Gin's Recovery()
middleware swallowed the panic and returned an empty 500.

Fixes gitea#45 / bug-1742.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y4M5MJM2zK6nU43da267Ku
2026-09-01 06:37:04 +00:00
fe8818b19b
fix(apiserver): pass client-asserted FromHost through unchanged (bug-1676, gitea #44)
CreateJob now does FromHost: req.FromHost, a straight passthrough of
whatever xcecli (or any other client) sends in JobSubmitRequest.FromHost,
instead of deriving it server-side via ctx.ClientIP().

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AgYRU3zDTqNGV5Jk8aiKLg
2026-08-24 15:14:33 +00:00
88c7c05daf
fix(apiserver): jobs list -l/-o validates against the wrong struct (bug-1674, gitea #41)
GetJobsInfo/GetJobByIDs validated the -o/-l field list against
models.JobSubmitRequest{} -- the job-submission payload -- instead of
models.JobInfo{}, the query/response model (the already-correct
pattern used elsewhere in this file at line 1131). Response-only
fields like pending_reason don't exist on JobSubmitRequest, so any
such field made the whole listing fail outright with "unsupported
query field", not just omit that column.

core_limit/mem_limit exist on JobSubmitRequest but not on JobInfo --
switching wholesale would have newly broken those two previously-valid
fields. Added CoreLimit/MemLimit to JobAdditionalFields, the existing
merge-in bucket already used for pids/pgids/interactive_job_url.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AgYRU3zDTqNGV5Jk8aiKLg
2026-08-24 14:23:38 +00:00
Lei Chen
3a2094a461 Merge branch 'working' into job_peek_lchen
job_peek_lchen: Add fetchfile feature (job output/error peek proxy)
working: HA/Raft infrastructure (consensus, replication, config mgmt)

Conflict resolution summary:
- controller.capnp: fetchfile @41, fetchfilereply @18, fetchfilemsg @30
- cluster.c: bifurcated OPCode_reply for fetchfile + config-push ACK
- job.c: kept full fetchfile pending-request subsystem (~183 lines)
- request.c: kept both xce_fetchfile_req() and XCE_RETRY_DEFAULTS
- helper.c: added missing preVoteRequest/preVoteReply mappings
- All 15 new test registrations (test_cbuf.c + 14 HA tests)
2026-07-22 13:57:25 +08:00
bc921666de feat: reject job submissions to 'lost_and_found' reserved queue in preCheckJobRequest 2026-05-14 03:19:34 +00:00
febb4a6c4d fix: preserve admin-set host state across worker re-registration in makeWorker_
When a worker re-joins and already exists in workermgr, makeWorker_()
now overrides the existing slot instead of returning EEXIST. Admin-set
flags (LOCKEDU, CLOSED, HIDDEN, CLOSED_BY_ADMIN) survive the override.

Add XCE_WORKER_CLOSED_BY_ADMIN flag to distinguish admin-closed hosts
from workload-closed ones. worker_ctrl sets/clears it alongside CLOSED.

Refactor worker_init/worker_destroy into copy_worker/kill_worker helpers
shared by both fresh creation and override paths.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 10:38:39 +00:00
Lei Chen
d2b47e3985 Init commit of jobs output/error message peek project. 2026-05-09 16:57:54 +08:00
d7afd72e5b fix front end issues 2026-05-01 21:22:45 +08:00
86a65d640a test(apiserver): add integration tests for P0+P1 backend features
Covers ~30 integration tests across 8 sections from the
apiserver-backend-integration-test-plan:

- §1: Cluster config push (store population, merge semantics, zero-preserve)
- §2: Host control (OnHostUpdated merge, validateHost, preCheck, batch ops)
- §3: Resource value push (all 7 value types, nil/unknown discriminant, overwrite)
- §4: Heartbeat keep-alive (interval update, zero-preserve)
- §5: Cross-feature (config+heartbeat, config+host, worker+config field preservation)
- §6: RBAC (cluster_admin bypass, algorithm confusion, expired tokens, hostgroup bypass)
- §7: Store thread safety (concurrent config pushes, host control+config, resource values)
- §8: Input safety (hostname length, empty lists, malformed payloads)

All tests use CGO_ENABLED=0 with //go:build !cgo build tag.
§8.1 hostname-length skipped pending P1-2b implementation.
2026-04-18 11:56:31 +00:00
f354a5ec78 feat: P0-2c register routes + RBAC for host control/select
Routes and RBAC entries already existed from prior P0-2a/2b work. This
adds three TDD tests that lock down the RBAC invariants:

- TestRBAC_HostControl_ClusterAdminOnly: cluster_admin has host:control
  with domain=all; basic_user must NOT have host:control
- TestRBAC_HostView_BasicUserHasAccess: basic_user has host:view with
  domain=all; queue_admin must NOT have host:view
- TestRBAC_HostControl_DomainTypeIsAll: cluster_admin host:control
  domain type must be "all" (not hostgroup)

All apiserver tests passing.
2026-04-17 09:32:21 +00:00
5807a05cbb fix: address P0-2b review issues - build tag and actionType guard
1. Add //go:build !cgo to control_host_test.go so the stub-testing
   tests only run in non-CGO builds, matching the stub.go build
   constraint. Clean up dead nil check in SignatureCompiles test.

2. Replace two independent if-statements for actionType mapping in
   ControlHost with a map lookup + explicit guard, consistent with
   ControlQueue's ActionMap pattern. Prevents silent zero-value
   actionType if SupportedHostActions expands later.
2026-04-17 08:25:05 +00:00
8124cda83e feat: P0-2b implement ControlHost via xcecli_hostctrl + fix allFlag race
- backend/clib.go: Replace ControlHost stub with real implementation
  using xcecli_hostctrl, following the same pattern as ControlQueue
- controllers/hosts.go: Remove package-level allFlag variable;
  preCheckHostControlRequest now returns (bool, error) so the allFlag
  is a local value in the caller, eliminating the data race
- controllers/queues.go: Same fix for preCheckQueueControlRequest
- Add unit tests for both preCheck functions verifying return values,
  error cases, and concurrent safety
- Add compile-time verification test for ControlHost stub signature
2026-04-17 08:17:56 +00:00
960ef62342 fix job array display issue 2026-04-14 18:23:49 +08:00
d9b5c361c2 submit job from xcecli 2026-04-14 15:55:52 +08:00
c9dc8c4676 Merge Go frontend (apiserver + xcecli) into meson build 2026-04-14 11:23:18 +08:00