Merge origin/scheduler-framework into merge-sf-20260903 (97 commits on ours: bug-1490/1494/1498/1509/1513/1521/1522 fixes, PEND-reason P1, scheduler-agent Phase 2, bug-924 owner-pid shim). Bookkeeping excluded per policy: .kiro/, .wolf/, CHANGES.md, reports/ (92 files dropped); .claude/ + CLAUDE.md keep ours. Key theirs-side changes absorbed: - bug-1938/1960: QUEUE_STAT_CLOSED no longer gates dispatch (closed queue refuses new submissions only; pending jobs still run) -- supersedes our QUEUE_CLOSED preason on the dispatch path; sched_queue_status_reason drops that branch - bug-1846: config push now stashes and applies between scheduling rounds (sched_stash_config/sched_apply_pending_config), applied at the same main-thread round boundary that preserves bug-1513 serialization - bug-1760: sched_check_pending_rejoin exposed + xcecli_is_connection_broken public predicate - bug-1936: execuser/exechome/execcwd/exdata carried on submission (superset of our bug-1500 execuser fallback; our job-typed resolveExecUser removed as dead code) - bug-1945~1962: apiserver auth hardening (HS256 pinning, default- secret rejection, logout revocation, bcrypt, RBAC) -- our API-key branch folded into controllers.ParseTokenClaims so both coexist - bug-1736: rawRunUsage job run-usage fields in the Go model - xce_test relocated src/controller/testsuite/test.c -> src/harness/test.c; controller suite split into controller_test.c + fixtures; their clitest fixtures spawn xce_test mid-suite under `meson test -j` Ours preserved across the merge: - PEND-reason P1 (codes 1-9, preason.h, job_preason machinery) -- gate re-ported into their relocated allocator (sched_election.c) after plugin_job_election, with attach on the allocator thread only (bug-1521 race discipline) - PEND-reason P0 main_reason semantics RECONCILED to theirs: the merge initially re-carried our first-detail-add-promotes-main_reason fallback in add_job_preason (4e2ac86), but their whole trace/verify stack treats "detail-only => main_reason 0" as a contract -- sched_trace_capture counts main_reason as its own occurrence ("Expected 2 Was 3" in test_trace_capture), the outcome renderer prints "main reason: none" for detail-only PENDs (test_repl_trace_e2e), and the committed .replay/.sim_verify baselines encode main_reason=0 (test_repl_trace_baseline byte compare, test_trace_equivalence off-vs-traced). The fallback is dropped from the shared lib; the user-visible "overall reason" survives as an API-layer view: rawJobToModel (convert_pure.go) promotes the first detail row to main_reason_name when main_reason is 0 (main_reason VALUE marshals unchanged). add_job_preason keeps only the NULL-reasons guard - bug-1498: sched_job_is_ready rejects non-PEND/suspended jobs - bug-1490/1781: xce_jobctrl_t.action as capnp union discriminant (their jobCtrlUnionDiscriminant, same semantics) - xcecli typed PendingReason field (53c00834), bug-1522 reconnect resend, xcecli_force_reconnect + their is_connection_broken both exported in client.h/libclient - t1/t2/t3 test-node hostconfig in src/configs/hosts.yaml (bug-1494), /xce/v1/events + /xce/v1/apikeys whitelist routes (agent Phase 2) - lib ASAN_OPTIONS=detect_leaks=0 test env + their oom-test block - drop-audit: our bug-1504 fix covered by their bug-1680 (no action) bug-924 follow-up (replaces the67f3f6adscratch-dir wipe): the take-theirs audit's one "drop" was re-evaluated and REVERTED instead of re-applied -- a global `find /tmp -name 'xce_*' -exec rm -rf` at suite start/exit races the LIVE /tmp/xce_* fixtures of sibling suites under `meson test -j` (harness_unit_test's clitest fixtures spawn xce_test as a child mid-suite; the wipe destroyed the parent's fixtures -> test_test_cli failures). Coverage instead: - mkdtemp_clean shim gains mkdtemp_clean_mark/sweep_since (testsuite_support/mkdtemp_clean.[ch]) - new case_scratch_sweep.[ch]: aggregate runners (controller_test.c, harness_test.c -- hundreds of cases per process) call case_scratch_sweep_begin/_end_ok from setUp/tearDown; a PASSED case's dirs are swept immediately in tearDown (after engine reaping), FAILED cases keep their dirs for post-mortem; XCE_MKDTEMP_KEEP_ALL=1 disables. One full controller_unit_test run was observed to leak 12G+ to ENOSPC without this, cascading into unrelated "Expected Non-NULL" failures - harness_unit_test_deps links the shim library as well Test reconciliations on adopt: - core-ready-gates: sched_queue_is_ready now gates only INACT_ADMIN|INACT_WINDOW (bug-1938/1960); CLOSED queues elect -- covered end-to-end by test_queue_admin_control.c - test_worker_resync: the 4 restart-replay RUN assertions predate the bug-1509 B3 marking (8858afed) landing on the same lineage; replayed RUN jobs now assert RUN|UNKWN|TRANSIENT (TWR_ASSERT_REPLAYED_RUN); terminal DONE/EXIT assertions unchanged - convert_pure.go: our job-typed resolveExecUser deleted (superseded by their bug-1780/1936 two-arg version) - test_preason: the 3 P0-promotion assertions reverted to the detail-keeps-main-0 contract; new tests cover limit-code detail rows and explicit-main-wins-over-later-details; apiserver convert_test.go gains the detail-only name-promotion cases Co-Authored-By: Claude Code <noreply@anthropic.com>
3.9 KiB
3.9 KiB
XCE Frontend (API Server + CLI)
Overview
Go-based REST API server (Gin) and CLI tool (Cobra) that expose cluster management operations. Two independent Go modules sharing no code at the module level.
Modules
| Module | Path | Entry Point |
|---|---|---|
apiserver |
src/frontend/apiserver/ |
cmd/main.go |
xcecli |
src/frontend/xcecli/ |
cmd/main.go |
schedulerclient (Phase 0-1, ✅ done) |
src/frontend/schedulerclient/ |
library, no entry point — read + write Go client for the scheduler agent's MCP Server, calls apiserver's REST API. go vet/go build/go test clean (45 tests). Read: types.go (URM types), client.go (interface), httpclient.go (resty impl), statusmap.go+_test.go (TDD, status mapping). Write (Phase 1): write.go+_test.go — JobSpec, SubmitJob/CancelJob/HoldJob/ReleaseJob, doPost (HTTP 200 + envelope-code dual check). |
mcp-server (Phase 0-1, ✅ done) |
src/frontend/mcp-server/ |
cmd/main.go — MCP Server exposing 10 tools (6 read-only + 4 write: scheduler_submit/cancel/hold/release); pure Go, CGO_ENABLED=0, uses schedulerclient as API client, authenticates via XCE_API_KEY env var. Uses github.com/mark3labs/mcp-go@v0.57.0 for MCP stdio transport. internal/templates.go — job template registry + general_serial (通用串行脚本); scheduler_submit has dry_run review mode (returns resolved JobSpec before submitting). Read-only tools annotated ReadOnlyHint=true; write tools DestructiveHint=true. go build/go vet/go test clean, binary ~12MB. |
Build & Test
/xce-build apiserver # → <build_dir>/frontend/apiserver/xce_apiserver, xcecli; runs unit tests (no live server required)
meson test -C <build_dir> --suite apiserver # CGO-backed backend tests
XCE=<build_dir>/frontend/xcecli/xcecli URL=<url> \
tests/regression/run.sh # integration (needs XCE and URL env vars)
API Server Architecture
cmd/ Entry point — init logger, config, stores, RBAC, auth, then start Gin
router/ Route registration and middleware chain
controllers/ HTTP handlers: one file per resource (jobs, queues, hosts, clusters, users...)
services/ Business logic
store/ Thread-safe in-memory key-value stores (generics-based)
models/ Request/response DTOs and domain structs
middleware/ CORS, rate limiting, IP whitelist, JWT validation
backend/ C library integration stubs
backend/auth Pluggable auth providers (configfile, ldap, oauth2, pam)
utils/ Logger (zap), config parsing, crypto helpers
common/ Global error codes and messages
config/ YAML configs and JSON schemas
Conventions
See .kiro/steering/api-standards.md (handler pattern, middleware chain, error codes, response format), .kiro/steering/error-handling.md (ReturnError early-return, zap logging), .kiro/steering/security.md (RBAC, JWT, auth backends), and .kiro/steering/testing.md (TDD, table-driven tests, state isolation).
Quick reference (from steering)
- Handler:
preCheck*() → service call → ReturnSuccess/ReturnError; return - Middleware chain: WhitelistURL → WhitelistIP → RateLimit → JWT → CORS → RBAC
- RBAC:
RequirePermission("action:scope")— permissions inconfig/rbac.yaml - Logging:
logger.Error("msg", zap.Error(err))— neverfmt.Println - Tests: co-located
_test.go,t.TempDir(), global state reset viat.Cleanup
Key Files
controllers/response.go— ReturnSuccess / ReturnError helperscontrollers/rbac_check.go— RBAC loading, RequirePermission middlewaremiddleware/validatehandler.go— JWT token validation middlewarebackend/auth/provider.go— AuthProvider interface definitionstore/store.go— Generic in-memory store implementationconfig/rbac.yaml— Role and permission definitionsconfig/auth.yaml— Auth provider configuration