Technology Stack

Architecture

xcecli (Go/Cobra) ──REST──► apiserver (Go/Gin) ──CGO──► C daemon layer
                                                          controller / scheduler / worker / autoscaler
                                                          ↑↑↑ all linked against libxce (src/lib/)

Request path inside apiserver: Gin Router → Middleware (CORS/RateLimit/IPWhitelist/JWT) → RBAC Check → Controller → Services → In-Memory Store

All controller state is in-memory, seeded at startup from YAML config. No persistent database in the API tier. C daemons communicate via TCP using a custom messaging layer: iomgr (epoll-based I/O loop), vtaskmgr (thread pool for packet dispatch), and xce_request/xce_reply for non-blocking send/receive. WAL + checkpoint provide persistence at the controller level.

Core Technologies

Go 1.23.4 — apiserver (REST API) and xcecli (CLI). Two independent Go modules, no go.work.
C (gnu17) — controller, worker, scheduler, autoscaler, libxce. No C++.
Gin v1.10.0 — HTTP router, middleware, binding
Cobra v1.9.1 — CLI framework for xcecli
Zap v1.27.0 — structured logging throughout (both Go modules)
Viper v1.19.0 / v1.20.1 — YAML config management
Cap'n Proto — wire serialization for WAL and daemon-to-daemon replication
iomgr / vtaskmgr / delaymgr — libxce async I/O framework: single-threaded epoll loop (iomgr), thread-pool packet dispatch (vtaskmgr), timer thread for periodic/blocking tasks (delaymgr)

Build Systems

System	Scope
Meson ≥ 1.5	C daemons + Go modules via `tools/go_build` wrapper
Make	Go development loop (build/test) from `src/frontend/`
Nix flakes	Hermetic dev environment, AppImage packaging

# Full build (C + Go, debug + ASAN)
/xce-build

# Go development loop (from src/frontend/)
make build-server          # → /tmp/apiserver
make build-cli             # → /tmp/xce_static (CGO_ENABLED=0)
make test                  # unit tests
make test-integration      # regression suite (needs XCE_BIN + SRV_URL)

# Nix package builds (from src/)
nix build .#xcecore        # C daemons + apiserver
nix build .#xcecore_debug  # debug + ASAN
nix build .#xcecli         # CLI only

Development Standards

Controller Pattern (Go)

// Struct: {Resource}Controller
type JobController struct { ... }

// Handler: {Action}{Resource}
func (c JobController) SubmitJob(ctx *gin.Context) {
    // Validate input
    req, err := preCheckSubmitJobRequest(ctx)
    if err != nil { ReturnError(ctx, code, err, nil); return }

    // Business logic
    result, err := c.svc.Submit(req)
    if err != nil { ReturnError(ctx, code, err, nil); return }

    ReturnSuccess(ctx, code, "submitted", result)
}

Early return on every error. Log with zap fields before returning.

Testing Requirements

TDD mandatory: Write failing test first, then implementation.
Go unit tests: Table-driven, no external dependencies, t.TempDir() for fixtures.
C tests: Add test case in testsuite/ before modifying implementation.
make test must pass without a live server.
make test-integration requires XCE_BIN + SRV_URL env vars.

Code Formatting

Go: gofmt / goimports (post-edit hooks configured)
C: clang-format using project config at xce/.clang-format

Auth and RBAC

Auth backend set in config/auth.yaml. Supports: configfile, LDAP, OAuth2, PAM.
Permissions as "action:scope" strings: "job:view:self", "queue:submit:all".
RBAC loaded from config/rbac.yaml, enforced by controllers.RequirePermission() middleware.

Key Technical Decisions

No Go workspace — apiserver and xcecli are independent modules; changes to shared types require updating both.
In-memory API state — no database in Go tier; state loaded from YAML at startup via store.LoadBuiltinData().
CGO bridge — backend/ package wraps C library calls; unit tests run CGO_ENABLED=0, backend tests need CGO_ENABLED=1 inside nix dev shell.
Swagger-first API docs — swaggo/swag annotations on handlers; docs served at runtime by Gin.
Structured logging — every log entry carries context via zap.String(), zap.Error(). Never use fmt.Println for operational output.
Single-writer HA — controller is always one active primary; standby election via Raft-style consensus (term-based voting, freshness gates) in the src/lib/raft/ shared module (raft.c/raft.h), with the single-writer constraint avoiding Raft log replication complexity.

4.6 KiB Raw Blame History