43 lines
3.3 KiB
Markdown
43 lines
3.3 KiB
Markdown
# Product Overview
|
||
|
||
XCE is an HPC (High-Performance Computing) cluster management system providing job scheduling, queue management, and cluster resource control. It targets organizations running batch workloads on on-premises or air-gapped compute clusters who need deterministic scheduling, multi-tenant access control, and zero external database dependencies.
|
||
|
||
## Core Capabilities
|
||
|
||
1. **Job lifecycle management** — submit, start, stop, restart, kill, and monitor jobs including job arrays. Resource usage data is collected and exposed via API.
|
||
2. **Queue and scheduling** — plugin-based scheduling engine for job-to-node placement with queue control (enable/disable) and expression-based resource filtering.
|
||
3. **Cluster and host management** — node status monitoring (online/offline/unavailable), cluster topology tracking, and host control operations.
|
||
4. **Multi-tenant RBAC + pluggable auth** — domain-based `action:scope` permission model (e.g., `job:view:self`). Auth backends: configfile, LDAP, OAuth2, PAM. Built-in IP whitelist and rate limiting.
|
||
5. **Controller persistence and HA** — WAL + checkpoint-based single-controller persistence with LSF-style priority election for standby failover and live replication over TCP (Cap'n Proto serialization).
|
||
|
||
## Target Use Cases
|
||
|
||
- **Local HPC batch computing** — long-running scientific/engineering workloads requiring deterministic scheduling, queue priority, and resource isolation.
|
||
- **Air-gapped / secure environments** — standalone deployments with file-based auth + SSL, no external cloud dependencies.
|
||
- **Multi-team shared clusters** — multiple teams sharing a cluster through RBAC with JWT token authentication.
|
||
- **No-shared-storage HPC** — controller uses local filesystem (ext4/xfs) for storage; job files transferred via API upload/download rather than network shares.
|
||
|
||
## Value Proposition
|
||
|
||
XCE provides a **fully integrated single-vendor stack** (no database setup, unified build system) with an open, buildable codebase (Nix flakes, AppImage packaging). Compared to Slurm/PBS, it eliminates the external database requirement. Compared to commercial LSF, it is open source and reproducibly buildable.
|
||
|
||
**Key architectural trade-off**: single-writer model throughout — no multi-master consensus. HA failover uses LSF-style priority election (simple, predictable) rather than Raft (10× complexity). All controller state is in-memory; persistence is WAL + checkpoint, not a traditional database.
|
||
|
||
## Active Development Context
|
||
|
||
The project is in a **persistence and HA transformation** (branch `persistent_ha`). Phases:
|
||
|
||
| Phase | Goal | Status |
|
||
|-------|------|--------|
|
||
| P0 | Restart survival (WAL + checkpoint) | Implementing |
|
||
| P1 | Standby election and takeover | Designed |
|
||
| P2 | Live replication primary → standby | Designed |
|
||
| P2.5 | Worker/scheduler auto-reconnect on failover | Designed |
|
||
| P3–P4 | Hardening, metrics, docs | Planned |
|
||
|
||
Key design decisions already locked:
|
||
- No `fork()` for snapshots (mutex inheritance deadlock in multithreaded controller)
|
||
- No Raft (single writer makes consensus overhead unjustified)
|
||
- No BoltDB (using dual zarray COW for application-level checkpointing)
|
||
- No application-layer encryption (trust OS boundary: chmod 0700, LUKS/dm-crypt)
|
||
- No WAL logging for job deletion (deferred to checkpoint time to simplify replay)
|