106 lines
5.4 KiB
Markdown
106 lines
5.4 KiB
Markdown
# Project Structure
|
|
|
|
## Organization Philosophy
|
|
|
|
**Layered monorepo, component-per-directory.** Each major subsystem lives under `src/<component>/` with its own Meson build file and CLAUDE.md. The Go frontend is isolated in `src/frontend/` as two independent Go modules. All C daemons share `src/lib/` (libxce) and public headers from `src/include/`.
|
|
|
|
```
|
|
src/
|
|
├── lib/ C — shared library (libxce): data structures, IPC, utilities
|
|
│ ├── raft/ C — reusable Raft consensus module (raft.c/raft.h)
|
|
│ └── testsuite/ C — libxce unit tests (test_*.c)
|
|
├── controller/ C — cluster controller daemon
|
|
├── worker/ C — per-node job execution daemon (xcecd)
|
|
├── scheduler/ C — job scheduling engine (plugin architecture)
|
|
├── autoscaler/ C — autoscaling engine (provider plugins)
|
|
├── frontend/ Go — apiserver (Gin REST) + xcecli (Cobra CLI)
|
|
├── xceadmin/ Python — admin CLI tool
|
|
├── include/ C — public shared headers consumed across all daemons
|
|
│ └── xce/ C — subset of public headers under the xce/ prefix
|
|
├── schema/ Cap'n Proto schema files (c.capnp, controller.capnp)
|
|
└── docs/ Project-wide documentation (markdown)
|
|
```
|
|
|
|
## C Daemon Layout Pattern
|
|
|
|
`worker/` and `scheduler/` follow:
|
|
|
|
```
|
|
src/<component>/
|
|
meson.build — build definition
|
|
CLAUDE.md — component-specific AI guide
|
|
include/ — component-internal headers
|
|
src/ — .c implementation files (flat, one domain per file)
|
|
testsuite/ — test programs (test_*.c)
|
|
```
|
|
|
|
**Exceptions**:
|
|
- `controller/` keeps its `.c` files flat in the component root (not in a nested `src/`), and does have its own `testsuite/`.
|
|
- `worker/` has no `testsuite/` — no `test_*.c` files exist anywhere in the component.
|
|
- `autoscaler/` does not follow this pattern at all: it has no CLAUDE.md, and its daemon code lives two levels deeper in `src/autoscaler/scaler/` (flat `.c` files plus its own `include/` and `testsuite/`), alongside `src/autoscaler/providers/`.
|
|
|
|
Plugin-based components add their own subdirectory:
|
|
- `scheduler/` — plugins in `src/plugins/`
|
|
- `autoscaler/` — providers in `src/autoscaler/providers/`
|
|
|
|
## Go Frontend Layout Pattern
|
|
|
|
**apiserver** (`src/frontend/apiserver/`):
|
|
|
|
| Package | Purpose |
|
|
|---------|---------|
|
|
| `cmd/` | Entry point, init sequence |
|
|
| `router/` | Route registration + middleware chain |
|
|
| `controllers/` | HTTP handlers — one file per resource |
|
|
| `services/` | Business logic |
|
|
| `store/` | Thread-safe in-memory KV stores (generics) |
|
|
| `models/` | Request/response DTOs, domain structs |
|
|
| `middleware/` | CORS, rate limiting, JWT, IP whitelist |
|
|
| `backend/` | CGO bridge to C library |
|
|
| `backend/auth/` | Pluggable auth providers |
|
|
| `utils/` | Zap logger wrapper, config parsing, crypto |
|
|
| `common/` | Global error codes and response helpers |
|
|
| `config/` | YAML configs + JSON schemas |
|
|
|
|
## Naming Conventions
|
|
|
|
### C
|
|
|
|
- **Files**: `snake_case.c` / `snake_case.h` — one domain per file
|
|
- **Public headers**: `src/include/<name>.h` (e.g. `persist.h`, `netio.h`, `resreq.h`) — the libxce API boundary; a small subset (`client_failover.h`, `cluster.h`, `cluster_def.h`, `errnox.h`, `types.h`) lives under `src/include/xce/` and is included as `xce/<name>.h`
|
|
- **Component headers**: `src/<component>/include/<name>.h`
|
|
- **Test files**: `test_<unit>.c` (e.g., `test_ha_cluster.c`, `test_persist_wal.c`)
|
|
- **Functions**: `snake_case` with component prefix — `ha_*`, `cluster_*`, `xcecd_*`, `xce_*`
|
|
- **Types**: `snake_case_t` suffix (e.g., `cluster_t`, `ha_state_t`)
|
|
- **Struct tags**: `struct <name>_s` (e.g., `struct cluster_s`)
|
|
- **Enums/Macros**: `UPPER_SNAKE_CASE` with component prefix (`CTRL_PRIMARY`, `HA_DEFAULT_*`)
|
|
|
|
### Go
|
|
|
|
- **Files**: one file per resource — `jobs.go`, `queues.go`, `hosts.go`
|
|
- **Tests**: `*_test.go` co-located; integration tests as `integration_test.go`, `<resource>_control_test.go`
|
|
- **Controller structs**: `{Resource}Controller` (e.g., `JobController`, `ClusterController`, `QueueController`); one exception — `hosts.go` uses `HostHandler`
|
|
- **Handler methods**: `{Action}{Resource}` (e.g., `CreateJob`, `GetJobsInfo`, `ControlJobByID`)
|
|
- **Permission strings**: `"action:scope"` (e.g., `"job:view:self"`)
|
|
- **JSON tags**: `snake_case`
|
|
|
|
## Test Organization
|
|
|
|
**C tests** (`testsuite/` inside each component):
|
|
- Standalone `.c` programs, one per unit: `test_<unit>.c`
|
|
- `test.c` provides the test runner harness
|
|
- Tiered naming: `test_p1_integration.c`, `test_e2e_wal.c`, `test_p3_hardening.c`
|
|
- Test-only symbols guarded by `#ifdef XCE_TESTING`
|
|
|
|
**Go tests** (co-located `_test.go`):
|
|
- Table-driven unit tests, `t.TempDir()` for fixtures
|
|
- Global state reset helpers (e.g., `resetRBACState()`) for test isolation
|
|
- `make test` = unit only; `make test-all` = unit + integration
|
|
|
|
## Code Organization Principles
|
|
|
|
- **One domain per file** in C — `ha.c` owns HA logic, `job.c` owns job logic. No megafiles.
|
|
- **Single Go module per binary** — apiserver and xcecli share no Go code at the module level.
|
|
- **Controllers delegate to services** — HTTP handlers validate input and delegate; business logic lives in `services/`.
|
|
- **Dependencies flow inward**: `controllers → services → store/backend`, never the reverse.
|
|
- **Cap'n Proto for wire formats** — any new inter-daemon or persistence protocol uses `.capnp` schemas in `src/schema/`.
|