151 lines
8.4 KiB
Markdown
151 lines
8.4 KiB
Markdown
# OpenWolf
|
|
|
|
@.wolf/OPENWOLF.md
|
|
|
|
This project uses OpenWolf for context management. Read and follow .wolf/OPENWOLF.md every session. Check .wolf/cerebrum.md before generating code. Check .wolf/anatomy.md before reading files.
|
|
|
|
|
|
# XCE Cluster Management System
|
|
|
|
## Overview
|
|
XCE is an HPC cluster management system providing job scheduling, queue management, and cluster resource control. The Go-based API/CLI tier fronts C system daemons (controller, worker, scheduler).
|
|
|
|
## Tech Stack
|
|
- **Go 1.23.4**: REST API server (`src/frontend/apiserver`), CLI tool (`src/frontend/xcecli`)
|
|
- **C (gnu17)**: Core daemons — controller, worker, scheduler, shared lib
|
|
- **Build**: Meson + Nix flakes for C components; `make` for Go components
|
|
- **Go key deps**: Gin v1.10.0, JWT v4, Zap logging, Viper config, Cobra CLI
|
|
|
|
## Development Commands
|
|
```bash
|
|
# Build with /xce-build skill (debug + ASAN, clean build, runs tests)
|
|
/xce-build # Build all components + run all tests
|
|
/xce-build lib # Build libxce + run lib tests
|
|
/xce-build controller # Build lib + controller + run controller tests
|
|
/xce-build scheduler # Build lib + scheduler + run scheduler tests
|
|
/xce-build worker # Build lib + worker (no unit tests registered)
|
|
/xce-build apiserver # Build lib + apiserver + xcecli + run Go unit tests
|
|
/xce-build xcecli # Build xcecli only + run xcecli Go tests
|
|
|
|
```
|
|
|
|
## Test-Driven Development
|
|
Follow TDD strictly throughout the full development process:
|
|
|
|
1. **Red** — Write a failing test that specifies the desired behaviour before writing any implementation code.
|
|
2. **Green** — Write the minimal implementation to make the test pass. Do not over-engineer.
|
|
3. **Refactor** — Clean up the implementation and tests while keeping all tests green.
|
|
|
|
### Rules
|
|
- Never implement a feature without a corresponding test written first.
|
|
- Tests must be runnable without a live server or external services (use table-driven tests, stubs, and `t.TempDir()` for fixtures).
|
|
- A change is not complete until `make test` (and `make test-integration` where applicable) passes.
|
|
- For C components, add a test case in the relevant `testsuite/` before modifying the implementation.
|
|
- Tests live next to the code they cover (`_test.go` for Go, `testsuite/` for C).
|
|
- Keep test scope narrow — unit-test business logic; use integration tests for HTTP/API surface.
|
|
|
|
## Architecture
|
|
```
|
|
Request → Gin Router → Middleware (CORS/RateLimit/IPWhitelist/JWT) → RBAC Check → Controller → Services → In-Memory Store
|
|
```
|
|
- **3 independent Go modules**: `apiserver`, `xcecli`, `bcctool` (no go.work)
|
|
- **Stateful API server**: in-memory stores seeded at startup via `store.LoadBuiltinData()`
|
|
- **Pluggable auth**: configfile / LDAP / OAuth2 / PAM (set in `config/auth.yaml`)
|
|
- **RBAC**: domain-based role-action-domain model loaded from `config/rbac.yaml`
|
|
|
|
## Connection Model and Async I/O
|
|
|
|
### Long-Lived Persistent Connections (CRITICAL)
|
|
All XCE daemon connections are **long-lived** — they are established once and reused for the lifetime of the role relationship:
|
|
|
|
| Connection | Direction | Owner | Purpose |
|
|
|---|---|---|---|
|
|
| Controller primary → standby | Primary dials out during election | Primary (`ha->standby_endpoints`) | Config push, WAL replication, heartbeat |
|
|
| Controller standby ← primary | Inbound on standby listener | Standby (`primary_ctrl_fd`) | Receives ctrlhello, WAL, config, heartbeat |
|
|
| Primary → worker | Primary dials out | Primary (`c->workers[i].ep`) | Job dispatch, heartbeat |
|
|
| Scheduler → primary | Scheduler dials out | Scheduler | Schedule requests |
|
|
| API server → primary | API server dials out | API server backend | Job submit, cluster queries |
|
|
|
|
Connections are **never** closed and re-opened between RPCs. Closing a connection signals a failure event (RDHUP) which triggers failover or reconnect logic.
|
|
|
|
### Async Non-Blocking I/O Model (CRITICAL)
|
|
All C daemon I/O runs through **iomgr** (epoll-based) + **vtaskmgr** (thread pool):
|
|
|
|
- **iomgr** — single-threaded epoll loop; dispatches inbound packets as vtasks; manages endpoint lifetime
|
|
- **vtaskmgr** — thread-pool executing vtask handlers; handlers must return quickly (no blocking I/O)
|
|
- **delaymgr** — timer thread for periodic/delayed tasks; the only place blocking operations (connect, file I/O) are permitted
|
|
- `xce_request(ep, &req)` — enqueues a packet to the endpoint's send buffer; **always non-blocking**
|
|
- `xce_reply(ep, host, &reply)` — enqueues a reply; **always non-blocking**
|
|
|
|
**Never block in a vtask handler.** Blocking calls (connect, read, usleep) belong in delaymgr tasks or dedicated threads.
|
|
|
|
### Primary Heartbeat to Standbys
|
|
The primary sends `OPCode_nop` to each standby every `heartbeat_interval_ms` via a self-perpetuating delaymgr one-shot task (`ha_start_primary_heartbeat`). The task re-arms itself only while role is PRIMARY or PRIMARY_COMMITTING — on demotion the loop stops automatically (no cancel API needed). This keeps the standby-side watchdog (`ha_arm_heartbeat_timeout_vtask`) from timing out during idle periods.
|
|
|
|
## Key Files
|
|
- `src/frontend/Makefile` — primary build/test orchestration
|
|
- `src/frontend/apiserver/cmd/main.go` — server startup, init sequence
|
|
- `src/frontend/apiserver/router/routers.go` — all routes and middleware chain
|
|
- `src/frontend/apiserver/controllers/rbac_check.go` — RBAC loading and enforcement
|
|
- `src/frontend/apiserver/config/rbac.yaml` — role/permission definitions
|
|
- `src/frontend/apiserver/config/auth.yaml` — auth provider selection
|
|
|
|
## Subdirectory Guidelines
|
|
- `src/frontend/` — See `src/frontend/CLAUDE.md`
|
|
- `src/controller/` — See `src/controller/CLAUDE.md`
|
|
- `src/worker/` — See `src/worker/CLAUDE.md`
|
|
- `src/scheduler/` — See `src/scheduler/CLAUDE.md`
|
|
- `src/lib/` — See `src/lib/CLAUDE.md`
|
|
|
|
# Agentic SDLC and Spec-Driven Development
|
|
|
|
Kiro-style Spec-Driven Development on an agentic SDLC
|
|
|
|
## Project Context
|
|
|
|
### Paths
|
|
- Steering: `.kiro/steering/`
|
|
- Specs: `.kiro/specs/`
|
|
|
|
### Steering vs Specification
|
|
|
|
**Steering** (`.kiro/steering/`) - Guide AI with project-wide rules and context
|
|
**Specs** (`.kiro/specs/`) - Formalize development process for individual features
|
|
|
|
### Active Specifications
|
|
- Check `.kiro/specs/` for active specifications
|
|
- Use `/kiro-spec-status [feature-name]` to check progress
|
|
|
|
## Development Guidelines
|
|
- Think in English, generate responses in English. All Markdown content written to project files (e.g., requirements.md, design.md, tasks.md, research.md, validation reports) MUST be written in the target language configured for this specification (see spec.json.language).
|
|
|
|
## Minimal Workflow
|
|
- Phase 0 (optional): `/kiro-steering`, `/kiro-steering-custom`
|
|
- Phase 1 (Specification):
|
|
- `/kiro-spec-init "description"` — initialize spec scaffold
|
|
- `/kiro-spec-quick "description"` — fast single-pass spec (requirements + design + tasks, no approval gates)
|
|
- `/kiro-spec-batch` — generate specs for all features in a roadmap in parallel
|
|
- `/kiro-spec-requirements {feature}`
|
|
- `/kiro-validate-gap {feature}` (optional: for existing codebase)
|
|
- `/kiro-spec-design {feature} [-y]`
|
|
- `/kiro-validate-design {feature}` (optional: design review)
|
|
- `/kiro-spec-tasks {feature} [-y]`
|
|
- Phase 2 (Implementation): `/kiro-impl {feature} [tasks]`
|
|
- `/kiro-validate-impl {feature}` (optional: after implementation)
|
|
- `/kiro-verify-completion {feature}` — validate all acceptance criteria are met
|
|
- Debugging: `/kiro-debug {feature}` — systematic debugging with spec context
|
|
- Code review: `/kiro-review` — review implementation against spec requirements
|
|
- Progress check: `/kiro-spec-status {feature}` (use anytime)
|
|
|
|
## Development Rules
|
|
- 3-phase approval workflow: Requirements → Design → Tasks → Implementation
|
|
- Human review required each phase; use `-y` only for intentional fast-track
|
|
- Use `/kiro-spec-quick` for small, well-understood features; full workflow for anything touching HA/persistence/RBAC
|
|
- Keep steering current and verify alignment with `/kiro-spec-status`
|
|
- Follow the user's instructions precisely, and within that scope act autonomously: gather the necessary context and complete the requested work end-to-end in this run, asking questions only when essential information is missing or the instructions are critically ambiguous.
|
|
|
|
## Steering Configuration
|
|
- Load entire `.kiro/steering/` as project memory
|
|
- Core files: `product.md`, `tech.md`, `structure.md`
|
|
- XCE custom files: `testing.md`, `error-handling.md`, `security.md`
|
|
- Add new custom files via `/kiro-steering-custom`
|