forked from lei.chen/myxce
Rongsong Shen
1599d06bc4
- create_lost_and_found(): provide hostattrs_t so makeWorker succeeds - collect_queues/collect_workers (dataex.c): skip lost_and_found sentinel so it is never pushed to apiserver or scheduler - mark_worker_unavail(): reset hosttype/hostmodel to "unknown" and zero all numeric attrs, then update resvals and slots — prevents stale capacity from being visible while the host is down - restore_worker(): apply the same attr reset on WAL recovery so restored-but-unavailable workers advertise no capacity until reconnect; also remove dead calloc() that was immediately overwritten by makeWorker - Extract reset_hostattrs_unknown() helper to eliminate copy-paste - Define XCE_LOST_AND_FOUND_NAME constant in config_repl.h; replace all six raw "lost_and_found" string literals across config_repl.c and dataex.c
163 lines
8 KiB
Markdown
163 lines
8 KiB
Markdown
# XCE Cluster Management System
|
|
|
|
## Overview
|
|
XCE is an HPC cluster management system providing job scheduling, queue management, and cluster resource control. The Go-based API/CLI tier fronts C/C++ system daemons (controller, worker, scheduler).
|
|
|
|
## Tech Stack
|
|
- **Go 1.23.4**: REST API server (`src/frontend/apiserver`), CLI tool (`src/frontend/xcecli`)
|
|
- **C/C++**: Core daemons — controller, worker, scheduler, shared lib
|
|
- **Build**: Meson + Nix flakes for C components; `make` for Go components
|
|
- **Go key deps**: Gin v1.10.0, JWT v4, Zap logging, Viper config, Cobra CLI
|
|
|
|
## Development Commands
|
|
```bash
|
|
# Full dev build — all components (debug + ASAN)
|
|
/xce-build # Preferred: builds everything via nix develop + meson
|
|
|
|
# From src/frontend/
|
|
make build-server # Build apiserver binary → /tmp/apiserver
|
|
make build-cli # Build xcecli binary → /tmp/xce_static (CGO_ENABLED=0)
|
|
make test # Unit tests (no live server required)
|
|
make test-integration # Regression suite (requires XCE_BIN and SRV_URL env vars)
|
|
make test-all # Unit + integration tests
|
|
|
|
# Nix package builds (from src/)
|
|
nix build --option sandbox false .#xcecore # C daemons + frontend/apiserver
|
|
nix build --option sandbox false .#xcecore_debug # Same with debug/ASAN
|
|
nix build --option sandbox false .#xcecli # CLI tool only
|
|
```
|
|
|
|
## Test-Driven Development
|
|
Follow TDD strictly throughout the full development process:
|
|
|
|
1. **Red** — Write a failing test that specifies the desired behaviour before writing any implementation code.
|
|
2. **Green** — Write the minimal implementation to make the test pass. Do not over-engineer.
|
|
3. **Refactor** — Clean up the implementation and tests while keeping all tests green.
|
|
|
|
### Rules
|
|
- Never implement a feature without a corresponding test written first.
|
|
- Tests must be runnable without a live server or external services (use table-driven tests, stubs, and `t.TempDir()` for fixtures).
|
|
- A change is not complete until `make test` (and `make test-integration` where applicable) passes.
|
|
- For C components, add a test case in the relevant `testsuite/` before modifying the implementation.
|
|
- Tests live next to the code they cover (`_test.go` for Go, `testsuite/` for C).
|
|
- Keep test scope narrow — unit-test business logic; use integration tests for HTTP/API surface.
|
|
|
|
## Architecture
|
|
```
|
|
Request → Gin Router → Middleware (CORS/RateLimit/IPWhitelist/JWT) → RBAC Check → Controller → Services → In-Memory Store
|
|
```
|
|
- **3 independent Go modules**: `apiserver`, `xcecli`, `bcctool` (no go.work)
|
|
- **Stateful API server**: in-memory stores seeded at startup via `store.LoadBuiltinData()`
|
|
- **Pluggable auth**: configfile / LDAP / OAuth2 / PAM (set in `config/auth.yaml`)
|
|
- **RBAC**: domain-based role-action-domain model loaded from `config/rbac.yaml`
|
|
|
|
## Key Files
|
|
- `src/frontend/Makefile` — primary build/test orchestration
|
|
- `src/frontend/apiserver/cmd/main.go` — server startup, init sequence
|
|
- `src/frontend/apiserver/router/routers.go` — all routes and middleware chain
|
|
- `src/frontend/apiserver/controllers/rbac_check.go` — RBAC loading and enforcement
|
|
- `src/frontend/apiserver/config/rbac.yaml` — role/permission definitions
|
|
- `src/frontend/apiserver/config/auth.yaml` — auth provider selection
|
|
|
|
## Subdirectory Guidelines
|
|
- `src/frontend/` — See `src/frontend/CLAUDE.md`
|
|
- `src/controller/` — See `src/controller/CLAUDE.md`
|
|
- `src/worker/` — See `src/worker/CLAUDE.md`
|
|
- `src/scheduler/` — See `src/scheduler/CLAUDE.md`
|
|
- `src/lib/` — See `src/lib/CLAUDE.md`
|
|
|
|
<!-- code-review-graph MCP tools -->
|
|
## MCP Tools: code-review-graph
|
|
|
|
**IMPORTANT: This project has a knowledge graph. ALWAYS use the
|
|
code-review-graph MCP tools BEFORE using Grep/Glob/Read to explore
|
|
the codebase.** The graph is faster, cheaper (fewer tokens), and gives
|
|
you structural context (callers, dependents, test coverage) that file
|
|
scanning cannot.
|
|
|
|
### When to use graph tools FIRST
|
|
|
|
- **Exploring code**: `semantic_search_nodes` or `query_graph` instead of Grep
|
|
- **Understanding impact**: `get_impact_radius` instead of manually tracing imports
|
|
- **Code review**: `detect_changes` + `get_review_context` instead of reading entire files
|
|
- **Finding relationships**: `query_graph` with callers_of/callees_of/imports_of/tests_for
|
|
- **Architecture questions**: `get_architecture_overview` + `list_communities`
|
|
|
|
Fall back to Grep/Glob/Read **only** when the graph doesn't cover what you need.
|
|
|
|
### Key Tools
|
|
|
|
| Tool | Use when |
|
|
|------|----------|
|
|
| `detect_changes` | Reviewing code changes — gives risk-scored analysis |
|
|
| `get_review_context` | Need source snippets for review — token-efficient |
|
|
| `get_impact_radius` | Understanding blast radius of a change |
|
|
| `get_affected_flows` | Finding which execution paths are impacted |
|
|
| `query_graph` | Tracing callers, callees, imports, tests, dependencies |
|
|
| `semantic_search_nodes` | Finding functions/classes by name or keyword |
|
|
| `get_architecture_overview` | Understanding high-level codebase structure |
|
|
| `refactor_tool` | Planning renames, finding dead code |
|
|
|
|
### Workflow
|
|
|
|
1. The graph auto-updates on file changes (via hooks).
|
|
2. Use `detect_changes` for code review.
|
|
3. Use `get_affected_flows` to understand impact.
|
|
4. Use `query_graph` pattern="tests_for" to check coverage.
|
|
|
|
## graphify
|
|
|
|
This project has a graphify knowledge graph at graphify-out/.
|
|
|
|
Rules:
|
|
- Before answering architecture or codebase questions, read graphify-out/GRAPH_REPORT.md for god nodes and community structure
|
|
- If graphify-out/wiki/index.md exists, navigate it instead of reading raw files
|
|
- For cross-module "how does X relate to Y" questions, prefer `graphify query "<question>"`, `graphify path "<A>" "<B>"`, or `graphify explain "<concept>"` over grep — these traverse the graph's EXTRACTED + INFERRED edges instead of scanning files
|
|
- After modifying code files in this session, run `graphify update .` to keep the graph current (AST-only, no API cost)
|
|
|
|
|
|
# Agentic SDLC and Spec-Driven Development
|
|
|
|
Kiro-style Spec-Driven Development on an agentic SDLC
|
|
|
|
## Project Context
|
|
|
|
### Paths
|
|
- Steering: `.kiro/steering/`
|
|
- Specs: `.kiro/specs/`
|
|
|
|
### Steering vs Specification
|
|
|
|
**Steering** (`.kiro/steering/`) - Guide AI with project-wide rules and context
|
|
**Specs** (`.kiro/specs/`) - Formalize development process for individual features
|
|
|
|
### Active Specifications
|
|
- Check `.kiro/specs/` for active specifications
|
|
- Use `/kiro:spec-status [feature-name]` to check progress
|
|
|
|
## Development Guidelines
|
|
- Think in English, generate responses in English. All Markdown content written to project files (e.g., requirements.md, design.md, tasks.md, research.md, validation reports) MUST be written in the target language configured for this specification (see spec.json.language).
|
|
|
|
## Minimal Workflow
|
|
- Phase 0 (optional): `/kiro:steering`, `/kiro:steering-custom`
|
|
- Phase 1 (Specification):
|
|
- `/kiro:spec-init "description"`
|
|
- `/kiro:spec-requirements {feature}`
|
|
- `/kiro:validate-gap {feature}` (optional: for existing codebase)
|
|
- `/kiro:spec-design {feature} [-y]`
|
|
- `/kiro:validate-design {feature}` (optional: design review)
|
|
- `/kiro:spec-tasks {feature} [-y]`
|
|
- Phase 2 (Implementation): `/kiro:spec-impl {feature} [tasks]`
|
|
- `/kiro:validate-impl {feature}` (optional: after implementation)
|
|
- Progress check: `/kiro:spec-status {feature}` (use anytime)
|
|
|
|
## Development Rules
|
|
- 3-phase approval workflow: Requirements → Design → Tasks → Implementation
|
|
- Human review required each phase; use `-y` only for intentional fast-track
|
|
- Keep steering current and verify alignment with `/kiro:spec-status`
|
|
- Follow the user's instructions precisely, and within that scope act autonomously: gather the necessary context and complete the requested work end-to-end in this run, asking questions only when essential information is missing or the instructions are critically ambiguous.
|
|
|
|
## Steering Configuration
|
|
- Load entire `.kiro/steering/` as project memory
|
|
- Default files: `product.md`, `tech.md`, `structure.md`
|
|
- Custom files are supported (managed via `/kiro:steering-custom`)
|