lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
myxce/CLAUDE.md
Rongsong Shen 7b39650229 refactor(ha): passive standby model — primary connects outbound to all peers 
Remove fetch_config_from_primary, ha_find_and_connect_to_primary, and the
tier-1/2/3 discovery path. Standbys never initiate outbound connections to
the primary; the primary owns all inter-controller channels.

Key changes:
- ha.c: add primary heartbeat loop (ha_start_primary_heartbeat) that sends
  OPCode_nop to connected standbys every heartbeat_interval_ms; loop
  self-cancels on demotion via role check before re-arm.
- ha.c: add outbound reconnect in primary_heartbeat_fn — for peers absent
  from standby_endpoints, open fresh outbound TLS connection (on_standby_connected
  sends ctrlhello; standby_ep_ctx_t mirrors vote_ep_ctx_t layout).
- ha.c: remove ha_find_and_connect_to_primary + helpers (ha_raw_tcp_connect,
  ha_tcp_connect_hook); remove election_timeout_fn (superseded by
  raft_election_timeout_fn); remove fetch_config_from_primary call from
  ha_elect_evaluate_and_signal STANDBY branch.
- cluster.c: OPCode_requestVote denial no longer reuses inbound connection;
  late joiners are bootstrapped via primary heartbeat outbound connect.
- config_repl.c: remove connect_ssl_raw, do_one_config_fetch,
  fetch_config_from_primary (~250 LOC).
- myapp.c: remove my_tag > 0 fetch branch; all standbys wait passively.
- ha.h: remove ha_find_and_connect_to_primary declaration + macros +
  ha_tcp_connect_hook seam; add ha_start_primary_heartbeat declaration.
- meson.build: remove test_ha_connect.c (tests deleted function).
- controller_test.c: remove test_ha_connect registration.
- CLAUDE.md, src/controller/CLAUDE.md: document passive standby model,
  inbound-vs-outbound connection ownership, async I/O constraint,
  heartbeat model.
- request.h: add xce_request declaration (removes scattered local externs).
2026-06-01 04:17:05 +00:00

212 lines
11 KiB
Markdown
Raw Blame History

# OpenWolf
@.wolf/OPENWOLF.md
This project uses OpenWolf for context management. Read and follow .wolf/OPENWOLF.md every session. Check .wolf/cerebrum.md before generating code. Check .wolf/anatomy.md before reading files.
# XCE Cluster Management System
## Overview
XCE is an HPC cluster management system providing job scheduling, queue management, and cluster resource control. The Go-based API/CLI tier fronts C system daemons (controller, worker, scheduler).
## Tech Stack
- **Go 1.23.4**: REST API server (`src/frontend/apiserver`), CLI tool (`src/frontend/xcecli`)
- **C (gnu17)**: Core daemons — controller, worker, scheduler, shared lib
- **Build**: Meson + Nix flakes for C components; `make` for Go components
- **Go key deps**: Gin v1.10.0, JWT v4, Zap logging, Viper config, Cobra CLI
## Development Commands
```bash
# Build with /xce-build skill (debug + ASAN, clean build, runs tests)
/xce-build # Build all components + run all tests
/xce-build lib # Build libxce + run lib tests
/xce-build controller # Build lib + controller + run controller tests
/xce-build scheduler # Build lib + scheduler + run scheduler tests
/xce-build worker # Build lib + worker (no unit tests registered)
/xce-build apiserver # Build lib + apiserver + xcecli + run Go unit tests
/xce-build xcecli # Build xcecli only + run xcecli Go tests
# From src/frontend/ (manual Go dev loop)
make build-server # Build apiserver binary → /tmp/apiserver
make build-cli # Build xcecli binary → /tmp/xce_static (CGO_ENABLED=0)
make test # Unit tests (no live server required)
make test-integration # Regression suite (requires XCE_BIN and SRV_URL env vars)
make test-all # Unit + integration tests
# Nix package builds (from src/)
nix build --option sandbox false .#xcecore # C daemons + frontend/apiserver
nix build --option sandbox false .#xcecore_debug # Same with debug/ASAN
nix build --option sandbox false .#xcecli # CLI tool only
```
## Test-Driven Development
Follow TDD strictly throughout the full development process:
1. **Red** — Write a failing test that specifies the desired behaviour before writing any implementation code.
2. **Green** — Write the minimal implementation to make the test pass. Do not over-engineer.
3. **Refactor** — Clean up the implementation and tests while keeping all tests green.
### Rules
- Never implement a feature without a corresponding test written first.
- Tests must be runnable without a live server or external services (use table-driven tests, stubs, and `t.TempDir()` for fixtures).
- A change is not complete until `make test` (and `make test-integration` where applicable) passes.
- For C components, add a test case in the relevant `testsuite/` before modifying the implementation.
- Tests live next to the code they cover (`_test.go` for Go, `testsuite/` for C).
- Keep test scope narrow — unit-test business logic; use integration tests for HTTP/API surface.
## Architecture
```
Request → Gin Router → Middleware (CORS/RateLimit/IPWhitelist/JWT) → RBAC Check → Controller → Services → In-Memory Store
```
- **3 independent Go modules**: `apiserver`, `xcecli`, `bcctool` (no go.work)
- **Stateful API server**: in-memory stores seeded at startup via `store.LoadBuiltinData()`
- **Pluggable auth**: configfile / LDAP / OAuth2 / PAM (set in `config/auth.yaml`)
- **RBAC**: domain-based role-action-domain model loaded from `config/rbac.yaml`
## Connection Model and Async I/O
### Long-Lived Persistent Connections (CRITICAL)
All XCE daemon connections are **long-lived** — they are established once and reused for the lifetime of the role relationship:
| Connection | Direction | Owner | Purpose |
|---|---|---|---|
| Controller primary → standby | Primary dials out during election | Primary (`ha->standby_endpoints`) | Config push, WAL replication, heartbeat |
| Controller standby ← primary | Inbound on standby listener | Standby (`primary_ctrl_fd`) | Receives ctrlhello, WAL, config, heartbeat |
| Primary → worker | Primary dials out | Primary (`c->workers[i].ep`) | Job dispatch, heartbeat |
| Scheduler → primary | Scheduler dials out | Scheduler | Schedule requests |
| API server → primary | API server dials out | API server backend | Job submit, cluster queries |
Connections are **never** closed and re-opened between RPCs. Closing a connection signals a failure event (RDHUP) which triggers failover or reconnect logic.
### Async Non-Blocking I/O Model (CRITICAL)
All C daemon I/O runs through **iomgr** (epoll-based) + **vtaskmgr** (thread pool):
- **iomgr** — single-threaded epoll loop; dispatches inbound packets as vtasks; manages endpoint lifetime
- **vtaskmgr** — thread-pool executing vtask handlers; handlers must return quickly (no blocking I/O)
- **delaymgr** — timer thread for periodic/delayed tasks; the only place blocking operations (connect, file I/O) are permitted
- `xce_request(ep, &req)` — enqueues a packet to the endpoint's send buffer; **always non-blocking**
- `xce_reply(ep, host, &reply)` — enqueues a reply; **always non-blocking**
**Never block in a vtask handler.** Blocking calls (connect, read, usleep) belong in delaymgr tasks or dedicated threads.
### Primary Heartbeat to Standbys
The primary sends `OPCode_nop` to each standby every `heartbeat_interval_ms` via a self-perpetuating delaymgr one-shot task (`ha_start_primary_heartbeat`). The task re-arms itself only while role is PRIMARY or PRIMARY_COMMITTING — on demotion the loop stops automatically (no cancel API needed). This keeps the standby-side watchdog (`ha_arm_heartbeat_timeout_vtask`) from timing out during idle periods.
## Key Files
- `src/frontend/Makefile` — primary build/test orchestration
- `src/frontend/apiserver/cmd/main.go` — server startup, init sequence
- `src/frontend/apiserver/router/routers.go` — all routes and middleware chain
- `src/frontend/apiserver/controllers/rbac_check.go` — RBAC loading and enforcement
- `src/frontend/apiserver/config/rbac.yaml` — role/permission definitions
- `src/frontend/apiserver/config/auth.yaml` — auth provider selection
## Subdirectory Guidelines
- `src/frontend/` — See `src/frontend/CLAUDE.md`
- `src/controller/` — See `src/controller/CLAUDE.md`
- `src/worker/` — See `src/worker/CLAUDE.md`
- `src/scheduler/` — See `src/scheduler/CLAUDE.md`
- `src/lib/` — See `src/lib/CLAUDE.md`
<!-- code-review-graph MCP tools -->
## MCP Tools: code-review-graph
**IMPORTANT: This project has a knowledge graph. ALWAYS use the
code-review-graph MCP tools BEFORE using Grep/Glob/Read to explore
the codebase.** The graph is faster, cheaper (fewer tokens), and gives
you structural context (callers, dependents, test coverage) that file
scanning cannot.
### When to use graph tools FIRST
- **Exploring code**: `semantic_search_nodes` or `query_graph` instead of Grep
- **Understanding impact**: `get_impact_radius` instead of manually tracing imports
- **Code review**: `detect_changes` + `get_review_context` instead of reading entire files
- **Finding relationships**: `query_graph` with callers_of/callees_of/imports_of/tests_for
- **Architecture questions**: `get_architecture_overview` + `list_communities`
Fall back to Grep/Glob/Read **only** when the graph doesn't cover what you need.
### Key Tools
| Tool | Use when |
|------|----------|
| `detect_changes` | Reviewing code changes — gives risk-scored analysis |
| `get_review_context` | Need source snippets for review — token-efficient |
| `get_impact_radius` | Understanding blast radius of a change |
| `get_affected_flows` | Finding which execution paths are impacted |
| `query_graph` | Tracing callers, callees, imports, tests, dependencies |
| `semantic_search_nodes` | Finding functions/classes by name or keyword |
| `get_architecture_overview` | Understanding high-level codebase structure |
| `refactor_tool` | Planning renames, finding dead code |
### Workflow
1. The graph auto-updates on file changes (via hooks).
2. Use `detect_changes` for code review.
3. Use `get_affected_flows` to understand impact.
4. Use `query_graph` pattern="tests_for" to check coverage.
## graphify
This project has a graphify knowledge graph at graphify-out/.
Rules:
- Before answering architecture or codebase questions, read graphify-out/GRAPH_REPORT.md for god nodes and community structure
- If graphify-out/wiki/index.md exists, navigate it instead of reading raw files
- For cross-module "how does X relate to Y" questions, prefer `graphify query "<question>"`, `graphify path "<A>" "<B>"`, or `graphify explain "<concept>"` over grep — these traverse the graph's EXTRACTED + INFERRED edges instead of scanning files
- After modifying code files in this session, run `graphify update .` to keep the graph current (AST-only, no API cost)
# Agentic SDLC and Spec-Driven Development
Kiro-style Spec-Driven Development on an agentic SDLC
## Project Context
### Paths
- Steering: `.kiro/steering/`
- Specs: `.kiro/specs/`
### Steering vs Specification
**Steering** (`.kiro/steering/`) - Guide AI with project-wide rules and context
**Specs** (`.kiro/specs/`) - Formalize development process for individual features
### Active Specifications
- Check `.kiro/specs/` for active specifications
- Use `/kiro-spec-status [feature-name]` to check progress
## Development Guidelines
- Think in English, generate responses in English. All Markdown content written to project files (e.g., requirements.md, design.md, tasks.md, research.md, validation reports) MUST be written in the target language configured for this specification (see spec.json.language).
## Minimal Workflow
- Phase 0 (optional): `/kiro-steering`, `/kiro-steering-custom`
- Phase 1 (Specification):
- `/kiro-spec-init "description"` — initialize spec scaffold
- `/kiro-spec-quick "description"` — fast single-pass spec (requirements + design + tasks, no approval gates)
- `/kiro-spec-batch` — generate specs for all features in a roadmap in parallel
- `/kiro-spec-requirements {feature}`
- `/kiro-validate-gap {feature}` (optional: for existing codebase)
- `/kiro-spec-design {feature} [-y]`
- `/kiro-validate-design {feature}` (optional: design review)
- `/kiro-spec-tasks {feature} [-y]`
- Phase 2 (Implementation): `/kiro-impl {feature} [tasks]`
- `/kiro-validate-impl {feature}` (optional: after implementation)
- `/kiro-verify-completion {feature}` — validate all acceptance criteria are met
- Debugging: `/kiro-debug {feature}` — systematic debugging with spec context
- Code review: `/kiro-review` — review implementation against spec requirements
- Progress check: `/kiro-spec-status {feature}` (use anytime)
## Development Rules
- 3-phase approval workflow: Requirements → Design → Tasks → Implementation
- Human review required each phase; use `-y` only for intentional fast-track
- Use `/kiro-spec-quick` for small, well-understood features; full workflow for anything touching HA/persistence/RBAC
- Keep steering current and verify alignment with `/kiro-spec-status`
- Follow the user's instructions precisely, and within that scope act autonomously: gather the necessary context and complete the requested work end-to-end in this run, asking questions only when essential information is missing or the instructions are critically ambiguous.
## Steering Configuration
- Load entire `.kiro/steering/` as project memory
- Core files: `product.md`, `tech.md`, `structure.md`
- XCE custom files: `testing.md`, `error-handling.md`, `security.md`
- Add new custom files via `/kiro-steering-custom`
Reference in a new issue View git blame Copy permalink