lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

chore: baseline snapshot — prior-session work + ha-consistency-invariants spec family

Browse source 
Pre-existing uncommitted work: ha-repl-hardening payload data plane (7.x),
heartbeat liveness fix (Req 1.3), connection-reuse ASAN fixes, doc cleanup.
New: ha-consistency-invariants umbrella (requirements/design/research) and
its four implementation sub-specs (specs generated via batch, cross-spec
review applied), steering roadmap + product.md Raft+CP decision.
This commit is contained in:
Rongsong Shen 2026-06-11 13:21:50 +00:00
parent 084244941b
commit 06135954ff
88 changed files with 25611 additions and 5285 deletions
Download patch file Download diff file Expand all files Collapse all files

77
src/docs/p1-2-host-select.md
View file

@ -1,71 +1,52 @@
# P1-2: Host Resource Requirement Select — Investigation
## Status
- [>] Under investigation (moved from apiserver-backend-tasks.md)
- **C layer (P1-2a): COMPLETE** — `eval_select_simple` wired in `xce_select.c`
(PEG grammar → AST → CSE, memoized eval). Scheduler integration (Phase 6) done.
See `src/docs/resreq-ast-cse-design.md` for architecture reference.
- **Go bridge (P1-2b): PENDING** — `EvalSelectForHost` remains a stub returning
`errNoCGO`. CGO boundary, resource value lookup from in-memory store, and
`FilterHostByResReqSelect` are still to implement.
---
## Background
The `select` field in an LSF-style resource requirement string (e.g. `select[mem>4096 && type==X86_64]`)
filters candidate hosts before slot allocation. Currently `EvalSelectForHost` is a no-CGO stub that
always returns `errNoCGO`.
The `select` field in an LSF-style resource requirement string (e.g.
`select[mem>4096 && type==X86_64]`) filters candidate hosts before slot
allocation. Currently `EvalSelectForHost` is a no-CGO stub that always returns
`errNoCGO`.
The full feature requires two layers:
1. **C parser + evaluator** — parse the boolean select expression into an AST and evaluate it against
a host's resource values.
2. **Go bridge** — call the C evaluator via CGO and wire it into `FilterHostByResReqSelect` in the
controllers layer.
1. **C parser + evaluator** — parse the boolean select expression into an
AST and evaluate it against a host's resource values.
2. **Go bridge** — call the C evaluator via CGO and wire it into
`FilterHostByResReqSelect` in the controllers layer.
---
## Sub-tasks
### #11 P1-2a: C select expression parser (PEG + eval)
### #11 P1-2a: C select expression parser (PEG + eval) ✓ COMPLETE
**Files implemented:**
- `include/resreq.h` — expression-tree types, enums, `extern` declarations.
- `lib/xce_select.peg` — PEG grammar for select boolean expressions.
- `lib/xce_select.c` — `parse_select()`, `eval_select_simple()`, CSE tree.
- `lib/meson.build` — `xce_select.peg` in build.
### #12 P1-2b: Go EvalSelectForHost + FilterHostByResReqSelect (remaining)
**Files to touch:**
- `include/resreq.h` — Replace empty `xce_resreq_select_` stub with expression-tree types, enums,
and `extern` declarations for `parse_select`, `eval_select`, `free_select`.
- `lib/xce_select.peg` — New PEG grammar for select boolean expressions (right-recursive, handles
`&&`, `||`, `!`, comparison operators, numeric literals, string literals, resource names).
- `lib/resreq.c` — Implement `parse_select()`, `eval_select()`, `free_select()`; update
`kill_primary_resreq()` to call `free_select()`.
- `lib/meson.build` — Add `xce_select.peg` to explicit source list so the PEG tool generates C.
- `lib/testsuite/` — TDD tests for parse/eval before implementing.
**Open questions:**
- Which PEG library is already available in the Nix/Meson build? (`peg`, `lemon`, `re2c`?)
- Exact grammar for comparison operators — LSF allows `mem > 4096`, `type == X86_64`, `swp >= 0`.
Need to confirm all supported operators and value types.
- Does `eval_select` receive a flat key-value map of resource values, or a pointer to the full
`xce_host_t` struct?
- Error reporting strategy — return a bool+error-string, or a tri-state (true/false/parse-error)?
### #12 P1-2b: Go EvalSelectForHost + FilterHostByResReqSelect (blocked by #11)
**Files to touch:**
- `backend/clib.go` — Implement `EvalSelectForHost(hostname, selectExpr string) error` using
`cgo.Handle` for callback context; add `#include "resreq.h"`, extern C declarations,
`xce_select_destroy` helper.
- `backend/clib.go` — Implement `EvalSelectForHost(hostname, selectExpr string) error`
using `cgo.Handle` for callback context; add `#include "resreq.h"`, extern C
declarations, `xce_select_destroy` helper.
- `controllers/hosts.go` — Implement `FilterHostByResReqSelect(hosts []models.HostInfo, expr string)`,
update `preCheckHostRequest()` limit from 4094 → 1024 bytes.
**Open questions:**
- Does `EvalSelectForHost` need to look up live resource values from the in-memory store, or are
values passed in directly by the C layer?
- `cgo.Handle` usage pattern — confirm whether a per-call handle or a session-level handle is needed.
- Does `EvalSelectForHost` look up live resource values from the in-memory
store, or are values passed directly from C?
- `cgo.Handle` usage pattern — per-call or session-level?
- Input validation: max expression length (1024 bytes), allowed characters.
---
## Investigation Checklist
- [ ] Survey existing PEG tooling in the Meson/Nix build environment.
- [ ] Read `include/resreq.h` current state to understand `xce_resreq_select_` stub.
- [ ] Trace how `kill_primary_resreq()` works today and where `free_select()` must be called.
- [ ] Confirm resource value structure passed to `eval_select`.
- [ ] Confirm LSF select grammar spec (operators, value types, nesting depth).
- [ ] Prototype PEG grammar for a subset (numeric comparisons only) and get it compiling.
- [ ] Decide error propagation strategy across the CGO boundary.
- [ ] Write TDD tests for C parser before any implementation.