Commit graph

1074 commits

Author SHA1 Message Date
f1641b1fbf docs: fix 9 P0 and 6 P1 issues from second adversarial review
P0 fixes (9):
- R-22: COW vstore find-slot-and-write not atomic
- R-23: vput grow path without mutex during COW
- R-24: Post-growth slot double-claimed by concurrent threads
- R-25: Missing key_snapshot field for checkpoint serialization
- R-26: malloc(0) for WAL_CHECKPOINT entries
- R-27: Fallback to older snapshot with trimmed WAL
- R-28: hdr.term > ha->term + 1 too restrictive
- R-29: _Atomic on enum type (implementation-defined)
- R-30: handle_jobsubmit() ignores hook return value
- R-31: malloc NULL check missing in standby receiver

P1 fixes (5):
- R-32: No heartbeat during persist_recover() on new primary
- R-33: Catch-up from dead primary removed from promotion
- R-34: Added total_wal_bytes field to persist_t

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 12:55:47 +00:00
54bbfa5865 docs: fix 6 additional P0/P1 review findings for persistence-and-ha-design
P0 fixes (3):
- R-16: vstore COW growth re-check used OOB a->data[old_caps].inuse
- R-17: Standby WAL write return value unchecked before sending ACK
- R-18: Chained CRC validation absent from replication stream

P1 fixes (3):
- R-19: Replicated entry LSN handling unspecified
- R-20: seg_offset in-memory and lost on crash
- R-21: WAL segment rotation prev_crc handling at recovery

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 12:28:39 +00:00
bb66b9766d docs: fix remaining P0/P1 review findings for persistence-and-ha-design
- P0: Fix concurrent vstore race under READ lock during COW mode
  (add xzarray mutex serialization around grow-and-assign section)
- P0: Array job sync replication latency bound (batch mode for
  array submits to avoid O(array_size × sync_timeout_ms) blocking)
- P1: rtti library ABI break documented with two-repo dependency note
2026-04-29 12:02:53 +00:00
8bab418c61 docs: address multi-perspective review findings for persistence-and-ha-design
Fixes 15 P0/P1 issues identified by coherence, feasibility, scope, and
security reviewers:

- P0: traversalLimitInWords fixed to constant bound (not file_size / 8)
- P0: Job ID allocator reset added to recovery sequence
- P0: acked_lsn declared _Atomic uint64_t in struct definition
- P0: ha.role declared _Atomic ctrl_role_t with acquire/release semantics
- P0: hooks_registered idempotency guard added to ha_state_t
- P0: MAX_HA_CANDIDATES reduced from 16 to 4 (speculative generality)
- P0: checkpoint_mutex (pthread_mutex_t) added to persist_t
- P1: Split-brain detection condition explicitly specified
- P1: Heartbeat interval minimum standardized to 2000ms
- P1: Degraded-mode restricted state removed from scope
- P1: Config version hash in heartbeat deferred to P3
- P1: WALEntry Cap'n Proto schema clarified as replication-only
- P1: Snapshot backward compatibility path documented
- P1: Snapshot total_size validated BEFORE malloc
- P1: WAL_QUEUE_CTRL/WAL_WORKER_CTRL replay made idempotent
- P1: mTLS gate enforced before replication code (P1-1)
- P1: PENDING reason ambiguity logged at DEBUG during replay

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 11:54:11 +00:00
607f9519e1 docs: address 30 adversarial review findings for persistence-and-ha-design
Three rounds of adversarial review plus a regression review identified
30 findings across CRITICAL/HIGH/MODERATE/LOW severities. Key fixes:

CRITICAL/HIGH:
- C11 atomics for COW inuse fields (vget, vput, vstore scan + growth)
- kvtab_gc must run inside same write-lock section as thaw
- WAL growth safeguard during in-progress checkpoint
- pthread_mutex_t checkpoint_mutex (not mylock_t) for cond var
- atomic_store_explicit(release) for ha.role demotion write

MODERATE:
- LSN strictly-increasing check (not consecutive) for WAL_CHECKPOINT gaps
- Hook unregistration on demotion + idempotent registration guard
- CLOCK_MONOTONIC recommendation for retention time comparisons
- Cross-kvmgr freeze ordering documented (WAL replay restores consistency)
- pthread_once + rand_r() replacing unsafe rng_seed_once/rand()
- should_retain lock requirement and merge-walk iterator lock doc
- Standby catch-up receive ordering (snapshot-first, then live WAL)
- applied_lsn==0 gap validation clarification

LOW:
- Demotion failure handling (fail-forward as STANDBY, no rollback)
- WAL_CRC_NONE (0xFFFFFFFF) sentinel replacing collision-prone 0
2026-04-29 10:43:02 +00:00
b38babb7d6 feat: implement Phase 4d const-fold transform for select AST
Adds sel_const_fold_pass() — a single-pass bottom-up const-fold that
runs after the Phase 4c flatten loop in parse_select():

  4d-1  EQ/NE with either operand SEL_ANY → SEL_TRUE / SEL_FALSE
        (checks both child slots — commutativity sort makes SEL_ANY
        position non-deterministic after makeSelectNode interning)
  4d-2  Identity laws: TRUE&&X→X, X&&TRUE→X, FALSE||X→X, X||FALSE→X
  4d-3  Annihilation laws: FALSE&&X→FALSE, TRUE||X→TRUE (symmetric)
  4d-4  Complement laws: !TRUE→FALSE, !FALSE→TRUE

Memory semantics:
  level==0 (API server): frees orphaned nodes via free_select_node /
    sel_free_tree; OOM from makeSelectNode returns original node safely
  level>0  (CSE/scheduler): never manually frees; annihilation folds
    return the already-interned left/right singleton directly (no
    redundant makeSelectNode call); OOM guard unconditional so CSE
    mode also degrades gracefully instead of returning NULL

9 new TDD tests added to test_select.c (RED→GREEN verified).
All 166 tests pass (157 prior + 9 Phase 4d).
2026-04-29 08:24:04 +00:00
c2ad96a5e0 docs: incorporate review feedback for resreq-ast-cse-design
- Rename grammar rules: or_expr→select_or, and_expr→select_and,
  not_expr→select_not, atom_expr→select_primary
- Eliminate SEL_BOOL_LITERAL: parse true/false directly as
  SEL_TRUE/SEL_FALSE; boolean resources used standalone (gpu,
  not gpu==true)
- Remove 'true'|'false' from value production in grammar
- Use LSF-style unquoted strings in examples (type==linux,
  not type=="linux")
- Document n-ary vs binary tree decision in §6.2
- Add §8.5 Evaluation Memoization: per-host select_eval_ctx_t
  with node_id-indexed bool_bits bitmap, num_vals, str_vals
  arrays; memoized eval_select with cache check; session lifecycle
  (pre-session capacity, per-host alloc/discard, post-session
  GC + node ID recompaction)
- Add node_id field to xce_select_node_t for eval context indexing
- Clarify §9 two-level sharing: select CSE is global, resreq
  sharing is scoped to (user, queue, project)
- Clarify §10 select vs rusage boundary: select evaluates against
  host snapshot, rusage checks host.available (total - booked)
2026-04-22 02:25:30 +00:00
19898a198a docs: address deep document-review findings for resreq-ast-cse-design
Review by adversarial, feasibility, and product-lens reviewers.
Key changes:

- Add host_eval_ctx_t struct to carry both numeric and string host
  resource values; xvec_idval_t is for job allocation, not host eval
- Fix SEL_FALSE case in eval_select (was returning -1 instead of 0)
- Add bval field to appendix xce_select_node_t struct
- Fix makeSelectNode key_add race (EEXIST now re-finds existing entry)
- Specify two-phase GC sweep to avoid iterator corruption in kvtab
- Add EDA cluster justification for CSE memory savings
- Use %.17g instead of %g for number-to-string coercion (full precision)
- Add #include "kv.h" dependency note for Phase 4
- Add runtime goroutine assertion requirement for CGO bridge safety
- Add gc_sweep_select_nodes() CGO call in Phase 6 session boundary
- Clarify job cleanup happens at session boundaries (no mid-session kill)
- Clarify sel_key_hash/sel_key_cmp must hash fields individually (padding)
2026-04-21 07:05:49 +00:00
98911a2154 docs: add controller code consistency findings to persistence design
Incorporates investigation of vtaskmgr.c, netio.c, cluster.c, job.c,
join.c, and worker.c into the persistence-and-ha-design document:

- Part 2.7: Controller connection context (workers/scheduler/API servers only)
- Part 3.8: COW lock hierarchy (kvmgr rwlock -> xzarray mutex)
- Part 3.9: Checkpoint timer uses delaymgr, not raw pthread
- Part 4.9: Endpoint deferred cleanup (close fd/SSL, defer slot cleanup)
- Part 4.10: mark_worker_unavail() already exists, only needs extending
- Part 6.5: Replication uses endpoint/iomgr/vtask SSL pipeline
- Part 8.3: Lock safety analysis for reply-after-hooks change
- P0-2a, P0-8a, P2-5a: New implementation steps
- Appendix A.3/A.4: standby_conn_t and ha_state_t use endpoint_t*
2026-04-21 04:46:20 +00:00
877741d008 docs: address document-review findings for resreq-ast-cse-design
Multi-persona review (coherence, feasibility, adversarial, product-lens)
identified 25 findings. Applied 14 fixes:

- Fix makeSelectNode key_add return code bug (rc2 → rc3)
- Resolve derefSelectNode contradiction: remove function and refcount
  field, adopt GC-only reclamation model
- Add boolean_literal grammar rule (true/false keywords) and
  SEL_BOOL_LITERAL node type + bval union member
- Clarify transform pipeline: makeSelectNode does commutativity swap
  internally; flattening/const-fold are post-parse tree walks
- Complete const-fold rules with annihilation laws
- Add SEL_NUM bit-cast semantics (±0 distinction, NaN payloads)
- Add resmgr=NULL fallback behavior spec
- Add runtime type safety assertions for debug builds
- Add GC trigger and session-based scheduler context
- Fix section numbering gap (renumber §5.6 → §5.5)
- Add gc_sweep_select_nodes/gc_mark_tree to Phase 2 declarations
- Update kill_primary_resreq instruction for GC model
- Add any keyword semantic inconsistency note (!= vs !(==))

4 open issues remain for implementation phase: string resource
interface, single-goroutine enforcement, kvtab iteration API,
CSE benefit quantification.
2026-04-21 03:29:31 +00:00
33b76d9a77 docs: add select expression AST with CSE/hash-consing design
Design document for implementing the select clause parser with:
- PEG grammar with LSF-compatible resname/string disambiguation
- Hash-consing CSE via intern table with TOCTOU-safe insertion
- Mark-and-sweep GC between scheduling sessions
- Explicit error propagation in eval_select (-1/0/1 tri-state)
- Parse-time type coercion for EQ/NE (e.g., type==5 converts 5 to "5")
- AST transform pipeline: commutativity canonicalization,
  associativity flattening, const-fold (type==any -> SEL_TRUE)
- Special keywords: any, SEL_TRUE, SEL_FALSE
- String host resource accessor (parallel res_idx->char* lookup)
- res_type_t <-> XCE_VAL_* bidirectional mapping
- Single-goroutine CGO bridge invariant
2026-04-20 09:43:07 +00:00
86a65d640a test(apiserver): add integration tests for P0+P1 backend features
Covers ~30 integration tests across 8 sections from the
apiserver-backend-integration-test-plan:

- §1: Cluster config push (store population, merge semantics, zero-preserve)
- §2: Host control (OnHostUpdated merge, validateHost, preCheck, batch ops)
- §3: Resource value push (all 7 value types, nil/unknown discriminant, overwrite)
- §4: Heartbeat keep-alive (interval update, zero-preserve)
- §5: Cross-feature (config+heartbeat, config+host, worker+config field preservation)
- §6: RBAC (cluster_admin bypass, algorithm confusion, expired tokens, hostgroup bypass)
- §7: Store thread safety (concurrent config pushes, host control+config, resource values)
- §8: Input safety (hostname length, empty lists, malformed payloads)

All tests use CGO_ENABLED=0 with //go:build !cgo build tag.
§8.1 hostname-length skipped pending P1-2b implementation.
2026-04-18 11:56:31 +00:00
0a3de8b040 fix: P1-3 address heartbeat timer leak, test drain, and store consistency
- COR-001: drain runHeartbeat goroutine via done channel before test returns,
  eliminating race against concurrent SetHeartbeatInterval tests
- COR-002: replace time.After with time.NewTimer + defer t.Stop() + t.Reset()
  to prevent timer goroutine leaks on context cancellation
- COR-005: guard HeartbeatInterval store write with > 0 check in OnClusterUpdated,
  keeping store consistent with SetHeartbeatInterval's zero-ignore semantics
2026-04-17 14:20:02 +00:00
ffc5b20cdb feat: P1-3 add heartbeat keep-alive goroutine
- add heartbeat.go with heartbeatIntervalSecs var, SetHeartbeatInterval, runHeartbeat (no build tag, CGO-free)
- add heartbeat_test.go with 5 tests covering interval validation and goroutine lifecycle
- add StartHeartbeat to clib.go (CGO impl calling xcecli_heartbeat)
- remove SetHeartbeatInterval stub from stub.go (now real code in heartbeat.go)
- remove TestSetHeartbeatInterval_StubNoOp from stub_funcs_test.go
- call SetHeartbeatInterval in OnClusterUpdated to track backend-pushed interval
- call backend.StartHeartbeat in cmd/main.go after backend registration
2026-04-17 14:12:40 +00:00
bc3103a6d5 feat: P1-1 handle resource value push via OPCode_notify
Add OPCode_notify handler in go_push_handler top-level switch
(not inside handlePush) to receive resource value updates from
the controller via xce_notify_data_t/schedUpdateResvals.

- Add 8 CGO union accessors for xce_val_t (which, i32, i64,
  u32, u64, fval, str, bool) since CGO cannot access union fields
- Add handleNotifyData() dispatching NotifyData_data_schedUpdateResvals
  events, iterating xce_resval_list_t and calling cResvalToDistribution
- Add cResvalToDistribution in convert.go with 7 separate switch
  cases for Value_u_* union discriminants (no combining i32/i64)
- Add rawResvalData + rawResvalToDistribution pure-Go version
  in convert_pure.go for testability without CGO
- Add 10 TDD tests covering all 7 value types, nil val guard,
  unknown discriminant, and boolean false
- All 42 backend tests passing
2026-04-17 13:54:03 +00:00
dc029a6d14 feat: P0-2d add no-CGO stubs for heartbeat, select, cleanup
Add four function stubs to backend/stub.go for the !cgo build path:
- StartHeartbeat(ctx context.Context) — no-op (heartbeat not available)
- SetHeartbeatInterval(secs int) — no-op (interval update not available)
- EvalSelectForHost(hostname, selectExpr string) error — returns errNoCGO
- Cleanup() — no-op (no C resources to release without CGO)

TDD: four tests in stub_funcs_test.go verify stubs behave correctly
(no-op functions don't panic, error stubs return errNoCGO). All backend
tests passing.
2026-04-17 10:02:11 +00:00
f354a5ec78 feat: P0-2c register routes + RBAC for host control/select
Routes and RBAC entries already existed from prior P0-2a/2b work. This
adds three TDD tests that lock down the RBAC invariants:

- TestRBAC_HostControl_ClusterAdminOnly: cluster_admin has host:control
  with domain=all; basic_user must NOT have host:control
- TestRBAC_HostView_BasicUserHasAccess: basic_user has host:view with
  domain=all; queue_admin must NOT have host:view
- TestRBAC_HostControl_DomainTypeIsAll: cluster_admin host:control
  domain type must be "all" (not hostgroup)

All apiserver tests passing.
2026-04-17 09:32:21 +00:00
5807a05cbb fix: address P0-2b review issues - build tag and actionType guard
1. Add //go:build !cgo to control_host_test.go so the stub-testing
   tests only run in non-CGO builds, matching the stub.go build
   constraint. Clean up dead nil check in SignatureCompiles test.

2. Replace two independent if-statements for actionType mapping in
   ControlHost with a map lookup + explicit guard, consistent with
   ControlQueue's ActionMap pattern. Prevents silent zero-value
   actionType if SupportedHostActions expands later.
2026-04-17 08:25:05 +00:00
8124cda83e feat: P0-2b implement ControlHost via xcecli_hostctrl + fix allFlag race
- backend/clib.go: Replace ControlHost stub with real implementation
  using xcecli_hostctrl, following the same pattern as ControlQueue
- controllers/hosts.go: Remove package-level allFlag variable;
  preCheckHostControlRequest now returns (bool, error) so the allFlag
  is a local value in the caller, eliminating the data race
- controllers/queues.go: Same fix for preCheckQueueControlRequest
- Add unit tests for both preCheck functions verifying return values,
  error cases, and concurrent safety
- Add compile-time verification test for ControlHost stub signature
2026-04-17 08:17:56 +00:00
ff940223ec docs: mark P0-2a complete, unblock P0-2b
Update task tracker: P0-2a is done (15 files, 8 TDD tests, 116/116
pass). P0-2b (Go-side ControlHost + allFlag race fix) is now unblocked.
Also fixed pre-existing goto-clean bug in worker_ctrl that leaked
write lock + reference count on HOST_OPEN/HOST_CLOSE no-op paths.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-17 07:47:30 +00:00
dcb1b4fe25 feat: P0-2a C-side plumbing for host control
Add the full call chain for host control operations from the Go API
server through the C library to the controller:

- schema/controller.capnp: HostControl struct + hostctrlmsg @21
  in Request.body union
- include/typemap.h: xce_hostctrl_t* hostctrlmsg in body union
  (placed after @20 clrpreasonmsg, matching capnp ordinal)
- include/client.h, xceapi.h, request.h: declarations for
  xcecli_hostctrl, xce_hostctrl, xce_hostctrl_req
- lib/client.c: xcecli_hostctrl (validate + delegate to xce_hostctrl)
- lib/xceapi.c: xce_hostctrl (validate ctx state + send + wait reply)
- lib/request.c: xce_hostctrl_req (validate hostname/empty/nil,
  accept only HOST_OPEN/HOST_CLOSE, reject HOST_TAG/UNTAG)
- lib/helper.c: Request_body_hostctrlmsg cases in
  xce_request_body() and xce_request_msg_name()
- controller/cluster.c: OPCode_hostcontrol dispatch
- controller/include/cluster.h: do_hostcontrol_op declaration
- controller/worker.c: do_hostcontrol_op + fix pre-existing
  goto-clean bug in worker_ctrl that leaked write lock and
  reference on HOST_OPEN/HOST_CLOSE no-op paths

TDD: 8 xce_hostctrl_req validation tests (null ep, null ctrl,
null/empty hostname, HOST_DO_NONE no-op, HOST_TAG/UNTAG reject,
negative action reject, HOST_OPEN/CLOSE accept via xce_request).

116/116 xcelib tests pass. CGO_ENABLED=0 Go backend tests pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-17 07:46:41 +00:00
ac7ac99ced refactor: make merge semantics explicit in OnHostConfigUpdated/OnClusterUpdated
Start from existing entry and apply this-source's fields, rather than
starting from incoming and restoring the other-source's fields. The new
pattern makes it immediately clear which fields each push type WRITES,
without having to reason about which fields are being "restored".

OnHostUpdated keeps the merged := *incoming + if ok guard because the
worker push also populates HostModel/HostType from attrs; config-sourced
values must only win when a prior config push has already set them.

All 37 backend unit tests pass.
2026-04-17 06:48:25 +00:00
893812e7aa feat: P0-1e expand handleConfigPush with host config and cluster dispatch
Wires the remaining cluster_config_t data into the notification pipeline:
- dispatchConfigPush() extracted as a pure-Go helper (CGO-free, testable)
  that drives OnQueueUpdated, OnResourceUpdated, OnHostConfigUpdated, and
  OnClusterUpdated from a single config push event
- handleConfigPush() delegates to dispatchConfigPush via the new helper,
  eliminating the previous inline loops
- OnHostConfigUpdated and OnClusterUpdated added to NotificationHandler
  interface and implemented with partial-merge semantics in DefaultHandler
- rawHostConfigData/rawHostConfigToModel and rawClusterData/rawClusterToModel
  added to convert_pure.go with their CGO extraction counterparts in convert.go

Safety fixes from code review:
- OnHostUpdated, OnHostConfigUpdated, OnClusterUpdated now copy *incoming to
  a local merged value before mutation, closing the TOCTOU window between
  store.Get and store.Put under concurrent pushes
- cClusterConfQueues and cClusterConfResources gain nil-pointer guards
  matching the existing guard in cClusterConfHostConfigs
- handleConfigPush gains an explicit gHandler nil check
- cClusterConfToClusterInfo returns nil on empty clustername (prevents phantom
  empty-key write to store.Clusters); handleConfigPush logs a warning when nil
- 36 unit tests passing (CGO_ENABLED=0)
2026-04-16 11:25:03 +00:00
00c4472d6c Add HostConfig + Cluster fields to models and update docs 2026-04-16 09:24:22 +00:00
960ef62342 fix job array display issue 2026-04-14 18:23:49 +08:00
d9b5c361c2 submit job from xcecli 2026-04-14 15:55:52 +08:00
c1c9f9e736 minor improve 2026-04-14 12:52:40 +08:00
897ca481b6 remove faked builtin data 2026-04-14 12:19:33 +08:00
c9dc8c4676 Merge Go frontend (apiserver + xcecli) into meson build 2026-04-14 11:23:18 +08:00
43dc12a36c save works 2026-03-06 18:20:30 +08:00
47fc32b02c save works 2026-03-05 16:03:37 +08:00
3f74f14687 save works 2026-03-04 18:18:53 +08:00
acfbec875b save works 2026-03-03 16:49:26 +08:00
f4b1ba4cd7 save works 2026-03-02 12:14:30 +08:00
b748e30d20 save works 2026-02-28 17:55:50 +08:00
b1faeb94b6 improve scheduler 2026-02-27 15:15:16 +08:00
638173dec9 improve plugin 2026-02-26 17:06:07 +08:00
598d381379 improve xkvmgr and testsuite for scheduler 2026-02-26 14:39:12 +08:00
2ee87d4ee6 use librtti for worker/queue/resource 2026-02-25 16:56:50 +08:00
532c68e6e1 - add implement job related objs based on librtti
- add lock()/unlock() to job object
2026-02-23 11:18:45 +08:00
f582742e77 add xzarray_t for replacement of zarray_t finally 2026-02-20 11:25:50 +08:00
ce69347581 - implement xstring by librtti 2026-02-14 14:38:52 +08:00
15fac3cda1 introduce kv 2026-02-13 17:05:41 +08:00
48ea4a0205 - introduce bitset
- add hash_block()
2026-02-13 11:01:33 +08:00
2cf636e607 format code 2026-02-12 11:22:27 +08:00
6cf9aa0979 - make new scheduler work for sequence jobs 2026-02-12 11:20:17 +08:00
e7ab1c16db save 2026-02-09 18:11:27 +08:00
12a6010f29 save work status 2026-02-09 11:42:30 +08:00
ff6d4eeccd fix typo of perflag && reformat .peg 2026-02-09 10:58:30 +08:00
12dcc2a60d - save tmp works
- save works
2026-02-06 17:31:02 +08:00