Commit graph

31 commits

Author SHA1 Message Date
86fe35e671 add kiro skills 2026-05-21 03:41:57 +00:00
bc921666de feat: reject job submissions to 'lost_and_found' reserved queue in preCheckJobRequest 2026-05-14 03:19:34 +00:00
e847a32546 add user rshen 2026-05-13 10:51:32 +00:00
febb4a6c4d fix: preserve admin-set host state across worker re-registration in makeWorker_
When a worker re-joins and already exists in workermgr, makeWorker_()
now overrides the existing slot instead of returning EEXIST. Admin-set
flags (LOCKEDU, CLOSED, HIDDEN, CLOSED_BY_ADMIN) survive the override.

Add XCE_WORKER_CLOSED_BY_ADMIN flag to distinguish admin-closed hosts
from workload-closed ones. worker_ctrl sets/clears it alongside CLOSED.

Refactor worker_init/worker_destroy into copy_worker/kill_worker helpers
shared by both fresh creation and override paths.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 10:38:39 +00:00
d47b3375e7 controller will pass resource <name, value> pair than <idx, value> pair 2026-05-13 07:18:03 +00:00
a07a74a943 Merge branch 'persistent_ha' into scheduler 2026-05-12 10:52:46 +00:00
475ed20bba use fmt_jobid() when job submission 2026-05-12 10:52:28 +00:00
ce2edf0dac refactor: represent time fields as int64 Unix seconds throughout the stack
Collapses a four-step conversion chain (time_t → int64 → time.Time →
RFC3339 string → string → time.Time) to a direct int64 pass-through:

  C time_t → rawJobData int64 → JobInfo int64 → JSON integer → QueryJob int64 → time.Unix() for display

Changes:
- apiserver/models/jobs.go: SubmitTime, QueueTime, StartTime, EndTime,
  ReserveTime in JobInfo and JobAdditionalFields: time.Time → int64
- apiserver/models/common.go: ActionControl.SubmitTime: time.Time → int64
- backend/convert_pure.go: remove time.Unix() calls; assign int64 directly
- backend/convert_test.go: update time assertions to compare int64 values
- xcecli/models/jobs.go: SubmitTime, StartTime, EndTime, ReserveTime in
  QueryJob: string/time.Time → int64; remove time import
- xcecli/common/output/time.go: replace parseJobTime(string) with
  unixToTime(int64) and fmtUnixTime(int64) helpers
- xcecli/common/output/page_print_jobs.go: replace parseJobTime() calls
  with unixToTime()/fmtUnixTime(); remove !startTime.IsZero() guard
  (unixToTime returns false for ts<=0, making it redundant)
- xcecli/common/output/page_print_array_elements_jobs.go: replace
  string-split+time.Parse submit-time block with fmtUnixTime(); remove
  unused time import

Side effect: eliminates the "0001-01-01T00:00:00Z" zero-time
serialization issue at the source — zero timestamps are now 0 (integer)
and unixToTime/fmtUnixTime return false/"-" for ts<=0 without special-
casing needed in callers.
2026-05-09 07:00:49 +00:00
a2476d0176 fix: xcecli jobs list -l — hide spurious start line and show pending reason
Three fixes:

1. ExecHosts.UnmarshalJSON: JSON null (sent when no allocation exists)
   was treated as a non-"-" string because json.Unmarshal(null, &str)
   returns nil with str="". This created a phantom HostSlot{NumSlots:1,
   HostName:""} producing "<1*>" for every PEND job. Now null and ""
   are both treated as empty hosts.

2. page_print_jobs.go: added !startTime.IsZero() guard so a zero
   timestamp ("0001-01-01T00:00:00Z") never produces a start line
   even if the status check is bypassed.

3. Added PENDING_REASON field to JobFieldsMap so it is requested from
   the API when -l is passed, and rendered in printJobsVerboseOutput
   for PEND/PSUSP jobs before the start-time section.
2026-05-09 05:15:31 +00:00
af578c29ca fix show exec host for pend jobs 2026-05-09 04:09:16 +00:00
da6d967a33 refactor: simplify HA backend peer handling
- parsePeer: use net.SplitHostPort (handles IPv6, stdlib convention)
- tryConnect: take Go strings, own CGO allocation internally
- RegisterToBackend: remove unreachable 'no HA candidates' error path
- Sync Peers field comment between utils and backend config structs
2026-05-08 03:41:11 +00:00
99d13f5254 feat: add HA candidate failover to API server backend
- BackendConfig.Peers []string carries host:port candidates from YAML (backend.peers)
- RegisterToBackend iterates Peers in order, connecting to first responsive primary
- parsePeer extracted to peers.go (no build tag) so it's testable without CGO
- Single-host path (no Peers) preserved unchanged
- Tests: parsePeer table-driven (11 cases), stub RegisterToBackend with/without peers
2026-05-08 03:31:35 +00:00
b94ff9eb01 feat: P2.5-7a map JOB_STAT_TRANSIENT (0x400) to TRANSIENT in job status API 2026-05-08 02:03:19 +00:00
d6008ead99 improve frontend jobid display 2026-05-07 02:36:24 +00:00
3ed7ef011a fix go jobId schema bug 2026-05-03 11:45:14 +08:00
d7afd72e5b fix front end issues 2026-05-01 21:22:45 +08:00
86a65d640a test(apiserver): add integration tests for P0+P1 backend features
Covers ~30 integration tests across 8 sections from the
apiserver-backend-integration-test-plan:

- §1: Cluster config push (store population, merge semantics, zero-preserve)
- §2: Host control (OnHostUpdated merge, validateHost, preCheck, batch ops)
- §3: Resource value push (all 7 value types, nil/unknown discriminant, overwrite)
- §4: Heartbeat keep-alive (interval update, zero-preserve)
- §5: Cross-feature (config+heartbeat, config+host, worker+config field preservation)
- §6: RBAC (cluster_admin bypass, algorithm confusion, expired tokens, hostgroup bypass)
- §7: Store thread safety (concurrent config pushes, host control+config, resource values)
- §8: Input safety (hostname length, empty lists, malformed payloads)

All tests use CGO_ENABLED=0 with //go:build !cgo build tag.
§8.1 hostname-length skipped pending P1-2b implementation.
2026-04-18 11:56:31 +00:00
0a3de8b040 fix: P1-3 address heartbeat timer leak, test drain, and store consistency
- COR-001: drain runHeartbeat goroutine via done channel before test returns,
  eliminating race against concurrent SetHeartbeatInterval tests
- COR-002: replace time.After with time.NewTimer + defer t.Stop() + t.Reset()
  to prevent timer goroutine leaks on context cancellation
- COR-005: guard HeartbeatInterval store write with > 0 check in OnClusterUpdated,
  keeping store consistent with SetHeartbeatInterval's zero-ignore semantics
2026-04-17 14:20:02 +00:00
ffc5b20cdb feat: P1-3 add heartbeat keep-alive goroutine
- add heartbeat.go with heartbeatIntervalSecs var, SetHeartbeatInterval, runHeartbeat (no build tag, CGO-free)
- add heartbeat_test.go with 5 tests covering interval validation and goroutine lifecycle
- add StartHeartbeat to clib.go (CGO impl calling xcecli_heartbeat)
- remove SetHeartbeatInterval stub from stub.go (now real code in heartbeat.go)
- remove TestSetHeartbeatInterval_StubNoOp from stub_funcs_test.go
- call SetHeartbeatInterval in OnClusterUpdated to track backend-pushed interval
- call backend.StartHeartbeat in cmd/main.go after backend registration
2026-04-17 14:12:40 +00:00
bc3103a6d5 feat: P1-1 handle resource value push via OPCode_notify
Add OPCode_notify handler in go_push_handler top-level switch
(not inside handlePush) to receive resource value updates from
the controller via xce_notify_data_t/schedUpdateResvals.

- Add 8 CGO union accessors for xce_val_t (which, i32, i64,
  u32, u64, fval, str, bool) since CGO cannot access union fields
- Add handleNotifyData() dispatching NotifyData_data_schedUpdateResvals
  events, iterating xce_resval_list_t and calling cResvalToDistribution
- Add cResvalToDistribution in convert.go with 7 separate switch
  cases for Value_u_* union discriminants (no combining i32/i64)
- Add rawResvalData + rawResvalToDistribution pure-Go version
  in convert_pure.go for testability without CGO
- Add 10 TDD tests covering all 7 value types, nil val guard,
  unknown discriminant, and boolean false
- All 42 backend tests passing
2026-04-17 13:54:03 +00:00
dc029a6d14 feat: P0-2d add no-CGO stubs for heartbeat, select, cleanup
Add four function stubs to backend/stub.go for the !cgo build path:
- StartHeartbeat(ctx context.Context) — no-op (heartbeat not available)
- SetHeartbeatInterval(secs int) — no-op (interval update not available)
- EvalSelectForHost(hostname, selectExpr string) error — returns errNoCGO
- Cleanup() — no-op (no C resources to release without CGO)

TDD: four tests in stub_funcs_test.go verify stubs behave correctly
(no-op functions don't panic, error stubs return errNoCGO). All backend
tests passing.
2026-04-17 10:02:11 +00:00
f354a5ec78 feat: P0-2c register routes + RBAC for host control/select
Routes and RBAC entries already existed from prior P0-2a/2b work. This
adds three TDD tests that lock down the RBAC invariants:

- TestRBAC_HostControl_ClusterAdminOnly: cluster_admin has host:control
  with domain=all; basic_user must NOT have host:control
- TestRBAC_HostView_BasicUserHasAccess: basic_user has host:view with
  domain=all; queue_admin must NOT have host:view
- TestRBAC_HostControl_DomainTypeIsAll: cluster_admin host:control
  domain type must be "all" (not hostgroup)

All apiserver tests passing.
2026-04-17 09:32:21 +00:00
5807a05cbb fix: address P0-2b review issues - build tag and actionType guard
1. Add //go:build !cgo to control_host_test.go so the stub-testing
   tests only run in non-CGO builds, matching the stub.go build
   constraint. Clean up dead nil check in SignatureCompiles test.

2. Replace two independent if-statements for actionType mapping in
   ControlHost with a map lookup + explicit guard, consistent with
   ControlQueue's ActionMap pattern. Prevents silent zero-value
   actionType if SupportedHostActions expands later.
2026-04-17 08:25:05 +00:00
8124cda83e feat: P0-2b implement ControlHost via xcecli_hostctrl + fix allFlag race
- backend/clib.go: Replace ControlHost stub with real implementation
  using xcecli_hostctrl, following the same pattern as ControlQueue
- controllers/hosts.go: Remove package-level allFlag variable;
  preCheckHostControlRequest now returns (bool, error) so the allFlag
  is a local value in the caller, eliminating the data race
- controllers/queues.go: Same fix for preCheckQueueControlRequest
- Add unit tests for both preCheck functions verifying return values,
  error cases, and concurrent safety
- Add compile-time verification test for ControlHost stub signature
2026-04-17 08:17:56 +00:00
ac7ac99ced refactor: make merge semantics explicit in OnHostConfigUpdated/OnClusterUpdated
Start from existing entry and apply this-source's fields, rather than
starting from incoming and restoring the other-source's fields. The new
pattern makes it immediately clear which fields each push type WRITES,
without having to reason about which fields are being "restored".

OnHostUpdated keeps the merged := *incoming + if ok guard because the
worker push also populates HostModel/HostType from attrs; config-sourced
values must only win when a prior config push has already set them.

All 37 backend unit tests pass.
2026-04-17 06:48:25 +00:00
893812e7aa feat: P0-1e expand handleConfigPush with host config and cluster dispatch
Wires the remaining cluster_config_t data into the notification pipeline:
- dispatchConfigPush() extracted as a pure-Go helper (CGO-free, testable)
  that drives OnQueueUpdated, OnResourceUpdated, OnHostConfigUpdated, and
  OnClusterUpdated from a single config push event
- handleConfigPush() delegates to dispatchConfigPush via the new helper,
  eliminating the previous inline loops
- OnHostConfigUpdated and OnClusterUpdated added to NotificationHandler
  interface and implemented with partial-merge semantics in DefaultHandler
- rawHostConfigData/rawHostConfigToModel and rawClusterData/rawClusterToModel
  added to convert_pure.go with their CGO extraction counterparts in convert.go

Safety fixes from code review:
- OnHostUpdated, OnHostConfigUpdated, OnClusterUpdated now copy *incoming to
  a local merged value before mutation, closing the TOCTOU window between
  store.Get and store.Put under concurrent pushes
- cClusterConfQueues and cClusterConfResources gain nil-pointer guards
  matching the existing guard in cClusterConfHostConfigs
- handleConfigPush gains an explicit gHandler nil check
- cClusterConfToClusterInfo returns nil on empty clustername (prevents phantom
  empty-key write to store.Clusters); handleConfigPush logs a warning when nil
- 36 unit tests passing (CGO_ENABLED=0)
2026-04-16 11:25:03 +00:00
00c4472d6c Add HostConfig + Cluster fields to models and update docs 2026-04-16 09:24:22 +00:00
960ef62342 fix job array display issue 2026-04-14 18:23:49 +08:00
d9b5c361c2 submit job from xcecli 2026-04-14 15:55:52 +08:00
897ca481b6 remove faked builtin data 2026-04-14 12:19:33 +08:00
c9dc8c4676 Merge Go frontend (apiserver + xcecli) into meson build 2026-04-14 11:23:18 +08:00