Commit graph

12 commits

Author SHA1 Message Date
ff1be48001 merge 2026-07-23 12:35:10 +08:00
Lei Chen
3a2094a461 Merge branch 'working' into job_peek_lchen
job_peek_lchen: Add fetchfile feature (job output/error peek proxy)
working: HA/Raft infrastructure (consensus, replication, config mgmt)

Conflict resolution summary:
- controller.capnp: fetchfile @41, fetchfilereply @18, fetchfilemsg @30
- cluster.c: bifurcated OPCode_reply for fetchfile + config-push ACK
- job.c: kept full fetchfile pending-request subsystem (~183 lines)
- request.c: kept both xce_fetchfile_req() and XCE_RETRY_DEFAULTS
- helper.c: added missing preVoteRequest/preVoteReply mappings
- All 15 new test registrations (test_cbuf.c + 14 HA tests)
2026-07-22 13:57:25 +08:00
9e0d87fe50 feat(job-array-single-wal-record): API-server-side (Go) local expansion of the compact push
Adds the Go/CGO receive path for Data.jobArray, mirroring task 5.3's
scheduler-side C implementation: rawJobArrayToModels() (pure-Go
base_jobid + i expansion), cJobArrayToModels() (CGO adapter sibling to
cJobToModel), dispatchJobArray() (dedup via the same store.Jobs.Has()
the ordinary Data_data_jobs path already writes through, single
confirmation log only when at least one element was newly
dispatched), and a new case C.Data_data_jobArray in handleDataset.

Elements route through OnJobUpdated, matching the ordinary single-job
push's own routing. The CGO boundary needed no changes -- both
C.xce_job_submit_t and C.Data_data_jobArray were already exposed from
prior tasks, including task 5.3b's extract_data() fix which this
receive path depends on identically to the scheduler's.

store.ArrayJobs (the array-level rollup aggregate) is deliberately
left unpopulated -- confirmed out of this spec's scope, deferred to a
future spec per requirements.md's Boundary Context.

Task 5.4 of job-array-single-wal-record (Requirements 6.2, 6.4, 6.5, 8.4).
2026-07-15 15:56:48 +00:00
d47b3375e7 controller will pass resource <name, value> pair than <idx, value> pair 2026-05-13 07:18:03 +00:00
Lei Chen
d2b47e3985 Init commit of jobs output/error message peek project. 2026-05-09 16:57:54 +08:00
ce2edf0dac refactor: represent time fields as int64 Unix seconds throughout the stack
Collapses a four-step conversion chain (time_t → int64 → time.Time →
RFC3339 string → string → time.Time) to a direct int64 pass-through:

  C time_t → rawJobData int64 → JobInfo int64 → JSON integer → QueryJob int64 → time.Unix() for display

Changes:
- apiserver/models/jobs.go: SubmitTime, QueueTime, StartTime, EndTime,
  ReserveTime in JobInfo and JobAdditionalFields: time.Time → int64
- apiserver/models/common.go: ActionControl.SubmitTime: time.Time → int64
- backend/convert_pure.go: remove time.Unix() calls; assign int64 directly
- backend/convert_test.go: update time assertions to compare int64 values
- xcecli/models/jobs.go: SubmitTime, StartTime, EndTime, ReserveTime in
  QueryJob: string/time.Time → int64; remove time import
- xcecli/common/output/time.go: replace parseJobTime(string) with
  unixToTime(int64) and fmtUnixTime(int64) helpers
- xcecli/common/output/page_print_jobs.go: replace parseJobTime() calls
  with unixToTime()/fmtUnixTime(); remove !startTime.IsZero() guard
  (unixToTime returns false for ts<=0, making it redundant)
- xcecli/common/output/page_print_array_elements_jobs.go: replace
  string-split+time.Parse submit-time block with fmtUnixTime(); remove
  unused time import

Side effect: eliminates the "0001-01-01T00:00:00Z" zero-time
serialization issue at the source — zero timestamps are now 0 (integer)
and unixToTime/fmtUnixTime return false/"-" for ts<=0 without special-
casing needed in callers.
2026-05-09 07:00:49 +00:00
3ed7ef011a fix go jobId schema bug 2026-05-03 11:45:14 +08:00
d7afd72e5b fix front end issues 2026-05-01 21:22:45 +08:00
bc3103a6d5 feat: P1-1 handle resource value push via OPCode_notify
Add OPCode_notify handler in go_push_handler top-level switch
(not inside handlePush) to receive resource value updates from
the controller via xce_notify_data_t/schedUpdateResvals.

- Add 8 CGO union accessors for xce_val_t (which, i32, i64,
  u32, u64, fval, str, bool) since CGO cannot access union fields
- Add handleNotifyData() dispatching NotifyData_data_schedUpdateResvals
  events, iterating xce_resval_list_t and calling cResvalToDistribution
- Add cResvalToDistribution in convert.go with 7 separate switch
  cases for Value_u_* union discriminants (no combining i32/i64)
- Add rawResvalData + rawResvalToDistribution pure-Go version
  in convert_pure.go for testability without CGO
- Add 10 TDD tests covering all 7 value types, nil val guard,
  unknown discriminant, and boolean false
- All 42 backend tests passing
2026-04-17 13:54:03 +00:00
893812e7aa feat: P0-1e expand handleConfigPush with host config and cluster dispatch
Wires the remaining cluster_config_t data into the notification pipeline:
- dispatchConfigPush() extracted as a pure-Go helper (CGO-free, testable)
  that drives OnQueueUpdated, OnResourceUpdated, OnHostConfigUpdated, and
  OnClusterUpdated from a single config push event
- handleConfigPush() delegates to dispatchConfigPush via the new helper,
  eliminating the previous inline loops
- OnHostConfigUpdated and OnClusterUpdated added to NotificationHandler
  interface and implemented with partial-merge semantics in DefaultHandler
- rawHostConfigData/rawHostConfigToModel and rawClusterData/rawClusterToModel
  added to convert_pure.go with their CGO extraction counterparts in convert.go

Safety fixes from code review:
- OnHostUpdated, OnHostConfigUpdated, OnClusterUpdated now copy *incoming to
  a local merged value before mutation, closing the TOCTOU window between
  store.Get and store.Put under concurrent pushes
- cClusterConfQueues and cClusterConfResources gain nil-pointer guards
  matching the existing guard in cClusterConfHostConfigs
- handleConfigPush gains an explicit gHandler nil check
- cClusterConfToClusterInfo returns nil on empty clustername (prevents phantom
  empty-key write to store.Clusters); handleConfigPush logs a warning when nil
- 36 unit tests passing (CGO_ENABLED=0)
2026-04-16 11:25:03 +00:00
960ef62342 fix job array display issue 2026-04-14 18:23:49 +08:00
c9dc8c4676 Merge Go frontend (apiserver + xcecli) into meson build 2026-04-14 11:23:18 +08:00