feat(job-array-single-wal-record): API-server-side (Go) local expansion of the compact push

Adds the Go/CGO receive path for Data.jobArray, mirroring task 5.3's
scheduler-side C implementation: rawJobArrayToModels() (pure-Go
base_jobid + i expansion), cJobArrayToModels() (CGO adapter sibling to
cJobToModel), dispatchJobArray() (dedup via the same store.Jobs.Has()
the ordinary Data_data_jobs path already writes through, single
confirmation log only when at least one element was newly
dispatched), and a new case C.Data_data_jobArray in handleDataset.

Elements route through OnJobUpdated, matching the ordinary single-job
push's own routing. The CGO boundary needed no changes -- both
C.xce_job_submit_t and C.Data_data_jobArray were already exposed from
prior tasks, including task 5.3b's extract_data() fix which this
receive path depends on identically to the scheduler's.

store.ArrayJobs (the array-level rollup aggregate) is deliberately
left unpopulated -- confirmed out of this spec's scope, deferred to a
future spec per requirements.md's Boundary Context.

Task 5.4 of job-array-single-wal-record (Requirements 6.2, 6.4, 6.5, 8.4).
This commit is contained in:
Rongsong Shen 2026-07-15 15:56:48 +00:00
parent 1cbd7df1a8
commit 9e0d87fe50
7 changed files with 345 additions and 9 deletions

View file

@ -3,6 +3,9 @@ package backend
import (
"apiserver/models"
"apiserver/store"
logger "apiserver/utils"
"go.uber.org/zap"
)
// DefaultHandler implements NotificationHandler by writing to stores.
@ -91,6 +94,52 @@ func (h *DefaultHandler) OnResourceLocationUpdated(res *models.ResourceDistribut
store.ResourceLocations.Put(key, *res)
}
// dispatchJobArray delivers a locally-expanded compact array push to the
// handler, one call per genuinely-new element (job-array-single-wal-record
// task 5.4, Req 6.2, 6.4, 6.5, 8.4). CGO-free (called from handleDataset's
// Data_data_jobArray case in clib.go; the elements themselves are already
// decoded by cJobArrayToModels/rawJobArrayToModels before this is called),
// mirroring dispatchConfigPush's split for testability without CGO.
//
// Dedup reuses store.Jobs itself — the SAME jobid-keyed store the ordinary
// single-job push path already writes through (handleDataset's
// Data_data_jobs case calls OnJobUpdated per job with no separate dedup
// step, relying on the map being keyed by jobid) — checked here BEFORE
// calling the handler so a redelivered push (e.g. after an apiserver
// reconnect) can never clobber a jobid whose state has since moved on.
// This mirrors the scheduler's sched_on_job_array() (handler.c) skip-if-
// exists behavior exactly, not a new dedup mechanism.
//
// Logs exactly one confirmation line (base jobid + array size + new-element
// count) when at least one element is genuinely new; a fully-duplicate
// redelivery (every index already exists) produces zero log lines, matching
// task 5.3's resolved semantics for the equivalent scheduler-side push.
// Returns the count of newly-dispatched elements (test-observable).
func dispatchJobArray(handler NotificationHandler, elements []*models.JobInfo, baseJobID, arraySize uint64) int {
if handler == nil {
return 0
}
nNew := 0
for _, job := range elements {
jobid := uint64(job.Unique_JobID)
if store.Jobs.Has(jobid) {
continue
}
handler.OnJobUpdated(job)
nNew++
}
if nNew > 0 {
logger.Info("job array push received and expanded",
zap.Uint64("base_jobid", baseJobID),
zap.Uint64("array_size", arraySize),
zap.Int("new_records", nNew))
}
return nNew
}
// dispatchConfigPush delivers converted config-push data to the handler (CGO-free, for testability).
func dispatchConfigPush(
handler NotificationHandler,