modelJobToSubmit/freeJobSubmit (submit direction) and
cJobToModel/cJobArrayToModels (receive direction, both single-job and
array-job paths) now carry from_host across the CGO boundary.
rawSubmitData/rawJobToModel (the pure-Go, CGO-free mapping layer) gain
the field too, completing the round-trip so JobInfo.FromHost reflects
whatever the controller actually has -- which, as of the prior two
commits, is now always the client-asserted hostname. backend/handler.go
needs no change: its existing blind overwrite in OnJobUpdated is
correct by construction once every controller push genuinely carries
the right value.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AgYRU3zDTqNGV5Jk8aiKLg
P0 (done, cluster-verified):
- main_reason was always 0: add_job_preason only set it when BOTH host and res
were NULL, but the allocator always attaches host/res detail rows. Now the
first detail add wins as main_reason, so the API's main_reason_name is
meaningful (was "unknown(0)"). Verified live: PEND jobs now report
main_reason=3/match_fail + per-host reason_name.
- limit plugin attach: host-based quota hit (hg1_slots_per_host) -> LIMIT_HOST
+ limit name; global quota hit (ug1_slots, ...) -> LIMIT_GLOBAL + limit name.
Previously both blocks were silent (job just stayed PEND).
- Extended the PREASON_* enum (was 3 codes) with LSF-style classes:
LIMIT_GLOBAL/LIMIT_HOST/QUEUE_CLOSED/QUEUE_DRAINING/TIME_WINDOW/HOST_UNAVAIL.
apiserver pendReasonName maps them; host/res detail rows now carry reason_name.
P1 (implemented):
- queue status: sched_queue_is_ready no longer a stub — closed/inactivated
queues are not electable; the fcfs election attaches QUEUE_CLOSED /
QUEUE_DRAINING to every PEND job in a skipped queue.
- worker filter: sched_try_alloc_on_host_ skips UNAVAIL/CLOSED workers and
attaches HOST_UNAVAIL instead of trying to allocate on them.
Unit tests: preason suite updated for the main_reason change + 3 new tests
(limit host/global, first-add-wins); 295 lib tests pass under ASAN.
apiserver backend tests pass.
Co-Authored-By: Claude <noreply@anthropic.com>
The pending_reason field was declared on models.JobInfo but never populated:
scheduler computed PEND reasons and the controller stored them on
job->pend_reasons, yet the data stopped there. Two gaps closed:
1. controller/apiserver.c: apiserver_job_hook_fun now handles
HOOK_DETAIL_SETPREASON by pushing the updated job to API servers (reusing
apiserver_job_update_fun). Without this, a job whose PEND reasons change
never reached the API server — only JOB_NEW/UPDATE/START/EXIT events did,
and setpreason fired none of those.
2. backend/convert.go + convert_pure.go: cJobToModel / rawJobToModel now
extract xce_job_t.pend_reasons (main_reason + host/res reasons) and
marshal them into models.JobInfo.PendingReason as JSONB. models/jobs.go:
pending_reason added to JOB_DEFAULT_COLUMN so default job queries return
it (it was only in JSONBJFields, never selected).
Verified live on t1/t2/t3: a job with resreq "type == nosuchhost" stays
PEND and returns
"pending_reason":{"main_reason":0,"host_reasons":[{"host":"t1","reason":3},
{"host":"t2","reason":3},{"host":"t3","reason":3}]}
(main_reason=3 PREASON_MATCH_FAIL per host) via both /jobs list, /jobs/:id
and the SSE stream. 4 new unit tests (rawJobToModel pendReason marshal +
name mapping), all apiserver backend tests pass.
Co-Authored-By: Claude <noreply@anthropic.com>
Two stacked defects in the apiserver job-control link, both of which kept
hold/release/cancel from ever reaching the controller:
1. controllers/jobs.go: validateJobControl and foundJob asserted
jobinfo.Data["array_idx"].(int64), but StoreToOrderMaps produces
array_idx as int (models.JobInfo.ArrayIdx is declared int) -> panic
-> Gin Recovery -> HTTP 500 before backend.ControlJob ran. Add
normalizeJobInt64 / jobArrayIdx helpers and use them in both
functions (job_status already had this fix via normalizeJobStatus;
array_idx was missed).
2. backend/clib.go: backend.ControlJob set xce_jobctrl_t.action to the
models.JOB_* constant (JOB_STOP=1 / JOB_RESUME=2 / JOB_KILL=3 /
JOB_CHKPNT=8), but that field is the capnp JobControl union
discriminant (JobControl_u_siginfo=0 / JobControl_u_ckptoption=1).
The controller's job_ctrl() selected the wrong union branch (or
none, since encode_JobControl drops the union pointer for
non-0/1 discriminants) and rejected every control with -EINVAL.
Add jobCtrlDiscriminant() and use it.
Also: resolveExecUser was defined in the cgo-only convert.go but is
exercised by a non-cgo test, so `make test` (CGO_ENABLED=0) failed to
build the backend package. Move it to convert_pure.go so it compiles in
both modes.
Tests: 3 controller regression tests (validateJobControl allowed /
not-allowed branches, foundJob array-child branch) + 1 backend
jobCtrlDiscriminant test. make test fully green. Cluster E2E verified:
hold -> PSUSP, release -> PEND, cancel -> EXIT on a PEND job, no 500,
no -EINVAL, all daemons healthy.
Co-Authored-By: Claude <noreply@anthropic.com>
Adds the Go/CGO receive path for Data.jobArray, mirroring task 5.3's
scheduler-side C implementation: rawJobArrayToModels() (pure-Go
base_jobid + i expansion), cJobArrayToModels() (CGO adapter sibling to
cJobToModel), dispatchJobArray() (dedup via the same store.Jobs.Has()
the ordinary Data_data_jobs path already writes through, single
confirmation log only when at least one element was newly
dispatched), and a new case C.Data_data_jobArray in handleDataset.
Elements route through OnJobUpdated, matching the ordinary single-job
push's own routing. The CGO boundary needed no changes -- both
C.xce_job_submit_t and C.Data_data_jobArray were already exposed from
prior tasks, including task 5.3b's extract_data() fix which this
receive path depends on identically to the scheduler's.
store.ArrayJobs (the array-level rollup aggregate) is deliberately
left unpopulated -- confirmed out of this spec's scope, deferred to a
future spec per requirements.md's Boundary Context.
Task 5.4 of job-array-single-wal-record (Requirements 6.2, 6.4, 6.5, 8.4).
Add OPCode_notify handler in go_push_handler top-level switch
(not inside handlePush) to receive resource value updates from
the controller via xce_notify_data_t/schedUpdateResvals.
- Add 8 CGO union accessors for xce_val_t (which, i32, i64,
u32, u64, fval, str, bool) since CGO cannot access union fields
- Add handleNotifyData() dispatching NotifyData_data_schedUpdateResvals
events, iterating xce_resval_list_t and calling cResvalToDistribution
- Add cResvalToDistribution in convert.go with 7 separate switch
cases for Value_u_* union discriminants (no combining i32/i64)
- Add rawResvalData + rawResvalToDistribution pure-Go version
in convert_pure.go for testability without CGO
- Add 10 TDD tests covering all 7 value types, nil val guard,
unknown discriminant, and boolean false
- All 42 backend tests passing
Wires the remaining cluster_config_t data into the notification pipeline:
- dispatchConfigPush() extracted as a pure-Go helper (CGO-free, testable)
that drives OnQueueUpdated, OnResourceUpdated, OnHostConfigUpdated, and
OnClusterUpdated from a single config push event
- handleConfigPush() delegates to dispatchConfigPush via the new helper,
eliminating the previous inline loops
- OnHostConfigUpdated and OnClusterUpdated added to NotificationHandler
interface and implemented with partial-merge semantics in DefaultHandler
- rawHostConfigData/rawHostConfigToModel and rawClusterData/rawClusterToModel
added to convert_pure.go with their CGO extraction counterparts in convert.go
Safety fixes from code review:
- OnHostUpdated, OnHostConfigUpdated, OnClusterUpdated now copy *incoming to
a local merged value before mutation, closing the TOCTOU window between
store.Get and store.Put under concurrent pushes
- cClusterConfQueues and cClusterConfResources gain nil-pointer guards
matching the existing guard in cClusterConfHostConfigs
- handleConfigPush gains an explicit gHandler nil check
- cClusterConfToClusterInfo returns nil on empty clustername (prevents phantom
empty-key write to store.Clusters); handleConfigPush logs a warning when nil
- 36 unit tests passing (CGO_ENABLED=0)
