add kiro-sdd documents

.kiro/specs/scheduler-controller-msg-gap/research.md

@@ -0,0 +1,98 @@
+# Research Log: scheduler-controller-msg-gap
+
+> Feature: scheduler-controller-msg-gap
+> Created: 2026-05-11
+
+## Discovery Type
+
+Extension (light discovery) — extending existing handler code in an established C codebase.
+
+## Summary
+
+The scheduler's `handler.c` dispatcher processes controller messages via `scheduler_req_handler`. Six message cases are stubbed out or empty-bodied. The `vtask_entry()` assembly layer in `src/lib/xcectx.c` rewrites all `pushmsg` into `datamsg`, making the `Request_body_pushmsg` branch dead code. Existing plugin hooks (`plugin_job_update`, `plugin_queue_update`, `plugin_worker_update`, `plugin_job_state_change`) exist for notifying plugins of state changes. A `plugin_job_rusage_update` hook does not yet exist and must be added. The `src/lib/helper.c` file already provides `xce_request_op_name()`, `xce_request_msg_name()`, and `xce_dataset_table()` — a new `xce_notify_event_name()` function is needed.
+
+## Research Log
+
+### Topic 1: Plugin Hook System
+
+**Source:** `src/scheduler/include/plugin.h`, `src/scheduler/src/plugin.c`
+
+**Findings:**
+- Hooks follow a `plugin_*_update(void* sched, xce_sched_*_t*)` pattern
+- Called after state mutation to notify registered plugins
+- Hooks iterate registered plugins and call their function pointers
+- `plugin_job_state_change(void*, xce_sched_job_t*, uint32_t)` takes an extra state parameter
+- No `plugin_job_rusage_update` hook exists yet — must be added
+
+**Implication:** New handlers should call the corresponding plugin hook after mutating scheduler state. The rusage handler needs a new hook typedef and global function.
+
+### Topic 2: Object Conversion Functions
+
+**Source:** `src/scheduler/src/schedobj.c`
+
+**Findings:**
+- `make_sched_job(void* arg, xce_job_t* j)` — maps jobid, times, state, user/queue/project pointers, resreq, mintasks/maxtasks, allocs, accounts
+- `make_sched_queue(void* arg, xce_queue_t* q)` — maps name, attrs, status, priority, accept_interval, sched_delay, resreq
+- `make_sched_worker(void* arg, xce_worker_t* w)` — maps name, status, idx, resvals zarray from `w->resvals_to_sched`
+- `kill_sched_*` functions free their respective objects
+
+**Implication:** Job/queue/worker update handlers should use `make_sched_*` for create-or-update semantics. For updates (existing objects), we need update-in-place logic since `make_sched_*` always creates new objects.
+
+### Topic 3: Resource and Shared Instance Initialization
+
+**Source:** `src/scheduler/src/scheduler.c`
+
+**Findings:**
+- `sched_init_resources()` — iterates `config->resources[]`, calls `makeResource(sched->resmgr, r)` for each, warns on NULL/duplicate
+- `sched_init_sharedInst()` — iterates `config->resvals[]`, uses `MOVE_PTR` to steal val pointers into `sched->sharedInst` zarray
+- Both called from `sched_init_config()` during initial config push
+- For runtime updates, we need incremental versions that can add new or update existing resources
+
+**Implication:** Dataset handlers for `Data_data_resources` and `Data_data_resvals` need new update functions that can both create and modify existing entries.
+
+### Topic 4: Existing Helper Functions
+
+**Source:** `src/lib/helper.c`, `src/include/helper.h`
+
+**Findings:**
+- `xce_request_op_name(xce_request_t* req)` — returns string for OPCode enum
+- `xce_request_msg_name(xce_request_t* req)` — returns string for Request body tag
+- `xce_dataset_table(int table)` — returns string for Data table type enum
+- All return `"unknown"` for unrecognized values
+- `xce_notify_event_name()` does NOT exist yet — must be created
+- Signature pattern: `char* func_name(enumerated_type_or_struct_ptr)` returning `char*`
+
+**Implication:** Create `xce_notify_event_name(int event)` following same pattern. Also consider `char* xce_sched_op_name(xce_sched_op_t op)` for the update request op enum.
+
+### Topic 5: Cap'n Proto Enum Values (NotifyData)
+
+**Source:** `src/schema/controller.capnp`
+
+**Findings:**
+All 19 NotifyData event tags:
+- `jobPidInfo(0)`, `jobFailInfo(1)`, `jobDispatchFail(2)`, `jobFinishInfo(3)`, `schedStartJob(4)`, `schedExitJob(5)`, `schedFailJob(6)`, `schedUpdateJob(7)`, `schedUpdateResvals(8)`, `stoppedJob(9)`, `resumedJob(10)`, `ckptinfo(11)`, `schedUpdateQueue(12)`, `taskPidInfo(13)`, `taskFailInfo(14)`, `taskFinish(15)`, `postDoneJob(16)`, `schedJobRusages(17)`, `updateJob(18)`
+
+**Implication:** The `xce_notify_event_name()` function must cover all 19 values. Only the scheduler-relevant ones need handler implementations (schedUpdateJob, schedUpdateQueue, schedUpdateResvals, schedJobRusages are the gap items).
+
+### Topic 6: Dead Code Analysis
+
+**Source:** `src/lib/xcectx.c` lines 93-106, `src/scheduler/src/handler.c` lines 172-197
+
+**Findings:**
+- `vtask_entry()` accumulates `pushmsg` rows into `ctx->dataset`, then rewrites `body_which = Request_body_datamsg` and `body.datamsg = ctx->dataset`
+- The `Request_body_pushmsg` case in `scheduler_req_handler` (lines 185-198) calls `sched_handle_jobpush()` and `sched_handle_workerpush()`, both of which are stubs returning 0
+- These functions can never be reached via normal message flow
+
+**Implication:** Safe to remove the pushmsg case branch and both stub functions.
+
+## Architecture Decisions
+
+1. **Handler pattern:** New notify handlers follow the same enqueue-to-pend-updates pattern as `sched_on_job_state_change` — create a `xce_sched_update_request_t`, set the op, enqueue via `enq_pend_update()`, process in `sched_update()` main loop.
+
+2. **Update-in-place for job/queue/worker:** For NotifyData update events, we need update semantics (find existing, mutate fields) rather than create semantics (make_sched_*). New `sched_update_job`, `sched_update_queue`, `sched_update_worker` functions will handle this.
+
+3. **New XCE_SCHED_OP enum values:** Need `XCE_SCHED_OP_UPDATE_RESVALS` for resource value updates and `XCE_SCHED_OP_JOB_RUSAGE` for rusage updates.
+
+4. **Plugin hook for rusage:** Add `plugin_job_rusage_hook_t` typedef and `plugin_job_rusage_update()` global function following existing patterns.
+
+5. **Helper function:** Add `xce_notify_event_name(int)` and `xce_sched_op_name(xce_sched_op_t)` to `src/lib/helper.c` and `src/include/helper.h`.