docs: save scheduler-snapshot-job-allocation and xce-test-cli-ux spec docs, sync OpenWolf state
Adds the previously-uncommitted spec.json/requirements.md/design.md/research.md for scheduler-snapshot-job-allocation (feature now fully implemented across tasks 1.1-4.5) and the in-progress xce-test-cli-ux spec's requirements.md/ spec.json. Updates brief.md, a security.md note on an unrelated deferred RBAC persistence gap found during this feature's cross-component survey, and syncs OpenWolf's anatomy/memory/token-ledger/daemon-log bookkeeping. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SSuXfK6nD1Nue5EjjdhjNZ
This commit is contained in:
parent
5948e07964
commit
b64682dce9
16 changed files with 3053 additions and 1353 deletions
|
|
@ -3,6 +3,8 @@
|
|||
## Problem
|
||||
Simulation-mode capacity/contention analysis is untrustworthy for any scenario seeded from a snapshot -- the common case, since real investigations start from a live snapshot + replay (`xce_scheduler -i -s <replay>`). Reported live: after loading a snapshot with 22 running jobs and replaying 79 sessions forward, `info summary` correctly counts 22 running jobs, but `info worker`'s "used" resource values only ever reflect 2 of them -- the rest permanently show zero contribution, making any capacity check against a snapshot-seeded scenario silently wrong.
|
||||
|
||||
A second, more severe defect in the same load path was found during this session's cross-component persistence survey (see Current State): every job restored from a snapshot -- PENDING or RUNNING -- gets a NULL `job->schedobj`, a field the live allocator dereferences unconditionally. This is a latent crash (same class as the already-fixed bug-1569 SIGSEGV), not just a wrong-value bug, and is folded into this spec since it's the same load-side reconstruction step and the same root cause pattern (a field silently missing from restoration, not from the wire).
|
||||
|
||||
## Current State
|
||||
- `xce_sched_core_load_snapshot()` (`snapshot_load.c`) restores a job's `state` correctly (so a snapshot-loaded job shows as RUN) but never restores `job->allocs`/`job->accounts` -- confirmed: `build_wire_job()` (`snapshot_wire.c:167`) carries an explicit `/* job->allocs/accounts: not captured */` comment.
|
||||
- This was a deliberate, documented scope decision from `scheduler-core` task 8.1 (`snapshot.c`'s own "Scope decisions" header comment), not an oversight -- there was no deep-copy helper for `xce_job_alloc_t` anywhere in the codebase, and aliasing it (instead of deep-copying) across two independently-alive cores would create a genuine double-free hazard the moment either core's normal job-lifecycle path frees it. The header explicitly flags the consequence: a restored RUNNING job "forgets what it is currently dispatched on."
|
||||
|
|
@ -12,29 +14,37 @@ Simulation-mode capacity/contention analysis is untrustworthy for any scenario s
|
|||
- No existing deep-copy helper for `xce_job_alloc_t`/`xce_host_alloc_t`/`xce_alloc_t` anywhere in the codebase (confirmed via grep for `dup_`/`copy_`/`clone_` conventions). `snapshot.c`/`snapshot_wire.c`/`snapshot_load.c` deliberately contain no capnp code (an existing, intentional architectural boundary) -- the deep-copy should be composed from the existing `make_alloc`/`make_host_alloc`/`make_job_alloc` constructors (`src/lib/xcealloc.c`), not the capnp-generated codec, and paired with the existing `free_JobAlloc_ptr` for cleanup.
|
||||
- Side-finding, already fixed and committed this session ahead of this spec (bug-1722): `make_job_account()`/`make_host_allocs()` in `xcealloc.c` had a live heap buffer overflow for any job spanning more than one host (`calloc(1, ...)` sized for one host pointer, loop wrote up to `nhosts-1`). Fixed independently before this spec starts, since this spec's own tests will exercise multi-host allocations through `make_job_account()` more than any existing code path does today.
|
||||
- **Direct precedent, same bug class, sibling persistence layer, already fixed (bug-1678, Gitea #43):** the controller's WAL had the identical gap for the exact same data -- after restarting all daemons, a previously-running job's `STATUS=RUN` recovered correctly on replay but `EXEC_HOST` came back permanently blank, because `WAL_JOB_DISPATCH` never persisted the job's allocation. Part of a larger campaign (bug-1687/1689/1690/1691/1692, all already committed before this session, plus bug-1697 start_time, committed this session) fixing WAL replay silently dropping various job fields on restart. Bug-1678's own writeup confirms `JobAlloc`/`xce_job_alloc_t` was already a full top-level capnp struct requiring **no schema change** -- the fix was pure save/load glue (`capnp_encode_joballoc()`/`decode_joballoc()` in `persist.c`, mirroring existing `capnp_encode_worker()`/`restore_job()` patterns), landed as `db142d41 fix(controller): persist dispatch-time resource allocation into WAL_JOB_DISPATCH`. This independently confirms the viability check's finding for the scheduler side (existing wire field, no schema change, pure glue) via a working, shipped implementation of the identical shape in a sibling persistence mechanism -- the controller's WAL and the scheduler's snapshot are separate, non-sharing persistence layers, so fixing one did not fix the other.
|
||||
- **`job->schedobj` -- undocumented, likely-live crash risk, found and confirmed viable-to-fix this session:** the live JOBS_PUSH path (`core_build_job()`, `core_events.c:199-266`) unconditionally builds `job->schedobj` for every job via `core_build_schedobj()` (`core_events.c:174-197`) -- required scaffolding the allocator dereferences with no NULL check (`sched_make_allocation()`/`sched_make_primary_allocation()`/`sched_make_compound_allocation()`, `scheduler.c:179-267`; also `plugins/limit.c`). `snapshot_load.c`'s job-reconstruction loop never builds it, so every snapshot-restored job (PENDING or RUNNING) gets `job->schedobj == NULL` -- the same failure shape as the already-fixed bug-1569 SIGSEGV, reproducible the moment a restored PENDING job reaches the allocator. Not mentioned in `snapshot.c`'s own "Scope decisions" header, and violates that file's own stated intent to keep job-reconstruction "field-for-field identical" to `core_build_job()`.
|
||||
- **`schedobj` is fully derivable from data this spec already restores -- confirmed, no new wire field needed.** `core_build_schedobj(obj, j, resreq)` (and the older, structurally identical `make_sched_obj()` in `schedobj.c`, used by `make_sched_job()`) builds the struct from exactly three inputs, all already present by the time job reconstruction reaches this step: (1) `job->user`/`queue`/`project` -- interned strings copied from `j->submit->user/queue/project_name`, already restored via `sched_get_user/queue/project`; (2) `job->resreq` -- already parsed from the snapshot's captured resreq string, its `tag`/`primary`/`compound`/`alter` fields just get wrapped in owner-linked `xce_{primary,compound}_schedobj_t` structs; (3) freshly-initialized empty `glimits`/`hlimits` (session-scoped limit-tracking bookkeeping, correctly starts empty for a restored job exactly as it does for a freshly-submitted one). `configver=0` and `obj->alloc`/`obj->reason` (and the primary/compound variant's own `reason`) are zero/NULL at construction in the live path too -- traced `obj->alloc`'s only writers (`scheduler.c:194/228/257`, inside `sched_make_allocation()`) and confirmed it is pure per-scheduling-round scratch state for an active PENDING-job allocation attempt, never persisted job state, so NULL-after-restore is correct for both a RUNNING job (never re-enters that path) and a PENDING job (gets it populated on its next evaluation, same as a fresh submission). One implementation wrinkle: `core_build_schedobj()` is `static` to `core_events.c`, and per the existing no-shared-helper convention (see Existing Spec Touchpoints), `snapshot_load.c` cannot call it directly -- the fix replicates the same construction logic locally, mirroring how job/queue/worker reconstruction already works there, not a call into `core_events.c`.
|
||||
|
||||
## Desired Outcome
|
||||
A job that is already running as of a snapshot's save point survives a save/load round-trip with its full allocation intact: `job->allocs` populated, `job->accounts` reconstructed via the existing `make_job_account()`, and worker/shared resource "used" accounting correctly reflects it from the moment the snapshot loads -- no longer dependent on whether its dispatch decision happens to also appear later in a replay stream. A snapshot file saved before this change still loads cleanly (no crash, no misinterpretation) -- those jobs simply have nothing to restore, the same `used=0` behavior as today, a clean degrade rather than a regression.
|
||||
|
||||
Every job restored from a snapshot -- PENDING or RUNNING -- also has a valid, non-NULL `job->schedobj`, rebuilt directly from its already-restored `resreq`/user/queue/project (no new data captured, no wire change), matching what a freshly-submitted job gets and eliminating the latent NULL-deref crash risk in the allocator for any restored PENDING job.
|
||||
|
||||
## Approach
|
||||
Extend snapshot save/load to populate/restore the wire type's already-existing `allocs` field -- no schema change:
|
||||
Extend snapshot save/load to populate/restore the wire type's already-existing `allocs` field, and locally reconstruct `job->schedobj` -- no schema change for either:
|
||||
1. New deep-copy helper(s), composed from the existing `make_alloc`/`make_host_alloc`/`make_job_alloc` constructors, that build a fresh, independent `xce_job_alloc_t` from a source one -- paired with the existing `free_JobAlloc_ptr` for cleanup.
|
||||
2. `build_wire_job()` (`snapshot_wire.c`) populates the deep-copied allocation when `job->allocs != NULL`.
|
||||
3. `xce_sched_core_load_snapshot()` (`snapshot_load.c`) restores it via the same deep-copy (never aliasing the source snapshot's own copy) and calls `make_job_account()` to rebuild `job->accounts` -- mirroring the exact pattern `sim_reconcile_unaccounted_jobs()` already established for its own narrower case.
|
||||
4. Backward compatibility is a plain NULL/`nhosts>0` guard, not the `internal_version` byte-blob pattern -- unnecessary for a real typed field.
|
||||
5. In the same `xce_sched_core_load_snapshot()` job-reconstruction loop, locally replicate `core_build_schedobj()`'s exact field-for-field logic (user/queue/project strings, resreq-tag-driven primary/compound/alter wrapper, fresh empty `glimits`/`hlimits`, `configver=0`) to build `job->schedobj` for every restored job, regardless of state. No wire/schema addition -- every input it needs is already restored earlier in the same reconstruction path. Cannot call `core_build_schedobj()` directly (`static` to `core_events.c`); this is a local reimplementation, consistent with the existing no-shared-helper convention already followed for job/queue/worker reconstruction.
|
||||
|
||||
## Scope
|
||||
- **In**: the deep-copy helper; `build_wire_job()` save-side wiring; `xce_sched_core_load_snapshot()` load-side wiring + `job->accounts` reconstruction; cross-core round-trip tests (multi-host, multi-resource, shared-resource allocations); an old-snapshot-file backward-compat test; a sim-mode end-to-end test proving `info worker`'s "used" value is correct immediately after loading a snapshot with running jobs (closing the originally reported gap).
|
||||
- **Out**: any change to `sim_reconcile_unaccounted_jobs()` itself (already correct for its own case, stays as-is); any capnp wire schema change (not needed); any change to `internal_state`/`internal_version` (unrelated field); the `job->priority` and queue `project_name`/`job_pick_plugin` gaps `snapshot.c`'s own "Scope decisions" also documents (separate, unrelated, pre-existing, not part of this fix).
|
||||
- **In**: the deep-copy helper; `build_wire_job()` save-side wiring; `xce_sched_core_load_snapshot()` load-side wiring + `job->accounts` reconstruction; local `job->schedobj` reconstruction for every restored job (PENDING and RUNNING); cross-core round-trip tests (multi-host, multi-resource, shared-resource allocations); a lightweight test proving a snapshot with no allocation/schedobj data (the natural NULL-field case, not an archived legacy file) still loads cleanly with today's `used=0` behavior; a sim-mode end-to-end test proving `info worker`'s "used" value is correct immediately after loading a snapshot with running jobs (closing the originally reported gap); a test proving a restored PENDING job can reach the allocator (`sched_make_allocation()`) without a NULL-deref (closing the schedobj gap).
|
||||
- **Backward compatibility note (pre-release product):** since these are typed, optional wire fields rather than the `internal_version` byte-blob pattern, an absent field simply decodes as NULL -- correct handling is a NULL guard that falls out of writing the load logic correctly, not a dedicated compatibility effort. No archived pre-feature snapshot file needs to be preserved/maintained as a fixture; a freshly-generated snapshot with no allocation data exercises the same code path.
|
||||
- **Out**: any change to `sim_reconcile_unaccounted_jobs()` itself (already correct for its own case, stays as-is); any capnp wire schema change (not needed); any change to `internal_state`/`internal_version` (unrelated field); the `job->priority` and queue `project_name`/`job_pick_plugin` gaps `snapshot.c`'s own "Scope decisions" also documents (separate, unrelated, pre-existing, not part of this fix); `job->rusages` (no wire slot exists at all -- would need an actual schema addition, unlike allocs/schedobj which need none -- tracked as a separate follow-on, not folded in here).
|
||||
|
||||
## Boundary Candidates
|
||||
- **Deep-copy helper** (`xcealloc.c`, or a new file if it grows past the codebase's ~800-line-per-file convention): the one piece of genuinely new logic.
|
||||
- **Deep-copy helper** (`xcealloc.c`, or a new file if it grows past the codebase's ~800-line-per-file convention): the one piece of genuinely new logic for the allocation half.
|
||||
- **Save-side wiring** (`build_wire_job`/`snapshot_wire.c`): small, mechanical once the helper exists.
|
||||
- **Load-side wiring** (`xce_sched_core_load_snapshot`/`snapshot_load.c`): small, mirrors `sim_reconcile_unaccounted_jobs()`'s existing `job->accounts` reconstruction pattern.
|
||||
- **Test coverage**: cross-core round-trip (multi-host/multi-resource/shared), backward-compat (old snapshot file), and the end-to-end sim-mode "used" accounting proof.
|
||||
- **`schedobj` local reconstruction** (`snapshot_load.c`): small, mechanical -- a local field-for-field mirror of `core_build_schedobj()`, no new data, no shared helper.
|
||||
- **Test coverage**: cross-core round-trip (multi-host/multi-resource/shared), backward-compat (old snapshot file), the end-to-end sim-mode "used" accounting proof, and the restored-PENDING-job-reaches-the-allocator schedobj proof.
|
||||
|
||||
## Out of Boundary
|
||||
- `job->priority`, queue `project_name`/`job_pick_plugin` -- both already-documented, unrelated pre-existing snapshot gaps.
|
||||
- `job->rusages` -- undocumented gap found this session, but needs a real wire schema addition; separate follow-on, not folded into this spec.
|
||||
- Any change to the live (non-snapshot) dispatch/accounting path.
|
||||
- `sim_reconcile_unaccounted_jobs()` itself.
|
||||
|
||||
|
|
@ -50,4 +60,5 @@ Extend snapshot save/load to populate/restore the wire type's already-existing `
|
|||
- No capnp code in `snapshot.c`/`snapshot_wire.c`/`snapshot_load.c` -- existing architectural boundary, `snapshot.c`'s own header explains why.
|
||||
- Must never alias the deep-copied allocation across two independently-alive cores -- the exact hazard the original scope decision flagged. Every restore must be a genuine deep copy, correctly paired with `free_JobAlloc_ptr` on cleanup.
|
||||
- `xce_sched_core_save_snapshot()`/`xce_sched_core_load_snapshot()` both already enforce `session_active==0` (-EBUSY guard) -- confirmed no concurrent job-list mutation race during the deep copy. This fix must preserve that guarantee, not weaken it.
|
||||
- `job->schedobj` reconstruction must exactly mirror `core_build_schedobj()`'s field set (user/queue/project strings, resreq-tag wrapper, empty `glimits`/`hlimits`, `configver=0`) -- no additional serialized data required or permitted, since everything it needs is already present after `job->resreq`/`user`/`queue`/`project` restoration earlier in the same reconstruction path. Do not call into `core_events.c`'s static builders; replicate locally.
|
||||
- This spec is the confirmed instance motivating a new standing invariant, formalized alongside this brief: `xce-audit-invariants`' scheduler dimension 4 ("Field-complete snapshot save/load parity") and `persistence.md`'s "Snapshot Save/Load Field-Completeness (Scheduler)" section. Tasks generated for this spec should include running `/xce-audit-invariants scheduler` (or at minimum manually re-checking dimension 4) as part of final validation, not just the feature's own unit tests -- proving THIS field round-trips doesn't by itself prove no OTHER field has the same gap.
|
||||
|
|
|
|||
290
.kiro/specs/scheduler-snapshot-job-allocation/design.md
Normal file
290
.kiro/specs/scheduler-snapshot-job-allocation/design.md
Normal file
|
|
@ -0,0 +1,290 @@
|
|||
# Design Document: scheduler-snapshot-job-allocation
|
||||
|
||||
## Overview
|
||||
|
||||
This feature extends the scheduler's existing snapshot save/load mechanism (`snapshot.c`/`snapshot_wire.c`/`snapshot_load.c`, `scheduler-core` task 8.1) so that a job's resource allocation and internal scheduling representation survive a save/load round-trip. Today, `xce_sched_core_load_snapshot()` restores a job's `state` correctly but silently drops two fields the job depends on: its resource allocation (`job->allocs`/`job->accounts`) and its internal scheduling representation (`job->schedobj`). The first causes `info worker`'s "used" resource values to permanently undercount snapshot-restored running jobs; the second is a latent NULL-dereference crash risk in the allocator for any restored PENDING job.
|
||||
|
||||
**Users**: operators running simulation-mode capacity/contention analysis via `xce_scheduler -i -s <replay>`, and any future consumer of the scheduler's snapshot format (e.g. the roadmap's planned `xce_test` `snapdump` tooling).
|
||||
|
||||
**Impact**: extends `build_wire_job()` (`snapshot_wire.c`) and `xce_sched_core_load_snapshot()` (`snapshot_load.c`); adds one new function to `xcealloc.c`/`xcealloc.h`. No wire schema change — both fixes restore data using capacity the wire format or the load path already has.
|
||||
|
||||
### Goals
|
||||
- Restore `job->allocs`/`job->accounts` for a job already RUNNING at snapshot save time, so worker/shared resource "used" accounting is correct immediately on load (Req 1).
|
||||
- Construct a valid, non-NULL `job->schedobj` for every restored job — PENDING or RUNNING — so it can reach the allocator without crashing (Req 2).
|
||||
- Preserve every existing snapshot/reconciliation guarantee, including clean loading of snapshot files that predate this feature (Req 3).
|
||||
|
||||
### Non-Goals
|
||||
- Restoring `job->priority` or a queue's `project_name`/`job_pick_plugin` — existing, separate, already-documented `snapshot.c` scope decisions, unchanged.
|
||||
- Capturing `job->rusages` — no wire slot exists for it at all today; closing that gap needs an actual schema addition, tracked as a separate follow-on.
|
||||
- Any change to `sim_reconcile_unaccounted_jobs()` (`scheduler-sim-eager-reconciliation`) — it already correctly handles the complementary case (a dispatch decision discoverable later in replay) and is not modified here.
|
||||
- Fixing `kill_sched_job()`'s incomplete per-job cleanup (it frees `job->rusages` only, not `resreq`/`schedobj`/`allocs`/`accounts`) — a pre-existing gap affecting every job today, live or restored; an unrelated, broader job-lifecycle concern (see `research.md` Risks).
|
||||
- Any change to live, non-snapshot dispatch or resource-accounting behavior.
|
||||
|
||||
## Boundary Commitments
|
||||
|
||||
### This Spec Owns
|
||||
- `dup_job_alloc()`: a new deep-copy function (`xcealloc.c`/`xcealloc.h`) that builds an independent `xce_job_alloc_t` from a source one.
|
||||
- `build_wire_job()` save-side wiring: populating the wire type's already-existing `allocs` field from `job->allocs` when non-NULL.
|
||||
- `xce_sched_core_load_snapshot()` load-side wiring: restoring `job->allocs`/`job->accounts` via the deep copy, and constructing `job->schedobj` for every restored job via a new local, static helper.
|
||||
- Keeping `snapshot.c`'s own "Scope decisions" header current: correcting the now-stale `allocs`/`accounts` bullet and adding `job->rusages` as a newly-documented gap, so the file's own source-level documentation and this spec's actual behavior never diverge.
|
||||
|
||||
### Out of Boundary
|
||||
- `job->rusages`, `job->priority`, queue `project_name`/`job_pick_plugin` — see Non-Goals.
|
||||
- `sim_reconcile_unaccounted_jobs()` and the wider simulation replay/decision pipeline.
|
||||
- The controller's WAL (`persist.c`, bug-1678) — a separate, non-sharing persistence mechanism already fixed independently.
|
||||
- `kill_sched_job()`'s cleanup completeness.
|
||||
- Any capnp schema change — `Job.allocs`/`SchedJob.allocs` already exist (`controller.capnp:326,341`).
|
||||
- The pre-existing resreq-parse-failure path itself (a job whose `resreq` fails to parse on load still reaches `sched_add_job()`/the picker today, independent of this feature) — this spec's `schedobj` guard tolerates that path without crashing on it, but does not close the narrow residual case where such a job is also PENDING and later reaches the allocator (see Components and Interfaces → Load-Side Job Reconstruction → Known limitation).
|
||||
|
||||
### Allowed Dependencies
|
||||
- `make_alloc`/`make_host_alloc`/`make_job_alloc`/`make_job_account` (`xcealloc.c`) — existing, unmodified.
|
||||
- `free_JobAlloc_ptr` (generated, `controller.capnp.h:5467`) — existing, NULL-safe.
|
||||
- `sched_get_user`/`sched_get_queue`/`sched_get_project` (`snapshot_load.c` already uses these for job reconstruction).
|
||||
|
||||
### Revalidation Triggers
|
||||
- Any change to `xce_job_alloc_t`/`xce_host_alloc_t`/`xce_alloc_t`'s field set (would require both `dup_job_alloc()` and the `JobAlloc`/`HostAlloc`/`ResAlloc` capnp schema to move together).
|
||||
- Any change to `core_build_schedobj()`'s field set (`core_events.c`) — `snapshot_load.c`'s local mirror must be updated in lockstep, since it is a deliberate duplication, not a shared call.
|
||||
- Any change to `xce_sched_obj_t`'s struct shape (`sched/types.h`).
|
||||
- Any change to `xce_sched_core_load_snapshot()`'s or `build_wire_job()`'s function signature.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Existing Architecture Analysis
|
||||
|
||||
**Save path** (`snapshot.c:487-509`, `xce_sched_core_save_snapshot()`): iterates `core->jobs`, calls `build_wire_job(job)` per job (`snapshot_wire.c:118-171`). `build_wire_job()` currently maps `jobid`/timestamps/`state`/queue-user-project names/`mintasks`/`maxtasks`/`resreq` (as a formatted string) onto the wire `xce_job_t`, but leaves the wire type's `allocs` field unset even when `job->allocs != NULL` (`snapshot_wire.c:167`'s own comment flags this).
|
||||
|
||||
**Load path** (`snapshot_load.c:283-340`, the job-reconstruction loop inside `xce_sched_core_load_snapshot()`): for each wire job `wj`, allocates a fresh `xce_sched_job_t`, restores `jobid`/timestamps/`state`, resolves `user`/`queue`/`project` via `sched_get_*()`, parses `resreq` from the wire's resreq string, calls `sched_add_job()`, then fires the picker's `on_job_push` hook. It never touches `job->schedobj`, `job->allocs`, or `job->accounts` — all three are left at their `calloc()`-zeroed NULL.
|
||||
|
||||
**Direct precedent for both halves of this fix, already shipped**:
|
||||
- bug-1569 (thread B) is the exact same `job->schedobj`-NULL-deref bug class, already fixed for the JOBS_PUSH path — `core_build_job()` (`core_events.c:199-266`) now unconditionally calls `core_build_schedobj()` (`core_events.c:174-197`) before the job becomes visible to the allocator, precisely because the allocator (`sched_make_allocation()` and its callees, `scheduler.c:179-267`) dereferences `job->schedobj` with no NULL check.
|
||||
- bug-1678 is the controller-side WAL analog of the `job->allocs` gap (`WAL_JOB_DISPATCH` never persisted the allocation), fixed with pure save/load glue and no schema change — direct precedent that this shape of fix is viable without touching the wire schema.
|
||||
|
||||
### Architecture Pattern & Boundary Map
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph SaveSide[Save Path]
|
||||
CoreJobs[core jobs list] --> BuildWireJob[build_wire_job]
|
||||
BuildWireJob --> WireJob[wire xce_job_t incl allocs]
|
||||
end
|
||||
|
||||
subgraph WireFormat[Snapshot Wire Format]
|
||||
WireJob --> SnapshotData[sched_snapshot_data_t]
|
||||
end
|
||||
|
||||
subgraph LoadSide[Load Path]
|
||||
SnapshotData --> LoadLoop[job reconstruction loop]
|
||||
LoadLoop --> ResreqParse[parse resreq]
|
||||
ResreqParse --> SchedobjBuild[local schedobj builder]
|
||||
ResreqParse --> AllocCopy[dup_job_alloc]
|
||||
AllocCopy --> AccountBuild[make_job_account]
|
||||
SchedobjBuild --> NewJob[restored xce_sched_job_t]
|
||||
AccountBuild --> NewJob
|
||||
NewJob --> AddJob[sched_add_job]
|
||||
AddJob --> Allocator[allocator and accounting]
|
||||
end
|
||||
|
||||
DupHelper[dup_job_alloc in xcealloc.c] --> AllocCopy
|
||||
```
|
||||
|
||||
**Architecture Integration**:
|
||||
- **Selected pattern**: pure save/load glue on both sides — no new abstraction layer, matching the existing `build_wire_*()`/reconstruction-loop pattern already used for every other snapshot entity.
|
||||
- **Domain boundaries**: `xcealloc.c` owns the deep-copy primitive (already owns the `make_*` family it composes from); `snapshot_wire.c` owns save-side field population; `snapshot_load.c` owns load-side reconstruction — unchanged ownership, just closing a gap in the existing seams.
|
||||
- **Existing patterns preserved**: no capnp code added to `snapshot.c`/`snapshot_wire.c`/`snapshot_load.c` (existing architectural boundary); job/queue/worker reconstruction in `snapshot_load.c` continues to avoid calling `core_events.c`'s static builders.
|
||||
- **New components rationale**: `dup_job_alloc()` is the one genuinely new function — no existing deep-copy helper for `xce_job_alloc_t` exists anywhere in the codebase. The schedobj builder is a local duplication, not a new abstraction (see Design Decisions in `research.md`).
|
||||
- **Steering compliance**: closes the confirmed instance motivating `persistence.md`'s "Snapshot Save/Load Field-Completeness (Scheduler)" section and `xce-audit-invariants`' scheduler dimension 4 — both already cite this spec.
|
||||
|
||||
### Technology Stack
|
||||
|
||||
| Layer | Choice / Version | Role in Feature | Notes |
|
||||
|-------|------------------|------------------|-------|
|
||||
| Scheduler core (C, gnu17) | existing `libxce` + scheduler codebase | All new/changed code | No new dependency |
|
||||
| Wire format | Cap'n Proto, existing `controller.capnp` schema | `Job.allocs`/`SchedJob.allocs @7: JobAlloc` already present | Zero schema changes |
|
||||
|
||||
## File Structure Plan
|
||||
|
||||
### Modified Files
|
||||
- `src/include/xcealloc.h` — add `extern xce_job_alloc_t* dup_job_alloc(const xce_job_alloc_t*);` declaration.
|
||||
- `src/lib/xcealloc.c` — add `dup_job_alloc()`: deep-copies a `xce_job_alloc_t` (all hosts, all per-host allocations, all shared-resource allocations) by composing `make_alloc()`/`make_host_alloc()`/`make_job_alloc()`, explicitly re-stamping each copied `xce_alloc_t`'s `from`/`duration` from the source (see `research.md` for why `make_alloc()`'s own `time(NULL)` stamp is insufficient here). 207-line file today, comfortably under the 800-line cap after this addition.
|
||||
- `src/scheduler/src/snapshot_wire.c` — `build_wire_job()` (currently 118-171): populate the wire job's `allocs` field via `dup_job_alloc(job->allocs)` when `job->allocs != NULL`, leaving it unset otherwise (identical shape to how every other optional field in this function is handled).
|
||||
- `src/scheduler/src/snapshot.c` — update the file's own "Scope decisions" header comment (currently lines 234-268): rewrite the `job->allocs`/`job->accounts` bullet, which currently states they "are NOT captured," to reflect that they now are (captured/restored by this spec); add a new bullet documenting `job->rusages` as a known, deliberate gap (no wire slot exists, would need a capnp schema addition, tracked as a separate follow-on) so a future `/xce-audit-invariants scheduler` dimension-4 pass sees it as documented rather than rediscovering it as undocumented. `job->schedobj` needs no bullet — it is fixed, not a gap.
|
||||
- `src/scheduler/src/snapshot_load.c` — `xce_sched_core_load_snapshot()`'s job-reconstruction loop (currently 283-340):
|
||||
- new static helper `snapshot_build_schedobj(xce_sched_obj_t*, xce_job_t*, xce_resreq_t*)`, a field-for-field local port of `core_build_schedobj()` (`core_events.c:174-197`), including its three tag-dispatch helpers (`primary`/`compound`/`alter`) ported the same way `core_build_job()`'s own helpers already were.
|
||||
- call site: after the existing resreq-parsing block, before `sched_add_job()` — allocate `job->schedobj`, call the new helper only when `job->resreq != NULL` (guards the pre-existing, unrelated resreq-parse-failure path without changing its behavior).
|
||||
- call site: restore `job->allocs` via `dup_job_alloc(wj->allocs)` when `wj->allocs != NULL`, then `job->accounts = make_job_account(job->allocs)` — mirrors the exact pattern `sim_reconcile_unaccounted_jobs()` (`sim_loop.c`) already established for its own narrower case.
|
||||
|
||||
### Test Files (new/extended)
|
||||
- `src/lib/testsuite/test_xcealloc.c` — new `dup_job_alloc()` unit tests (this file already exists from this session's unrelated bug-1722 fix, wired into `xcelib_test`).
|
||||
- `src/scheduler/testsuite/test_snapshot_save_load.c` — extend with cross-core round-trip tests covering `allocs`/`accounts`/`schedobj`, and the backward-compatibility case.
|
||||
- `src/scheduler/testsuite/test_sim_e2e_original_motivation.c` — extend `test_e2e_original_motivation_long_replay` (the test that already reproduces the originally-reported "info worker only reflects 2 of 22 running jobs" symptom) to assert the fix.
|
||||
|
||||
## System Flows
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Loader as xce_sched_core_load_snapshot
|
||||
participant Job as restored xce_sched_job_t
|
||||
participant Helper as snapshot_build_schedobj
|
||||
participant Dup as dup_job_alloc
|
||||
participant Acct as make_job_account
|
||||
|
||||
Loader->>Job: allocate, restore jobid/state/timestamps/user/queue/project
|
||||
Loader->>Job: parse resreq from wire string
|
||||
alt resreq parsed successfully
|
||||
Loader->>Helper: build schedobj(job->resreq, wj->submit->user/queue/project)
|
||||
Helper-->>Job: job->schedobj populated
|
||||
else resreq parse failed (pre-existing, unrelated gap)
|
||||
Loader-->>Job: job->schedobj left NULL, unchanged from today
|
||||
end
|
||||
alt wj->allocs present
|
||||
Loader->>Dup: dup_job_alloc(wj->allocs)
|
||||
Dup-->>Job: job->allocs populated (independent copy)
|
||||
Loader->>Acct: make_job_account(job->allocs)
|
||||
Acct-->>Job: job->accounts populated
|
||||
else wj->allocs absent (old snapshot or never-dispatched job)
|
||||
Loader-->>Job: job->allocs/accounts left NULL, same as today
|
||||
end
|
||||
Loader->>Loader: sched_add_job(core, job)
|
||||
Loader->>Loader: picker on_job_push hook
|
||||
```
|
||||
|
||||
Both new construction steps run before `sched_add_job()` so no code can observe a job with a NULL `schedobj` (for a job with valid `resreq`) or a stale/aliased `allocs` after this feature lands — the ordering mirrors `core_build_job()`'s own (build fully, then register).
|
||||
|
||||
## Requirements Traceability
|
||||
|
||||
| Requirement | Summary | Components | Interfaces | Flows |
|
||||
|-------------|---------|------------|------------|-------|
|
||||
| 1.1, 1.2, 1.3 | Restore a running job's allocation, reflected in accounting immediately | `dup_job_alloc`, load-side wiring | `dup_job_alloc()`, `make_job_account()` | Load sequence, "wj->allocs present" branch |
|
||||
| 1.4 | Restored allocation independent of the source core | `dup_job_alloc` | `dup_job_alloc()` | Architecture: deep-copy pattern decision |
|
||||
| 2.1, 2.2, 2.4 | Every restored job gets a usable scheduling representation | `snapshot_build_schedobj` | local static helper | Load sequence, schedobj branch |
|
||||
| 2.3 | schedobj built from data the snapshot already captures, no new wire data | `snapshot_build_schedobj` | uses `job->resreq`, `wj->submit->*` | Architecture: Existing Architecture Analysis |
|
||||
| 3.1, 3.2, 3.3, 3.4 | No regression to reconciliation, other fields, documented gaps, live path | (no new component — verified by leaving these paths untouched) | n/a | Non-Goals |
|
||||
| 3.5 | Old/absent-allocation snapshots load cleanly | load-side wiring | `wj->allocs != NULL` guard | Load sequence, "wj->allocs absent" branch |
|
||||
|
||||
## Components and Interfaces
|
||||
|
||||
| Component | Domain/Layer | Intent | Req Coverage | Key Dependencies (P0/P1) | Contracts |
|
||||
|-----------|--------------|--------|---------------|---------------------------|-----------|
|
||||
| `dup_job_alloc` | `xcealloc.c` (libxce) | Deep-copy a job's resource allocation, independent of the source | 1.1, 1.4 | `make_alloc`/`make_host_alloc`/`make_job_alloc` (P0) | Service |
|
||||
| Save-side wiring | `snapshot_wire.c` `build_wire_job()` | Populate the wire job's `allocs` field | 1.1 | `dup_job_alloc` (P0) | Service |
|
||||
| Load-side allocation restore | `snapshot_load.c` `xce_sched_core_load_snapshot()` | Restore `job->allocs`/`job->accounts` | 1.1, 1.2, 1.3, 1.4, 3.5 | `dup_job_alloc` (P0), `make_job_account` (P0) | Service |
|
||||
| `snapshot_build_schedobj` | `snapshot_load.c` (local, static) | Construct `job->schedobj` for every restored job | 2.1, 2.2, 2.3, 2.4 | `job->resreq` (P0, must already be parsed) | Service |
|
||||
|
||||
### Persistence / Scheduler Snapshot Domain
|
||||
|
||||
#### `dup_job_alloc`
|
||||
|
||||
| Field | Detail |
|
||||
|-------|--------|
|
||||
| Intent | Build a fresh, independent `xce_job_alloc_t` from a source one, for safe cross-core snapshot restoration |
|
||||
| Requirements | 1.1, 1.4 |
|
||||
|
||||
**Responsibilities & Constraints**
|
||||
- Deep-copies every host allocation and every per-host resource allocation, plus any shared-resource allocations (`nres_shared`/`allocs_shared`).
|
||||
- Never aliases any pointer from the source — every string is independently duplicated (via the existing `make_*` constructors' own `strdup()` calls), every struct independently `calloc()`'d.
|
||||
- Explicitly re-stamps each copied `xce_alloc_t`'s `from`/`duration` from the source, since `make_alloc()` itself stamps `from = time(NULL)` rather than accepting the caller's original value (see `research.md`).
|
||||
- Returns `NULL` on any allocation failure partway through; caller is responsible for treating a `NULL` return the same as "no allocation data" (Req 3.5's degrade path), not as a fatal error. A rare OOM partway through the copy may leak the partially-built pieces — this matches this file's own existing convention (`make_host_account()`/`make_job_account()` do not unwind partial state on OOM either; `xce_sched_core_save_snapshot()`'s own `arr_append()` failure handling just `break`s without freeing prior entries). Full rollback-on-OOM is a deliberately rejected stronger guarantee: OOM-during-snapshot-load is not a scenario this codebase invests in recovering from cleanly anywhere else in this file family, and a leak on that path does not violate Req 1.4's independence/no-corruption guarantee (independence and freedom from leaks are separate properties).
|
||||
|
||||
**Dependencies**
|
||||
- Outbound: `make_alloc`, `make_host_alloc`, `make_job_alloc` (`xcealloc.c`, P0 — the copy is composed entirely from these).
|
||||
|
||||
**Contracts**: Service [x] / API [ ] / Event [ ] / Batch [ ] / State [ ]
|
||||
|
||||
##### Service Interface
|
||||
```c
|
||||
xce_job_alloc_t* dup_job_alloc(const xce_job_alloc_t* src);
|
||||
```
|
||||
- Preconditions: `src` may be `NULL` (returns `NULL`); if non-NULL, `src->nhosts >= 0` and `src->allocs` has exactly `nhosts` entries (the existing wire/constructor invariant).
|
||||
- Postconditions: on success, returns a new `xce_job_alloc_t*` with no pointer shared with `src`, including `from`/`duration` on every copied `xce_alloc_t`. On failure (partial or total allocation exhaustion), returns `NULL`; a partially-built copy may leak on this path, matching this file's existing OOM-handling convention (see Responsibilities & Constraints) rather than promising full rollback.
|
||||
- Invariants: the returned value, once passed to `free_JobAlloc_ptr()`, cleans up exactly as any other `xce_job_alloc_t` — no new free path is introduced.
|
||||
|
||||
**Implementation Notes**
|
||||
- Integration: called from both `build_wire_job()` (save side, copying a live job's allocation into the wire struct) and `xce_sched_core_load_snapshot()` (load side, copying the wire struct's allocation into the restored job) — same function, both directions, since both need the identical independence guarantee.
|
||||
- Validation: covered by the persistence-completeness test matrix below (multi-host, multi-resource, shared-resource, and independence-after-mutation cases).
|
||||
- Risks: none beyond the `from`/`duration` fidelity nuance already resolved in `research.md`.
|
||||
|
||||
#### Load-Side Job Reconstruction (`snapshot_load.c`)
|
||||
|
||||
| Field | Detail |
|
||||
|-------|--------|
|
||||
| Intent | Restore `job->allocs`/`job->accounts`/`job->schedobj` as part of the existing per-job reconstruction loop |
|
||||
| Requirements | 1.1, 1.2, 1.3, 2.1, 2.2, 2.3, 2.4, 3.5 |
|
||||
|
||||
**Responsibilities & Constraints**
|
||||
- Must run before `sched_add_job()` so no other code ever observes a partially-reconstructed job (see System Flows ordering).
|
||||
- Must guard `schedobj` construction on `job->resreq != NULL` — the pre-existing resreq-parse-failure path is unchanged, not fixed, by this feature.
|
||||
- Must treat `wj->allocs == NULL` (old snapshot file, or a job that was never dispatched) as the normal, non-error case — no WARN, no behavior change from today.
|
||||
- **Known limitation, explicitly out of boundary (not closed by this spec)**: a PENDING job whose `resreq` fails to parse on load (pre-existing gap — e.g. the target core's `resmgr` lacks a resource type the source core had, plausible specifically in the cross-core scenarios Req 1.4 exercises) still reaches `sched_add_job()`/the picker with `job->schedobj == NULL`, since this component's guard skips schedobj construction whenever `resreq` is NULL. If that job is later selected by the picker, it hits the same bug-1569-class NULL-deref this feature otherwise closes. This is not a new regression — every restored job hit this before this feature, unconditionally — but it means Requirement 2.2 is closed for the general case (successful resreq parse) and not for this narrow, already-broken sub-case. Fixing it requires changing the pre-existing resreq-parse-failure path itself (e.g. not registering such a job with the picker at all), which is a distinct, unrelated defect and explicitly out of this spec's boundary — tracked here rather than silently assumed away.
|
||||
|
||||
**Dependencies**
|
||||
- Inbound: `xce_sched_core_load_snapshot()`'s existing job loop (P0 — this is an extension of it, not a new call site).
|
||||
- Outbound: `dup_job_alloc` (P0), `make_job_account` (P0), the new `snapshot_build_schedobj` local helper (P0).
|
||||
|
||||
**Contracts**: Service [x] / API [ ] / Event [ ] / Batch [ ] / State [ ]
|
||||
|
||||
##### Service Interface
|
||||
```c
|
||||
static void snapshot_build_schedobj(xce_sched_obj_t* obj, xce_job_t* wj,
|
||||
xce_resreq_t* resreq);
|
||||
```
|
||||
- Preconditions: `obj` freshly `calloc()`'d (zeroed); `resreq` already successfully parsed; `wj` the wire job carrying `wj->submit->user/queue/project_name`.
|
||||
- Postconditions: `obj->tag`/`obj->u.{primary,compound,alter}` set per `resreq->tag`; `obj->user`/`queue`/`project` are fresh string copies; `obj->glimits`/`hlimits` initialized empty; `obj->configver = 0`; `obj->alloc`/`obj->reason` left NULL (correct: both are per-scheduling-round scratch state, never persisted job state — see `research.md`).
|
||||
- Invariants: field-for-field identical output to `core_build_schedobj()` given the same inputs — any future divergence is a bug, not an accepted difference (Revalidation Trigger above).
|
||||
|
||||
**Implementation Notes**
|
||||
- Integration: called once per restored job, immediately after resreq parsing succeeds.
|
||||
- Validation: covered by the "restored PENDING job reaches the allocator without crashing" test (Testing Strategy below) — the direct regression test for bug-1569's failure shape.
|
||||
- Risks: duplicated logic could drift from `core_build_schedobj()` over time; mitigated by the Revalidation Trigger and `/xce-audit-invariants scheduler` dimension 4.
|
||||
|
||||
## Data Models
|
||||
|
||||
### Snapshot Entity Field-Completeness (Persistence Review)
|
||||
|
||||
Per-field disposition for every field this feature touches, per `persistence.md`'s "Snapshot Save/Load Field-Completeness (Scheduler)" standing invariant:
|
||||
|
||||
| Entity | Field | Captured on save? | Restored on load? | Disposition |
|
||||
|--------|-------|--------------------|--------------------|-------------|
|
||||
| `xce_sched_job_t` | `allocs` | Yes (this feature, `dup_job_alloc` via `build_wire_job`) | Yes (this feature) | Fixed by this spec |
|
||||
| `xce_sched_job_t` | `accounts` | N/A (derived, never itself serialized) | Yes (this feature, via `make_job_account(job->allocs)`) | Fixed by this spec |
|
||||
| `xce_sched_job_t` | `schedobj` | N/A (derived from `resreq`/user/queue/project, never itself serialized) | Yes (this feature, `snapshot_build_schedobj`) | Fixed by this spec |
|
||||
| `xce_alloc_t` | `from`, `duration` | Yes (wire round-trip, pre-existing) | Yes (this feature explicitly re-stamps both after `make_alloc()`) | Fixed by this spec (functionally inert today, but field-complete) |
|
||||
| `xce_sched_job_t` | `priority` | No | No | Pre-existing, documented gap (`snapshot.c` Scope Decisions) — unchanged |
|
||||
| `xce_sched_job_t` | `rusages` | No (no wire slot exists) | No | Pre-existing gap, undocumented at the source until this spec — explicitly out of scope, needs a schema addition (see Non-Goals); this spec adds it to `snapshot.c`'s own "Scope decisions" header so it reads as documented, not rediscovered, on the next audit (see File Structure Plan) |
|
||||
| `xce_sched_queue_t` | `project_name`, `job_pick_plugin` | No | No | Pre-existing, documented gap — unchanged |
|
||||
|
||||
### Domain Model
|
||||
No new domain concepts — `xce_job_alloc_t`/`xce_host_alloc_t`/`xce_alloc_t`/`xce_job_account_t`/`xce_sched_obj_t` are all pre-existing structs (`sched/types.h`, `xce/types.h`). This feature restores existing relationships (job → allocation → per-host/per-resource accounts; job → scheduling representation) across a save/load boundary, it does not introduce new ones.
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Error Strategy
|
||||
This is an internal, in-process reconstruction path with no external I/O of its own — errors are allocation failures (OOM) or pre-existing malformed-input states, both handled by degrading to today's behavior rather than crashing or aborting the load.
|
||||
|
||||
### Error Categories and Responses
|
||||
- **Allocation failure inside `dup_job_alloc()`**: returns `NULL`; caller leaves `job->allocs`/`job->accounts` at NULL — identical to the "old snapshot, no allocation data" case (Req 3.5). Not treated as a load failure.
|
||||
- **`job->resreq == NULL` at the schedobj-construction call site** (pre-existing, unrelated resreq-parse-failure path): `snapshot_build_schedobj()` is not called; `job->schedobj` stays NULL, identical to today's behavior for this already-existing edge case. Not a new regression — this feature does not change what happens to a job whose resreq fails to parse, only what happens to a job whose resreq parses successfully. **Known limitation**: if such a job is PENDING and later reaches the allocator, it still hits the bug-1569-class crash this feature otherwise closes — see Components and Interfaces → Load-Side Job Reconstruction for why fixing that sub-case is out of this spec's boundary.
|
||||
- **`wj->allocs == NULL`** (old snapshot file, or a job that was pending/never dispatched at save time): no restoration attempted, no WARN — this is the normal case, not an error (Req 3.5).
|
||||
|
||||
### Monitoring
|
||||
No new logging beyond what's necessary to debug a `dup_job_alloc()` allocation failure (a `WARN`, consistent with how every other optional-field restoration failure in `xce_sched_core_load_snapshot()` is already logged, e.g. the existing worker-resval-restoration `WARN` at `snapshot_load.c:200-204`).
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- **Unit — `dup_job_alloc()` (`src/lib/testsuite/test_xcealloc.c`)**: single-host/single-resource copy; multi-host copy; multi-resource-per-host copy; shared-resource (`nres_shared`/`allocs_shared`) copy; independence proof (mutate/free the source after copying, confirm the copy is unaffected — the direct test of the Req 1.4 guarantee); `from`/`duration` fidelity (copy preserves the source's exact original values, not `make_alloc()`'s own `time(NULL)` stamp).
|
||||
- **Integration — cross-core snapshot round-trip (`test_snapshot_save_load.c`)**: save a core with a mix of RUNNING jobs (multi-host, multi-resource, shared-resource allocations) and PENDING jobs, load into a second, independent core; assert `job->allocs`/`job->accounts` match the source for RUNNING jobs, `job->schedobj` is non-NULL for every job, and freeing either core's job list does not corrupt or crash the other (Req 1.4). Extend with a snapshot containing no allocation data at all (the natural NULL case, not an archived legacy file — see Requirement 3's backward-compat criterion) and confirm it loads cleanly with `used=0`.
|
||||
- **Regression — restored job reaches the allocator (`test_snapshot_save_load.c` or a new case in the same file)**: load a snapshot containing a PENDING job, run it through `sched_make_allocation()` (or the picker→allocator path), assert no crash/UB — the direct regression test for the bug-1569-class failure this feature closes (Req 2.2).
|
||||
- **E2E — sim-mode "used" accounting (`test_sim_e2e_original_motivation.c`, extending `test_e2e_original_motivation_long_replay`)**: reproduce the originally-reported scenario (a snapshot with multiple running jobs, replayed forward) and assert `info worker`'s "used" values correctly reflect every running job's contribution immediately after load, not just the ones whose dispatch also appears later in replay.
|
||||
- **No regression — existing reconciliation (`test_sim_reconcile_unaccounted.c`, `test_sim_snapshot_reconcile_gap.c`)**: run unchanged; both must continue passing exactly as today, confirming `sim_reconcile_unaccounted_jobs()`'s complementary case is untouched (Req 3.1).
|
||||
|
||||
## Constraints (Invariant Cross-Check)
|
||||
|
||||
- **No capnp code in `snapshot.c`/`snapshot_wire.c`/`snapshot_load.c`** — preserved; `dup_job_alloc()` is composed entirely from existing plain-C constructors, no capnp codec call added to these three files.
|
||||
- **No aliasing across independently-alive cores** — preserved; `dup_job_alloc()` is a genuine deep copy, verified by the independence unit test above.
|
||||
- **`session_active == 0` (`-EBUSY`) guard on both save and load** — unchanged; this feature adds no new mutation path outside the existing per-job reconstruction loop, which already runs entirely under this guard.
|
||||
|
||||
**`xce-audit-invariants` scheduler dimension walkthrough** (all four dimensions checked against this design, per the standing dimension list in the `xce-audit-invariants` skill):
|
||||
|
||||
1. **Driver state-equivalence** (`scheduler-driver-equivalence.md`) — not implicated. All call sites of `xce_sched_core_load_snapshot()` are in `sim.c`; normal-mode `driver.c` never loads a snapshot, so there is no normal-mode/sim-mode divergence to introduce. This fix arguably *strengthens* dimension 1: restored jobs now get their accounting via the same `make_job_account()` call the live dispatch path already uses, rather than diverging from it as they do today (no `accounts` at all).
|
||||
2. **Core module I/O isolation** (`core.c`/`core_events.c`/`core_session.c`/`core_clock.c`, mechanically enforced by `test_core_isolation.c`) — not implicated. Every changed/new file (`xcealloc.c`, `snapshot_wire.c`, `snapshot_load.c`, `snapshot.c`) is outside the isolated core-family set; no I/O is added to any of them, and none of them are added to or removed from `sched_core_isolation_lib`'s membership.
|
||||
3. **Picker→allocator `jbuf` capacity-1 rendezvous** (`scheduler/CLAUDE.md`) — not implicated. Snapshot load runs before/outside any scheduling session round; nothing in this design touches `scheduler.c`'s handoff buffer, raises its capacity, or adds a second producer/consumer.
|
||||
4. **Field-complete snapshot save/load parity** (`persistence.md`'s "Snapshot Save/Load Field-Completeness (Scheduler)"; this spec is dimension 4's own confirmed instance) — directly addressed. The Data Models field table above enumerates every field this spec touches or is aware of, correctly separating documented from undocumented gaps; `snapshot.c`'s own header is kept current (File Structure Plan) so the next audit reads `rusages` as documented rather than rediscovering it. A full re-audit of untouched entities (worker/user/project) is deliberately not duplicated here — that is what running `/xce-audit-invariants scheduler` (or at minimum re-checking dimension 4) as final task validation is for, per the brief's own Constraints section.
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
# Requirements Document
|
||||
|
||||
## Project Description (Input)
|
||||
Engineers using `xce_scheduler -i -s <replay>` to run simulation-mode capacity/contention analysis from a live snapshot + replay -- the common investigation workflow -- currently get silently wrong results for any job that was already RUNNING as of the snapshot's save point. `xce_sched_core_load_snapshot()` restores such a job's `state` correctly (so it shows as RUN), but never restores `job->allocs`/`job->accounts` (`build_wire_job()` in `snapshot_wire.c` carries an explicit `/* job->allocs/accounts: not captured */` comment) -- this was a deliberate, documented scope decision from `scheduler-core` task 8.1, made because no deep-copy helper for `xce_job_alloc_t` existed and aliasing it across two independently-alive cores would create a double-free hazard. As a result, `sched_account_jobs()` (which recomputes every worker's `resInUse` each round by walking jobs with `JOB_IS_RUN(state) && accounts != NULL`) never charges these jobs' resources, so `info worker`'s "used" values permanently undercount snapshot-restored running jobs -- reported live as: after loading a snapshot with 22 running jobs and replaying 79 sessions forward, `info summary` correctly counts 22 running jobs but `info worker`'s "used" resource values only ever reflect 2 of them. This should change so that a job already running as of a snapshot's save point survives the save/load round-trip with its full allocation intact -- `job->allocs` populated via a new deep-copy helper (composed from the existing `make_alloc`/`make_host_alloc`/`make_job_alloc` constructors, paired with `free_JobAlloc_ptr` for cleanup, never aliasing the source snapshot's own copy), `job->accounts` reconstructed via the existing `make_job_account()`, and worker/shared resource "used" accounting correct from the moment the snapshot loads -- while a snapshot file saved before this change still loads cleanly with today's `used=0` behavior for its jobs (clean degrade, not a regression). The wire type already has an `allocs` field mirroring the C struct with no schema change needed, so this is confined to save-side wiring in `build_wire_job()` (`snapshot_wire.c`) and load-side wiring in `xce_sched_core_load_snapshot()` (`snapshot_load.c`), following the exact reconstruction pattern `sim_reconcile_unaccounted_jobs()` (`scheduler-sim-eager-reconciliation`) already established for the complementary case (a dispatch decision discoverable later in replay) -- this spec closes the other half, where the dispatch decision predates the snapshot and can never appear in the subsequent replay stream.
|
||||
|
||||
A second, more severe defect in the same load path was found and folded into this spec's scope during a cross-component persistence survey: every job restored from a snapshot -- PENDING or RUNNING -- gets a NULL `job->schedobj`, a field the live allocator (`sched_make_allocation()` and its callees in `scheduler.c`, plus plugins like `limit.c`) dereferences unconditionally, the same failure shape as the already-fixed bug-1569 SIGSEGV, reproducible the moment a restored PENDING job reaches the allocator. Unlike `allocs`, this needs no new captured data at all -- `core_build_schedobj()` (the live construction path, `core_events.c`) builds `schedobj` purely from the job's `user`/`queue`/`project` names and its `resreq` (already parsed from the resreq string this spec's load path already restores), plus freshly-initialized, session-scoped empty bookkeeping (`glimits`/`hlimits`) and reset defaults (`configver=0`, a NULL `alloc`/`reason` that are themselves confirmed to be pure per-scheduling-round scratch state, never persisted job state, in the live path too). The fix is to locally replicate `core_build_schedobj()`'s exact field-for-field logic inside `xce_sched_core_load_snapshot()`'s job-reconstruction loop for every restored job -- `core_build_schedobj()` itself is `static` to `core_events.c` and cannot be called directly, consistent with the existing convention that `snapshot_load.c` never shares helpers with `core_events.c`'s builders.
|
||||
|
||||
## Boundary Context
|
||||
- **In scope**: capturing and restoring a running job's resource allocation across a snapshot save/load cycle, so worker and shared resource utilization is correct immediately after loading a snapshot; reconstructing every restored job's internal scheduling representation (regardless of whether the job is pending or running) so it can be evaluated by the scheduler's normal allocation logic without failure; loading snapshot files created before this feature existed without error or behavior change for the jobs they contain.
|
||||
- **Out of scope**: restoring a job's priority, or a queue's owning-project/picker-plugin-override fields (pre-existing, separate, already-documented snapshot gaps, unchanged by this feature); capturing a job's latest controller-reported resource-usage report in a snapshot (a distinct, currently-uncaptured piece of job state that would require the snapshot format itself to gain new capacity, unlike the fields this feature restores) -- tracked as a separate future feature; any change to how simulation-mode reconciles a job's allocation from a dispatch decision appearing later in a replay stream (an existing, separate mechanism this feature does not modify); any change to live, non-snapshot job dispatch or resource-accounting behavior.
|
||||
- **Adjacent expectations**: this feature is a direct extension of the scheduler's existing snapshot save/load capability -- it relies on that capability's existing file format and loading behavior remaining otherwise unchanged. It complements the existing replay-based reconciliation mechanism (which recovers a job's allocation when its dispatch decision appears later in a replay): together, a job's allocation should be recoverable regardless of whether its dispatch happened before the snapshot was taken or is discoverable later in replay.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement 1: Resource allocation restoration for jobs already running at snapshot time
|
||||
**Objective:** As an operator running simulation-mode capacity/contention analysis from a snapshot, I want a job that was already running when the snapshot was taken to keep its resource allocation after the snapshot loads, so that worker and shared resource utilization I inspect is accurate from the moment the snapshot loads, not only for jobs whose dispatch happens to also appear later in replay.
|
||||
|
||||
#### Acceptance Criteria
|
||||
1. When the scheduler loads a snapshot containing a job in a running state that has recorded resource-allocation data, the scheduler shall restore that job's resource allocation as part of loading the snapshot.
|
||||
2. When a running job's resource allocation is restored while loading a snapshot, the scheduler shall include that job's resource consumption in its assigned worker's and any shared resource's in-use totals immediately, without requiring any further replay activity.
|
||||
3. The scheduler shall report a restored running job's resource-in-use contribution identically to how it reports the same contribution for a job whose allocation was established through normal dispatch.
|
||||
4. While a snapshot has been loaded into a scheduler instance, the scheduler shall treat the restored resource-allocation data as independent of the snapshot-producing instance's own data, such that later activity or shutdown in either instance does not corrupt or crash the other.
|
||||
|
||||
### Requirement 2: Valid scheduling representation for every restored job
|
||||
**Objective:** As an operator running simulation-mode replay from a snapshot, I want every job loaded from a snapshot -- pending or running -- to be fully usable by the scheduler's normal allocation logic, so that replay does not crash or misbehave when the scheduler evaluates a restored pending job for placement.
|
||||
|
||||
#### Acceptance Criteria
|
||||
1. When the scheduler loads any job from a snapshot, the scheduler shall construct that job's internal scheduling representation so the job is usable by the scheduler's allocation logic.
|
||||
2. When a pending job restored from a snapshot is evaluated for resource allocation, the scheduler shall process it without crashing or producing undefined behavior.
|
||||
3. The scheduler shall reconstruct a restored job's scheduling representation entirely from information the snapshot format already captures (its queue, user, project, and resource request), without requiring any additional information to be present in the snapshot file.
|
||||
4. The scheduler shall treat a job restored from a snapshot the same as a normally-submitted job with the same queue, user, project, and resource request, for the purposes of allocation evaluation.
|
||||
|
||||
### Requirement 3: No regression to existing snapshot, reconciliation, and accounting behavior
|
||||
**Objective:** As a maintainer relying on the scheduler's existing snapshot and reconciliation guarantees, I want this feature to leave them intact, so existing workflows, tests, and documented scope decisions continue to hold.
|
||||
|
||||
#### Acceptance Criteria
|
||||
1. The scheduler shall continue to correctly reconcile a job's resource allocation through the existing replay-based reconciliation mechanism for any job whose dispatch decision appears within the replay stream.
|
||||
2. The scheduler shall continue to save and restore every other previously-supported job, queue, worker, and session field in a snapshot exactly as before, with no change in behavior.
|
||||
3. The scheduler shall continue to not restore a job's priority or a queue's owning-project/picker-plugin-override fields, consistent with the scheduler's existing, separately-documented scope decisions.
|
||||
4. While loading a snapshot, the scheduler shall not alter its live, non-snapshot job dispatch or resource-accounting behavior.
|
||||
5. While a snapshot does not contain resource-allocation data for a given job (for example, one saved before this feature existed), the scheduler shall load it successfully and report that job's resource consumption as it does today (no contribution), rather than treating the absence as an error condition.
|
||||
73
.kiro/specs/scheduler-snapshot-job-allocation/research.md
Normal file
73
.kiro/specs/scheduler-snapshot-job-allocation/research.md
Normal file
|
|
@ -0,0 +1,73 @@
|
|||
## Summary
|
||||
- **Feature**: `scheduler-snapshot-job-allocation`
|
||||
- **Discovery Scope**: Extension (existing scheduler snapshot save/load subsystem, `scheduler-core` task 8.1)
|
||||
- **Key Findings**:
|
||||
- `core_build_schedobj()` (`core_events.c:174-197`) is a pure function of data `xce_sched_core_load_snapshot()`'s job loop already has in scope by the time this feature's insertion point runs — no new captured/wire data is needed for `job->schedobj`, only a local, mechanical port of the same 3-field construction (mirrors `scheduler-core`'s own no-shared-helper convention already used for job/queue/worker reconstruction).
|
||||
- This exact bug class (allocator dereferences `job->schedobj` unconditionally, no NULL check) already happened once and was fixed: bug-1569 thread B, `core_events.c:86-109`'s own header comment documents it verbatim as the reason `core_build_schedobj()` exists at all. `snapshot_load.c` was never updated to carry the same fix — it is the one remaining code path with the identical gap.
|
||||
- `xce_sched_core_load_snapshot()`'s `in` parameter is `const sched_snapshot_data_t*` (`snapshot_load.c:59`) and, per `sim.c`'s 4 call sites, the same decoded snapshot can plausibly be loaded into more than one core. This rules out the ownership-transfer/MOVE pattern `sim_reconcile_unaccounted_jobs()` uses (`sim_loop.c:635-637`, safe there only because its source is a single-use, about-to-be-freed replay decision) — the allocation restore here must be a genuine deep copy.
|
||||
- `xce_alloc_t.from`/`.duration` (`xcealloc.c:30-31`) are write-only today — grepped the entire tree, confirmed the only reads are wire round-trip test assertions (`test_e2e_capnp.c`), no live scheduling/accounting logic consumes either field. `make_alloc()` itself stamps `from = time(NULL)` rather than accepting it as a parameter, so a naive deep-copy built by calling `make_alloc()` and stopping there would silently reset every restored allocation's `from` to snapshot-load time. Functionally inert today, but a genuine per-field fidelity gap the persistence review gate would flag — cheap to close (one extra assignment per copied `xce_alloc_t`), so the design closes it rather than accepting the drift.
|
||||
- `kill_sched_job()` (`schedobj.c:17-28`), the only per-item destructor registered for `core->jobs` (`core.c:93`), frees `job->rusages` only — it does not free `job->resreq`, `job->schedobj` (no `reset_sched_obj()` call), `job->allocs`, or `job->accounts`. This is a pre-existing gap affecting every job today (live-dispatched or restored), not introduced by this feature. Out of scope: fixing job cleanup is a job-lifecycle concern independent of snapshot restoration, would affect all jobs not just restored ones, and Requirement 2.4 ("treat a restored job the same as a normally-submitted job") is satisfied by structural parity, leak included, not by this spec silently fixing an unrelated defect.
|
||||
- capnp schema confirms both `Job.allocs @7: JobAlloc` and `SchedJob.allocs @7: JobAlloc` exist with matching field numbers (`src/schema/controller.capnp:326,341`), and `JobAlloc`/`HostAlloc`/`ResAlloc` map directly onto `xce_job_alloc_t`/`xce_host_alloc_t`/`xce_alloc_t` with no translation layer — zero schema changes required, confirming the brief's viability check.
|
||||
- `xce_sched_core_load_snapshot()`'s existing job-reconstruction loop (`snapshot_load.c:288-340`) already lets a job with a resreq-parse failure proceed to `sched_add_job()` with `job->resreq == NULL` (pre-existing, unrelated gap, not fixed here). Both new construction steps (schedobj, allocs) must tolerate this without crashing — schedobj construction is naturally skipped when `job->resreq == NULL` (mirrors `core_build_schedobj()`'s own precondition), and allocs restoration is independent of `resreq` entirely.
|
||||
|
||||
## Research Log
|
||||
|
||||
### Exact insertion point and construction inputs for `job->schedobj`
|
||||
- **Context**: Requirement 2 needs every restored job to get a valid `job->schedobj`. Needed to confirm it's genuinely derivable with no new captured data, and find the precise insertion point.
|
||||
- **Sources Consulted**: `core_events.c:174-266` (`core_build_schedobj`, `core_build_primary/compound/alter_schedobj`, `core_build_job`), `schedobj.c:17-211` (`make_sched_obj`, `kill_sched_job`, sibling live-path implementation), `snapshot_load.c:58-371` (full `xce_sched_core_load_snapshot` body), `sched/types.h:30-116` (`xce_sched_obj_t`, `xce_sched_job_t` struct shapes).
|
||||
- **Findings**: `core_build_schedobj(obj, j, resreq)` needs only `j->submit->user/queue/project_name` (raw wire strings — `snapshot_load.c`'s loop variable `wj` is the exact same `xce_job_t*` type) and `resreq` (`job->resreq`, already parsed by the time the loop reaches line 332). `glimits`/`hlimits` start empty; `configver`/`alloc`/`reason` start at zero/NULL in the live path too (confirmed `obj->alloc`'s only writers, `scheduler.c:194,228,257`, are pure per-round allocator scratch state, never persisted job state).
|
||||
- **Implications**: The fix is a local, static helper in `snapshot_load.c` replicating `core_build_schedobj()`'s exact 3-part body, called right after the existing resreq-parsing block (after `snapshot_load.c:332`) and before `sched_add_job()` (line 334) — guarded on `job->resreq != NULL`.
|
||||
|
||||
### Deep-copy vs. move semantics for `job->allocs`
|
||||
- **Context**: `sim_reconcile_unaccounted_jobs()` already reconstructs `job->allocs`/`job->accounts` from a replay decision using MOVE_PTR ownership transfer, not a deep copy. Needed to confirm whether the same pattern applies here.
|
||||
- **Sources Consulted**: `sim_loop.c:574-716` (`sim_reconcile_unaccounted_jobs`, full body incl. its ownership-hazard comment), `snapshot_load.c:58-60` (`xce_sched_core_load_snapshot` signature — `const sched_snapshot_data_t* in`), caller count from CodeGraph (`sim.c`, 4 call sites).
|
||||
- **Findings**: `sim_reconcile_unaccounted_jobs()`'s source (`entry->body.dispatch->alloc`) is a single-use, about-to-be-freed replay decision — MOVE is safe there specifically because nothing else will ever reference it again. `xce_sched_core_load_snapshot()`'s source is a `const`-qualified snapshot struct with multiple call sites in `sim.c`, i.e., plausibly loaded into more than one core or inspected again after load.
|
||||
- **Implications**: Confirms the brief's original approach — a genuine deep copy (composed from `make_alloc`/`make_host_alloc`/`make_job_alloc`), never an alias/move, paired with `free_JobAlloc_ptr` for cleanup.
|
||||
|
||||
### Field fidelity of the deep copy: `xce_alloc_t.from`/`.duration`
|
||||
- **Context**: `make_alloc()` (the constructor the deep-copy composes from) sets `->from = time(NULL)` internally rather than accepting the source's original value. Needed to determine whether this matters.
|
||||
- **Sources Consulted**: `grep -rn "\.from\b\|->from\b"` / `"\.duration\b\|->duration\b"` across `src/` for `xce_alloc_t` usage.
|
||||
- **Findings**: Both fields are write-only in live code (`xcealloc.c`'s own constructor) — the only reads are capnp wire round-trip test assertions (`test_e2e_capnp.c:794,896-897`), never live scheduling/accounting logic.
|
||||
- **Implications**: Functionally inert today, but a real field-fidelity gap if left unaddressed — the deep-copy explicitly re-sets `->from`/`->duration` to the source's original values after calling `make_alloc()`, rather than accepting `make_alloc()`'s own `time(NULL)` stamp. Cheap, and keeps the copy a genuine field-complete mirror rather than a "doesn't matter yet" shortcut.
|
||||
|
||||
## Architecture Pattern Evaluation
|
||||
|
||||
| Option | Description | Strengths | Risks / Limitations | Notes |
|
||||
|--------|-------------|-----------|---------------------|-------|
|
||||
| Deep-copy via existing `make_*` constructors (selected) | Compose `dup_job_alloc()` from `make_alloc`/`make_host_alloc`/`make_job_alloc`, explicitly re-stamping `from`/`duration` | Reuses proven, tested constructors; correct independence guarantee; no capnp code added to snapshot files | One new function + explicit field re-stamp | Matches bug-1678's controller-side precedent (pure save/load glue, no schema change) |
|
||||
| Alias/share the source snapshot's `xce_job_alloc_t*` directly | Just assign `job->allocs = wj->allocs` | Zero new code | Double-free the moment either core's normal job-lifecycle path frees it, or a `const`-violation if `wj` is truly shared across cores | Rejected — this is the exact hazard `scheduler-core`'s original scope decision flagged |
|
||||
| Reuse `sim_reconcile_unaccounted_jobs()`'s MOVE_PTR pattern | Move the wire struct's pointer into the job, null the source | Less code than a full deep copy | Requires `wj` to be non-const and single-use; `in` is `const` and reusable | Rejected — wrong ownership model for this call site (see Research Log) |
|
||||
| Call `core_build_schedobj()` directly from `snapshot_load.c` | Reuse the live construction function instead of duplicating it | No duplicated logic | `core_build_schedobj()` is `static` to `core_events.c`; calling it would violate the existing, deliberate no-shared-helper boundary between `core_events.c` and `snapshot_load.c` | Rejected — local replication chosen instead, consistent with existing convention |
|
||||
|
||||
## Design Decisions
|
||||
|
||||
### Decision: Local, static `schedobj` builder in `snapshot_load.c` rather than exporting `core_build_schedobj()`
|
||||
- **Context**: `job->schedobj` needs identical construction logic in two files that must not share helpers.
|
||||
- **Alternatives Considered**:
|
||||
1. Export `core_build_schedobj()` from `core_events.c` and call it from `snapshot_load.c`.
|
||||
2. Duplicate the construction logic locally in `snapshot_load.c` as a new static helper.
|
||||
- **Selected Approach**: Option 2 — a new static helper in `snapshot_load.c`, field-for-field identical to `core_build_schedobj()`.
|
||||
- **Rationale**: `scheduler-core`'s existing, documented convention (cited in `snapshot.c`'s own header) deliberately avoids `snapshot_load.c` depending on `core_events.c`'s static, unexported builders — the same reasoning that already applies to job/queue/worker reconstruction in this file.
|
||||
- **Trade-offs**: Small duplication (roughly 25 lines) vs. preserving an existing, intentional architectural boundary. Duplication accepted.
|
||||
- **Follow-up**: If `core_build_schedobj()`'s field set ever changes, `snapshot_load.c`'s mirror must be updated in lockstep (recorded as a Revalidation Trigger in `design.md`).
|
||||
|
||||
### Decision: Genuine deep copy for `job->allocs`, with explicit `from`/`duration` re-stamping
|
||||
- **Context**: See Research Log entries above.
|
||||
- **Alternatives Considered**: alias/share; MOVE_PTR ownership transfer (both rejected, see Architecture Pattern Evaluation table).
|
||||
- **Selected Approach**: New `dup_job_alloc(const xce_job_alloc_t*)` in `xcealloc.c`, composed from `make_alloc`/`make_host_alloc`/`make_job_alloc`, explicitly copying `from`/`duration` after each `make_alloc()` call.
|
||||
- **Rationale**: Only option that satisfies the independence requirement (Req 1.4) without a capnp-layer round-trip inside the snapshot files (an existing architectural boundary).
|
||||
- **Trade-offs**: None significant — this is the approach the brief already converged on; this session's discovery only refined the `from`/`duration` fidelity detail.
|
||||
- **Follow-up**: none.
|
||||
|
||||
## Risks & Mitigations
|
||||
- **Risk**: `snapshot_load.c`'s local schedobj-builder drifts out of sync with `core_build_schedobj()` if the latter gains a new field later. — **Mitigation**: recorded as a Revalidation Trigger in `design.md`; `/xce-audit-invariants scheduler` dimension 4 is the standing check that would catch this class of drift.
|
||||
- **Risk**: A job whose resreq failed to parse (pre-existing, unrelated `snapshot_load.c` gap) could crash schedobj construction if not guarded. — **Mitigation**: schedobj construction is explicitly gated on `job->resreq != NULL`, matching `core_build_schedobj()`'s own implicit precondition; the pre-existing resreq-failure behavior itself is unchanged (out of scope).
|
||||
- **Risk**: `kill_sched_job()` not freeing `resreq`/`schedobj`/`allocs`/`accounts` is a pre-existing leak this feature's newly-populated fields will also flow through. — **Mitigation**: none needed for this spec — Requirement 2.4 asks for structural parity with live jobs, which this preserves (leak included); fixing job cleanup is an unrelated, broader job-lifecycle concern, explicitly out of boundary.
|
||||
- **Risk** (found during `/kiro-validate-design` review): a PENDING job whose `resreq` fails to parse on load (pre-existing gap, plausible via a resmgr mismatch between the saving and loading cores) still reaches `sched_add_job()`/the picker with `job->schedobj == NULL`, since `snapshot_build_schedobj()`'s guard skips construction whenever `resreq` is NULL — if later selected by the picker, it hits the same bug-1569-class crash. Not a new regression (every restored job hit this unconditionally before this feature), but Requirement 2.2 is closed only for the general case, not this narrow sub-case. — **Mitigation**: documented explicitly in `design.md` (Components and Interfaces → Load-Side Job Reconstruction, and Out of Boundary) as a known, unclosed limitation rather than silently implied as covered; the actual fix belongs to the pre-existing resreq-parse-failure path itself, a distinct, unrelated defect.
|
||||
- **Risk** (found during `/kiro-validate-design` review): `dup_job_alloc()`'s originally-drafted postcondition ("no leak on the failure path") was stricter than this file's established OOM-handling convention (`make_host_account()`/`make_job_account()` don't unwind partial state either). — **Mitigation**: relaxed the postcondition in `design.md` to explicitly allow a leak on the rare OOM path, matching the file's convention; confirmed this does not violate Req 1.4 (independence and leak-freedom are separate properties).
|
||||
|
||||
## References
|
||||
- `.wolf/buglog.json` bug-1569 — the exact prior instance of this bug class (`job->schedobj` unconditionally dereferenced by the allocator with no NULL check), already fixed once for the JOBS_PUSH path.
|
||||
- `.wolf/buglog.json` bug-1678 — controller-side precedent (WAL_JOB_DISPATCH never persisting `job->allocs`), same shape, sibling persistence mechanism, already fixed with pure save/load glue and no schema change.
|
||||
- `.kiro/steering/persistence.md` — "Snapshot Save/Load Field-Completeness (Scheduler)" section, the standing invariant this spec is the confirmed instance of.
|
||||
- `src/schema/controller.capnp:326,341,438,443,448` — `Job.allocs`/`SchedJob.allocs`/`JobAlloc`/`HostAlloc`/`ResAlloc` schema, confirming zero schema changes needed.
|
||||
23
.kiro/specs/scheduler-snapshot-job-allocation/spec.json
Normal file
23
.kiro/specs/scheduler-snapshot-job-allocation/spec.json
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
{
|
||||
"feature_name": "scheduler-snapshot-job-allocation",
|
||||
"created_at": "2026-08-27T09:07:31Z",
|
||||
"updated_at": "2026-08-27T10:48:20Z",
|
||||
"language": "en",
|
||||
"phase": "tasks-generated",
|
||||
"approvals": {
|
||||
"requirements": {
|
||||
"generated": true,
|
||||
"approved": true
|
||||
},
|
||||
"design": {
|
||||
"generated": true,
|
||||
"approved": true
|
||||
},
|
||||
"tasks": {
|
||||
"generated": true,
|
||||
"approved": true
|
||||
}
|
||||
},
|
||||
"ready_for_implementation": true
|
||||
}
|
||||
|
||||
92
.kiro/specs/xce-test-cli-ux/requirements.md
Normal file
92
.kiro/specs/xce-test-cli-ux/requirements.md
Normal file
|
|
@ -0,0 +1,92 @@
|
|||
# Requirements Document
|
||||
|
||||
## Introduction
|
||||
|
||||
`xce_test` (`src/controller/testsuite/test.c`) is the REPL/scripting client engineers use — both interactively and via piped scripts — to drive controller, worker, scheduler, and HA soak/E2E testing. A `/cli-ux` review found that its interrupt handling, flag validation, and output-stream discipline do not give scripted callers a reliable pass/fail signal, and give interactive users no way to recover from a stuck command without losing their session. This feature makes `xce_test`'s command-line surface predictable and fail-loud: a mistyped flag, a stuck RPC, or a misrouted error message should never be silently absorbed or misreported. `xce_test`'s existing exit-code/auto-variable verdict machinery (`xce-test-e2e-primitives`, implemented) and its per-command `help <cmd>` texts are unaffected and are the foundation this feature builds on.
|
||||
|
||||
## Boundary Context
|
||||
|
||||
- **In scope**: process-level `-h`/`--help`/`--version` behavior; uniform rejection of unrecognized or malformed command-line flags across every `xce_test` command; consistent routing of error/diagnostic text to stderr and command-result data to stdout; Ctrl-C (SIGINT) behavior that cancels the in-flight operation or the current input line instead of silently doing nothing or exiting the session; a machine-readable output mode for the tabular listing commands (`workerlist`, `joblist`, `reslist`); command-not-found guidance that tells the user when a command exists in a different context; adding a long-form flag equivalent, consistently named per concept, for every command's existing short-only flags; command-name tab completion at the interactive prompt.
|
||||
- **Out of scope**: the `xce_test` → `xce_harness`-style rename (tracked separately, to be done later); any change to controller/worker/scheduler wire protocol or RPC semantics; the auto-variable, exit-code-verdict, or shell-escape (`!`) mechanisms already shipped by `xce-test-e2e-primitives`; a full curses/TUI mode; color/theming; a `.xce_testrc` auto-executed startup-file feature (tracked as a separate, later task); changing what a successful command's result data actually contains; **renaming, removing, or changing the meaning of any existing short flag** (e.g. resolving `-m`'s four different meanings, or `-c`'s three, by reassigning a letter) — that is a breaking change to every script already using these flags and is deferred to a future, explicitly-scoped decision, not bundled into this feature; tab completion of flag *values* (jobids, hostnames, resource names) — only command-*name* completion is in scope; a per-command `-h`/`--help` flag handled by each `cmd_*` function individually — the existing in-session `help <cmd>` command already serves this need and is not being duplicated.
|
||||
- **Adjacent expectations**: `xce-test-e2e-primitives` explicitly deferred structured/JSON output as out of its boundary — this feature now owns that gap for the listing commands only (`jobsub`/`joblist`'s existing `$jobid`/`$jobcount` auto-variable capture is unaffected and remains the scripted way to read those specific values). Cancelling a client-side wait on Ctrl-C does not need to cancel the request on the controller/worker/scheduler side — the daemon's own RPC handling and timeouts are unaffected. Existing scripts that parse the current human-readable table output must keep working unchanged when the new machine-readable mode is not requested. Requirement 4's process-level `-h`/`--help` and the existing in-session `help <cmd>` command remain the sole help surfaces — Requirement 7's long-form flags are additive documentation of an existing command's own flags, not a new help mechanism.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement 1: Responsive Interrupt Handling
|
||||
|
||||
**Objective:** As an operator running `xce_test` interactively against a live or degraded HA cluster, I want Ctrl-C to cancel whatever is currently happening instead of silently doing nothing or exiting my whole session, so that I can recover control during a stuck command without losing my session state.
|
||||
|
||||
#### Acceptance Criteria
|
||||
1. When the operator presses Ctrl-C while a command is blocked awaiting a cluster RPC reply, the xce_test session shall abandon the wait immediately, without waiting for the RPC's own timeout to elapse, and return control to the prompt.
|
||||
2. When the operator presses Ctrl-C at the command prompt with no command in flight, the xce_test session shall discard the currently-typed input line and redisplay the prompt.
|
||||
3. If a command is interrupted by Ctrl-C, then the xce_test session shall report that command as failed, consistent with how any other failing command is reported.
|
||||
4. The xce_test session shall exit only as a direct result of an explicit `quit` command, an EOF at the top-level prompt, or an abort-on-failure condition in scripted mode — never as a side effect of a single Ctrl-C keypress.
|
||||
5. While a command is being interrupted by Ctrl-C, the xce_test session shall leave the active worker/scheduler/client context and its underlying connection intact.
|
||||
|
||||
### Requirement 2: Fail-Loud Flag Validation
|
||||
|
||||
**Objective:** As an author of HA soak/E2E test scripts driven through `xce_test`, I want every command to reject a flag it doesn't recognize or that is missing a required value, so that a typo in a script fails loudly instead of silently running with an unintended default.
|
||||
|
||||
#### Acceptance Criteria
|
||||
1. When any xce_test command is invoked with a flag it does not recognize, the xce_test session shall reject the command, report the invalid flag, and return a nonzero result, instead of running the command while ignoring that flag.
|
||||
2. When any xce_test command is invoked with a flag that requires a value but none is given, the xce_test session shall reject the command and report which flag is missing its value, instead of proceeding with an empty or default value.
|
||||
3. The xce_test session shall apply Acceptance Criteria 1 and 2 uniformly across every command that accepts flags — no command shall silently ignore an unrecognized or malformed flag.
|
||||
4. When the xce_test session rejects a command for an invalid or malformed flag, the xce_test session shall include that command's usage summary in the error output.
|
||||
|
||||
### Requirement 3: Consistent Error and Output Stream Routing
|
||||
|
||||
**Objective:** As an author of scripts that pipe or redirect `xce_test`'s output, I want every error and diagnostic message on stderr and every command result on stdout, so that I can reliably separate machine-parseable results from human diagnostics no matter which command fails.
|
||||
|
||||
#### Acceptance Criteria
|
||||
1. The xce_test session shall write every error, warning, or diagnostic message to stderr.
|
||||
2. The xce_test session shall write a command's successful result data to stdout only.
|
||||
3. The xce_test session shall apply Acceptance Criteria 1 and 2 uniformly across every command and every error path, including flag-validation errors, RPC failures, and internal errors.
|
||||
4. While xce_test's stdout and stderr are redirected to different destinations (a script or pipe), the xce_test session shall route a given message to the same stream it would use in an interactive TTY session.
|
||||
|
||||
### Requirement 4: Process-Level Help and Version Discovery
|
||||
|
||||
**Objective:** As a new or infrequent user of `xce_test`, I want `-h`/`--help` and `--version` to behave the way they do in other command-line tools, so that I can learn how to use `xce_test` without already knowing its internals.
|
||||
|
||||
#### Acceptance Criteria
|
||||
1. When xce_test is invoked with `-h` or `--help`, the xce_test process shall print usage help and exit without starting the interactive/scripted command session.
|
||||
2. When xce_test is invoked with `--version`, the xce_test process shall print its version information and exit without starting the interactive/scripted command session.
|
||||
3. The xce_test process's `-h`/`--help` output shall describe the in-session `help` command as the way to list and learn available commands.
|
||||
|
||||
### Requirement 5: Machine-Readable Output for Listing Commands
|
||||
|
||||
**Objective:** As an author of an automated soak/E2E script, I want the tabular listing commands to offer a structured, machine-parseable output mode, so that my script can reliably extract fields without depending on fixed-width column alignment.
|
||||
|
||||
#### Acceptance Criteria
|
||||
1. Where a machine-readable output mode is requested for `workerlist`, `joblist`, or `reslist`, the xce_test session shall emit each record in a structured, parseable format (for example, JSON) containing the same fields as that command's human-readable table.
|
||||
2. When no machine-readable output mode is requested, the xce_test session shall continue to print the existing human-readable table for `workerlist`, `joblist`, and `reslist`, unchanged.
|
||||
3. The xce_test session shall not truncate or misalign a field's value in machine-readable output regardless of the field value's length.
|
||||
4. The xce_test session shall document, via `help <cmd>`, how to request the machine-readable output mode for each command that supports it.
|
||||
|
||||
### Requirement 6: Context-Aware Command-Not-Found Guidance
|
||||
|
||||
**Objective:** As a user new to `xce_test`'s top/worker/scheduler/client context tree, I want an unrecognized command to tell me if it exists in a different context, so that I can find where to run it without trial and error.
|
||||
|
||||
#### Acceptance Criteria
|
||||
1. When a typed command is not found in the active context, and a command of that exact name exists in a different context, the xce_test session shall report the command as not found in the current context and identify the context where it does exist.
|
||||
2. When a typed command is not found in the active context and does not exist in any context, the xce_test session shall report it as not found without naming an unrelated context.
|
||||
3. The xce_test session shall not change the active context or execute any command as a result of a not-found lookup.
|
||||
|
||||
### Requirement 7: Long-Form, Cross-Command-Consistent Flags
|
||||
|
||||
**Objective:** As a user or script author who has learned one `xce_test` command's flags, I want a long-form equivalent of every flag, named the same way for the same concept across every command that accepts it, so that muscle memory from one command doesn't mislead me in another and my scripts can be self-documenting.
|
||||
|
||||
#### Acceptance Criteria
|
||||
1. The xce_test session shall provide a long-form flag equivalent for every flag that is currently available only as a short (single-letter) flag.
|
||||
2. The xce_test session shall use the same long-form flag name for the same real-world concept (for example: a target host, a command line to run, a resource name, a job identifier) in every command that accepts that concept, regardless of what short letter each command already uses for it.
|
||||
3. When a long-form flag equivalent is added for an existing short flag, the xce_test session shall leave that short flag's accepted values, default, and behavior unchanged.
|
||||
4. The xce_test process's help output for a command (`help <cmd>`) shall list both the short and long form of each of that command's flags.
|
||||
|
||||
### Requirement 8: Command-Name Tab Completion
|
||||
|
||||
**Objective:** As an interactive user, I want tab completion for command names at the prompt, so that I can discover and enter commands without memorizing the full command table or re-running `help`.
|
||||
|
||||
#### Acceptance Criteria
|
||||
1. While the session is interactive and the operator presses tab while typing a command name, the xce_test session shall offer completions drawn only from the command names valid in the currently active context.
|
||||
2. When the partially-typed text unambiguously matches exactly one command name in the active context, pressing tab shall complete it in full.
|
||||
3. When the partially-typed text matches more than one command name in the active context, pressing tab shall list the matching candidates without altering the input line.
|
||||
4. The xce_test session shall not offer a command name as a completion candidate when that command is not valid in the currently active context.
|
||||
22
.kiro/specs/xce-test-cli-ux/spec.json
Normal file
22
.kiro/specs/xce-test-cli-ux/spec.json
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
{
|
||||
"feature_name": "xce-test-cli-ux",
|
||||
"created_at": "2026-08-26T08:38:15Z",
|
||||
"updated_at": "2026-08-26T09:44:21Z",
|
||||
"language": "en",
|
||||
"phase": "requirements-generated",
|
||||
"approvals": {
|
||||
"requirements": {
|
||||
"generated": true,
|
||||
"approved": false
|
||||
},
|
||||
"design": {
|
||||
"generated": false,
|
||||
"approved": false
|
||||
},
|
||||
"tasks": {
|
||||
"generated": false,
|
||||
"approved": false
|
||||
}
|
||||
},
|
||||
"ready_for_implementation": false
|
||||
}
|
||||
|
|
@ -18,6 +18,8 @@ hostsGroup.POST("/control", RequirePermission("host:control"), handler)
|
|||
|
||||
**cluster_admin** bypasses all permission checks via early return — do not add new bypass paths outside this single mechanism.
|
||||
|
||||
**Known gap (not yet scoped): runtime RBAC mutations are not persisted.** `store.Roles`/`store.UserRoles` (`store/stores.go`) are loaded from `config/rbac.yaml` at startup via `LoadRBACConfig()`, but are also mutated at runtime with no write-back to disk or any DB — `RoleController.AddPermissionsForRole` (`controllers/roles.go`), and `store.AddRoleForUser`/`DeleteRoleForUser` (called from `AuthController.Logon`'s auto-role-assignment and the user-role-assignment endpoints in `controllers/users.go`). On apiserver restart, any role/permission granted or user/role assignment made via API since the last startup silently reverts to whatever `rbac.yaml` says, with no error or audit trail of the loss. This is the same save/restore field-completeness failure shape documented in `persistence.md` (state that "recovers correctly" while silently dropping data it depends on), just in the apiserver's in-memory RBAC store rather than a WAL/snapshot. Flagged during the `scheduler-snapshot-job-allocation` cross-component persistence survey (2026-08-27); intentionally deferred rather than fixed inline — needs its own scoped design (write-through to `rbac.yaml`? a small embedded store? explicit "runtime-only, does not survive restart" documented as accepted scope?) rather than a quick patch.
|
||||
|
||||
## Permission String Format
|
||||
|
||||
`"action:scope"` — e.g., `"queue:submit"`, `"host:view"`, `"queue:control"`.
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
{
|
||||
"last_scanned": "2026-08-27T07:00:04.610Z",
|
||||
"git_head": "446437bfce44e53b9cc622b397befb813a3d77bf",
|
||||
"last_scanned": "2026-08-27T19:00:00.801Z",
|
||||
"git_head": "de6ad514e01d4907fdf1d1728a17445ace3a7481",
|
||||
"file_count": 504
|
||||
}
|
||||
File diff suppressed because it is too large
Load diff
|
|
@ -1,6 +1,6 @@
|
|||
# anatomy.md
|
||||
|
||||
> Auto-maintained by OpenWolf. Last scanned: 2026-08-27T07:51:49.234Z
|
||||
> Auto-maintained by OpenWolf. Last scanned: 2026-08-28T05:12:31.323Z
|
||||
> Files: 507 tracked | Anatomy hits: 0 | Misses: 0
|
||||
|
||||
## ./
|
||||
|
|
@ -9,7 +9,7 @@
|
|||
- `.gitignore` — Git ignore rules (~402 tok)
|
||||
- `.mcp.json` (~7 tok)
|
||||
- `.opencode.json` (~7 tok)
|
||||
- `CLAUDE.md` — OpenWolf (~1526 tok)
|
||||
- `CLAUDE.md` — OpenWolf (~1765 tok)
|
||||
- `README.md` — Project documentation (~612 tok)
|
||||
|
||||
## .agent-shell/transcripts/
|
||||
|
|
@ -25,10 +25,7 @@
|
|||
## .codegraph/
|
||||
|
||||
- `.gitignore` — Git ignore rules (~61 tok)
|
||||
- `codegraph.db-shm` (~26215 tok)
|
||||
- `codegraph.db-wal` (~0 tok)
|
||||
- `daemon.log` (~22564 tok)
|
||||
- `daemon.pid` (~45 tok)
|
||||
- `daemon.log` (~23680 tok)
|
||||
|
||||
## .kiro/settings/rules/
|
||||
|
||||
|
|
@ -488,6 +485,15 @@
|
|||
- `spec.json` (~131 tok)
|
||||
- `tasks.md` — Implementation Plan (~1962 tok)
|
||||
|
||||
## .kiro/specs/scheduler-snapshot-job-allocation/
|
||||
|
||||
- `brief.md` — Brief: scheduler-snapshot-job-allocation (~4469 tok)
|
||||
- `design.md` — Design Document: scheduler-snapshot-job-allocation (~7948 tok)
|
||||
- `requirements.md` — Requirements Document (~2514 tok)
|
||||
- `research.md` — Summary (~3634 tok)
|
||||
- `spec.json` (~133 tok)
|
||||
- `tasks.md` — Implementation Plan (~2835 tok)
|
||||
|
||||
## .kiro/specs/worker-join-refactor/
|
||||
|
||||
- `design.md` — Design Document: worker-join-refactor (~4709 tok)
|
||||
|
|
@ -529,11 +535,11 @@
|
|||
- `config-design.md` — Configuration Design: What Belongs in Shared Config (~2053 tok)
|
||||
- `error-handling.md` — Error Handling Standards (~1367 tok)
|
||||
- `ha-async-io.md` — HA Protocol & Async I/O (~4721 tok)
|
||||
- `persistence.md` — WAL & Persistence Durability (~3425 tok)
|
||||
- `persistence.md` — WAL & Persistence Durability (~3851 tok)
|
||||
- `product.md` — Product Overview (~1509 tok)
|
||||
- `roadmap.md` — Roadmap (~15324 tok)
|
||||
- `scheduler-driver-equivalence.md` — Scheduler Driver Equivalence (~1520 tok)
|
||||
- `security.md` — Security Standards (~1636 tok)
|
||||
- `security.md` — Security Standards (~1960 tok)
|
||||
- `structure.md` — Project Structure (~1348 tok)
|
||||
- `tech.md` — Technology Stack (~1500 tok)
|
||||
- `testing.md` — Testing Standards (~976 tok)
|
||||
|
|
@ -785,15 +791,12 @@
|
|||
## src/build-asan/controller/sched_push_unit_test.p/
|
||||
|
||||
- `dataex.c.o` (~255400 tok)
|
||||
- `egress.c.o` (~246954 tok)
|
||||
- `sched.c.o` (~267906 tok)
|
||||
- `testsuite_test_job_array_sched_push.c.o` (~264250 tok)
|
||||
|
||||
## src/build-asan/controller/sched_worker_change_unit_test.p/
|
||||
## src/scheduler/src/
|
||||
|
||||
- `dataex.c.o` (~255400 tok)
|
||||
- `egress.c.o` (~246954 tok)
|
||||
- `snapshot_load.c` — include <errno.h> (~5573 tok)
|
||||
- `snapshot.c` — include <errno.h> (~8498 tok)
|
||||
|
||||
## src/tools/
|
||||
## src/scheduler/testsuite/
|
||||
|
||||
- `deploy-ha.sh` (~8298 tok)
|
||||
- `test_sim_e2e_original_motivation.c` (~17120 tok)
|
||||
|
|
|
|||
|
|
@ -58,7 +58,8 @@
|
|||
},
|
||||
"dashboard": {
|
||||
"enabled": true,
|
||||
"port": 18791
|
||||
"port": 18791,
|
||||
"host": "127.0.0.1"
|
||||
},
|
||||
"designqc": {
|
||||
"enabled": true,
|
||||
|
|
@ -79,6 +80,40 @@
|
|||
},
|
||||
"agents": [
|
||||
"claude"
|
||||
]
|
||||
],
|
||||
"reads": {
|
||||
"duplicate_mode": "warn",
|
||||
"skeleton_hints": true
|
||||
},
|
||||
"bash": {
|
||||
"filter_mode": "suggest",
|
||||
"governor": {
|
||||
"mode": "replace",
|
||||
"threshold_tokens": 2000,
|
||||
"families": {
|
||||
"grep_flood": "replace",
|
||||
"file_print": "replace",
|
||||
"git_show": "replace",
|
||||
"test": "suggest",
|
||||
"build": "suggest",
|
||||
"unknown": "suggest"
|
||||
}
|
||||
}
|
||||
},
|
||||
"context": {
|
||||
"session_digest_budget_tokens": 1500,
|
||||
"reinjection_interval": 25,
|
||||
"state_budgets": {
|
||||
".wolf/cerebrum.md": 2000,
|
||||
".wolf/STATUS.md": 1000
|
||||
},
|
||||
"budgets": {
|
||||
"claude": 1500,
|
||||
"codex": 1200,
|
||||
"gemini": 1200,
|
||||
"opencode": 1200,
|
||||
"cursor": 800
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,40 +1,7 @@
|
|||
{
|
||||
"last_heartbeat": "2026-08-27T07:57:45.420Z",
|
||||
"last_heartbeat": "2026-08-28T06:29:24.365Z",
|
||||
"engine_status": "running",
|
||||
"execution_log": [
|
||||
{
|
||||
"task_id": "project-suggestions",
|
||||
"status": "failed",
|
||||
"timestamp": "2026-05-21T08:54:59.810Z",
|
||||
"duration_ms": 120927,
|
||||
"error": "claude -p failed: spawnSync /bin/sh ETIMEDOUT"
|
||||
},
|
||||
{
|
||||
"task_id": "project-suggestions",
|
||||
"status": "failed",
|
||||
"timestamp": "2026-05-21T08:59:01.535Z",
|
||||
"duration_ms": 241724,
|
||||
"error": "claude -p failed: spawnSync /bin/sh ETIMEDOUT"
|
||||
},
|
||||
{
|
||||
"task_id": "project-suggestions",
|
||||
"status": "failed",
|
||||
"timestamp": "2026-05-21T08:59:01.536Z",
|
||||
"duration_ms": 120843,
|
||||
"error": "claude -p failed: spawnSync /bin/sh ETIMEDOUT"
|
||||
},
|
||||
{
|
||||
"task_id": "memory-consolidation",
|
||||
"status": "success",
|
||||
"timestamp": "2026-05-21T09:00:00.558Z",
|
||||
"duration_ms": 1
|
||||
},
|
||||
{
|
||||
"task_id": "anatomy-rescan",
|
||||
"status": "success",
|
||||
"timestamp": "2026-05-21T13:00:00.039Z",
|
||||
"duration_ms": 32
|
||||
},
|
||||
{
|
||||
"task_id": "project-suggestions",
|
||||
"status": "failed",
|
||||
|
|
@ -621,6 +588,38 @@
|
|||
"status": "success",
|
||||
"timestamp": "2026-08-27T07:00:04.613Z",
|
||||
"duration_ms": 4599
|
||||
},
|
||||
{
|
||||
"task_id": "memory-consolidation",
|
||||
"status": "success",
|
||||
"timestamp": "2026-08-27T09:00:00.005Z",
|
||||
"duration_ms": 3
|
||||
},
|
||||
{
|
||||
"task_id": "cerebrum-reflection",
|
||||
"status": "failed",
|
||||
"timestamp": "2026-08-27T09:01:47.656Z",
|
||||
"duration_ms": 0,
|
||||
"error": "ai_task is no longer supported: OpenWolf makes no model calls. Remove this task from .wolf/cron-manifest.json."
|
||||
},
|
||||
{
|
||||
"task_id": "project-suggestions",
|
||||
"status": "failed",
|
||||
"timestamp": "2026-08-27T09:01:49.211Z",
|
||||
"duration_ms": 0,
|
||||
"error": "ai_task is no longer supported: OpenWolf makes no model calls. Remove this task from .wolf/cron-manifest.json."
|
||||
},
|
||||
{
|
||||
"task_id": "anatomy-rescan",
|
||||
"status": "success",
|
||||
"timestamp": "2026-08-27T19:00:00.802Z",
|
||||
"duration_ms": 786
|
||||
},
|
||||
{
|
||||
"task_id": "anatomy-rescan",
|
||||
"status": "success",
|
||||
"timestamp": "2026-08-28T01:00:00.042Z",
|
||||
"duration_ms": 32
|
||||
}
|
||||
],
|
||||
"dead_letter_queue": [],
|
||||
|
|
|
|||
|
|
@ -671,3 +671,68 @@
|
|||
[2026-08-27T03:40:01.458Z] [WARN] Skipping broadcast for large file: cache/buglog.db (3100672 bytes)
|
||||
[2026-08-27T07:00:00.012Z] [INFO] Executing task: Full anatomy rescan
|
||||
[2026-08-27T07:00:04.615Z] [INFO] Task Full anatomy rescan completed in 4599ms
|
||||
[2026-08-27T08:29:08.256Z] [INFO] WebSocket client connected
|
||||
[2026-08-27T08:43:06.924Z] [WARN] Skipping broadcast for large file: buglog.json (1601913 bytes)
|
||||
[2026-08-27T08:45:04.714Z] [WARN] Skipping broadcast for large file: cache/buglog.db (3100672 bytes)
|
||||
[2026-08-27T08:48:08.540Z] [INFO] WebSocket client connected
|
||||
[2026-08-27T08:58:14.010Z] [INFO] Config changed — hot-reload not fully implemented yet
|
||||
[2026-08-27T09:00:00.002Z] [INFO] Executing task: Consolidate old memory
|
||||
[2026-08-27T09:00:00.005Z] [INFO] Task Consolidate old memory completed in 3ms
|
||||
[2026-08-27T09:00:07.451Z] [INFO] WebSocket client connected
|
||||
[2026-08-27T09:01:47.655Z] [INFO] Executing task: Cerebrum reflection
|
||||
[2026-08-27T09:01:47.656Z] [ERROR] Task Cerebrum reflection failed (attempt 1): ai_task is no longer supported: OpenWolf makes no model calls. Remove this task from .wolf/cron-manifest.json.
|
||||
[2026-08-27T09:01:49.210Z] [INFO] Executing task: AI suggestions
|
||||
[2026-08-27T09:01:49.210Z] [ERROR] Task AI suggestions failed (attempt 1): ai_task is no longer supported: OpenWolf makes no model calls. Remove this task from .wolf/cron-manifest.json.
|
||||
[2026-08-27T09:05:26.770Z] [WARN] Skipping broadcast for large file: token-ledger.json (3109332 bytes)
|
||||
[2026-08-27T09:08:09.709Z] [WARN] Skipping broadcast for large file: token-ledger.json (3112649 bytes)
|
||||
[2026-08-27T09:13:16.843Z] [WARN] Skipping broadcast for large file: token-ledger.json (3113649 bytes)
|
||||
[2026-08-27T09:15:59.551Z] [WARN] Skipping broadcast for large file: token-ledger.json (3117062 bytes)
|
||||
[2026-08-27T09:17:04.301Z] [WARN] Skipping broadcast for large file: token-ledger.json (3117849 bytes)
|
||||
[2026-08-27T09:18:20.974Z] [WARN] Skipping broadcast for large file: token-ledger.json (3119041 bytes)
|
||||
[2026-08-27T09:22:13.514Z] [WARN] Skipping broadcast for large file: token-ledger.json (3119279 bytes)
|
||||
[2026-08-27T09:30:33.084Z] [WARN] Skipping broadcast for large file: token-ledger.json (3119597 bytes)
|
||||
[2026-08-27T09:34:46.844Z] [WARN] Skipping broadcast for large file: token-ledger.json (3120556 bytes)
|
||||
[2026-08-27T09:38:16.439Z] [WARN] Skipping broadcast for large file: token-ledger.json (3123001 bytes)
|
||||
[2026-08-27T09:46:35.061Z] [WARN] Skipping broadcast for large file: token-ledger.json (3123001 bytes)
|
||||
[2026-08-27T09:50:56.962Z] [WARN] Skipping broadcast for large file: token-ledger.json (3123770 bytes)
|
||||
[2026-08-27T10:15:50.946Z] [WARN] Skipping broadcast for large file: token-ledger.json (3126885 bytes)
|
||||
[2026-08-27T10:22:45.731Z] [WARN] Skipping broadcast for large file: token-ledger.json (3127403 bytes)
|
||||
[2026-08-27T10:31:19.363Z] [WARN] Skipping broadcast for large file: token-ledger.json (3129770 bytes)
|
||||
[2026-08-27T10:39:14.677Z] [WARN] Skipping broadcast for large file: token-ledger.json (3131784 bytes)
|
||||
[2026-08-27T10:45:25.068Z] [WARN] Skipping broadcast for large file: token-ledger.json (3132162 bytes)
|
||||
[2026-08-27T10:48:50.104Z] [WARN] Skipping broadcast for large file: token-ledger.json (3132887 bytes)
|
||||
[2026-08-27T11:26:19.473Z] [WARN] Skipping broadcast for large file: token-ledger.json (3133892 bytes)
|
||||
[2026-08-27T11:50:46.368Z] [WARN] Skipping broadcast for large file: buglog.json (1607012 bytes)
|
||||
[2026-08-27T11:53:49.671Z] [WARN] Skipping broadcast for large file: token-ledger.json (3136514 bytes)
|
||||
[2026-08-27T12:07:17.636Z] [WARN] Skipping broadcast for large file: cache/buglog.db (3100672 bytes)
|
||||
[2026-08-27T12:08:47.902Z] [WARN] Skipping broadcast for large file: token-ledger.json (3136976 bytes)
|
||||
[2026-08-27T12:59:15.643Z] [WARN] Skipping broadcast for large file: token-ledger.json (3139876 bytes)
|
||||
[2026-08-27T13:17:03.299Z] [WARN] Skipping broadcast for large file: token-ledger.json (3140739 bytes)
|
||||
[2026-08-27T13:33:52.632Z] [WARN] Skipping broadcast for large file: token-ledger.json (3141491 bytes)
|
||||
[2026-08-27T13:46:04.006Z] [WARN] Skipping broadcast for large file: token-ledger.json (3141443 bytes)
|
||||
[2026-08-27T13:50:26.238Z] [WARN] Skipping broadcast for large file: token-ledger.json (3141443 bytes)
|
||||
[2026-08-27T13:59:55.168Z] [WARN] Skipping broadcast for large file: token-ledger.json (3141939 bytes)
|
||||
[2026-08-27T14:16:51.350Z] [WARN] Skipping broadcast for large file: token-ledger.json (3143360 bytes)
|
||||
[2026-08-27T14:31:57.954Z] [WARN] Skipping broadcast for large file: token-ledger.json (3144268 bytes)
|
||||
[2026-08-27T14:55:36.064Z] [INFO] Config changed — hot-reload not fully implemented yet
|
||||
[2026-08-27T14:55:36.065Z] [WARN] Skipping broadcast for large file: token-ledger.json (3144268 bytes)
|
||||
[2026-08-27T15:03:46.683Z] [WARN] Skipping broadcast for large file: token-ledger.json (3145863 bytes)
|
||||
[2026-08-27T15:22:56.129Z] [WARN] Skipping broadcast for large file: token-ledger.json (3147160 bytes)
|
||||
[2026-08-27T15:34:11.725Z] [WARN] Skipping broadcast for large file: token-ledger.json (3148971 bytes)
|
||||
[2026-08-27T15:43:52.725Z] [WARN] Skipping broadcast for large file: token-ledger.json (3149387 bytes)
|
||||
[2026-08-27T16:09:05.717Z] [WARN] Skipping broadcast for large file: token-ledger.json (3152327 bytes)
|
||||
[2026-08-27T19:00:00.015Z] [INFO] Executing task: Full anatomy rescan
|
||||
[2026-08-27T19:00:00.802Z] [INFO] Task Full anatomy rescan completed in 786ms
|
||||
[2026-08-28T01:00:00.010Z] [INFO] Executing task: Full anatomy rescan
|
||||
[2026-08-28T01:00:00.042Z] [INFO] anatomy rescan skipped: index matches the tree (stat sweep + git HEAD)
|
||||
[2026-08-28T01:00:00.042Z] [INFO] Task Full anatomy rescan completed in 32ms
|
||||
[2026-08-28T06:29:24.362Z] [INFO] Scheduled task: Full anatomy rescan (0 */6 * * *)
|
||||
[2026-08-28T06:29:24.363Z] [INFO] Scheduled task: Consolidate old memory (0 2 * * *)
|
||||
[2026-08-28T06:29:24.364Z] [INFO] Scheduled task: Token audit report (0 0 * * 1)
|
||||
[2026-08-28T06:29:24.364Z] [INFO] Scheduled task: Cerebrum reflection (0 3 * * 0)
|
||||
[2026-08-28T06:29:24.364Z] [INFO] Scheduled task: AI suggestions (0 4 * * 1)
|
||||
[2026-08-28T06:29:24.365Z] [INFO] File watcher started on .wolf/
|
||||
[2026-08-28T06:29:24.366Z] [INFO] OpenWolf daemon started
|
||||
[2026-08-28T06:29:24.369Z] [INFO] Dashboard server listening on 127.0.0.1:18791
|
||||
[2026-08-28T06:29:56.631Z] [INFO] WebSocket client connected
|
||||
[2026-08-28T06:30:15.500Z] [INFO] WebSocket client connected
|
||||
|
|
|
|||
148
.wolf/memory.md
148
.wolf/memory.md
|
|
@ -2826,3 +2826,151 @@
|
|||
| 00:51 | Edited src/tools/deploy-ha.sh | 3→3 lines | ~26 |
|
||||
| 00:51 | Edited src/tools/deploy-ha.sh | 3→3 lines | ~28 |
|
||||
| 00:51 | Edited src/tools/deploy-ha.sh | 3→3 lines | ~19 |
|
||||
| 01:34 | Created src/lib/testsuite/test_xcealloc.c | — | ~936 |
|
||||
| 01:35 | Edited src/lib/meson.build | 2→3 lines | ~38 |
|
||||
| 01:35 | Edited src/lib/testsuite/test.c | 1→2 lines | ~20 |
|
||||
| 01:35 | Edited src/lib/testsuite/test.c | 3→4 lines | ~22 |
|
||||
| 01:36 | Edited src/lib/xcealloc.c | modified if() | ~35 |
|
||||
| 01:36 | Edited src/lib/xcealloc.c | modified if() | ~41 |
|
||||
| 01:45 | Created .kiro/specs/scheduler-snapshot-job-allocation/brief.md | — | ~2503 |
|
||||
| 01:51 | Edited .kiro/specs/scheduler-snapshot-job-allocation/brief.md | modified spec() | ~493 |
|
||||
| 01:51 | Edited .kiro/specs/scheduler-snapshot-job-allocation/brief.md | inline fix | ~102 |
|
||||
| 01:56 | Edited .claude/skills/xce-audit-invariants/SKILL.md | expanded (+20 lines) | ~470 |
|
||||
| 01:57 | Edited .kiro/steering/persistence.md | modified AND() | ~547 |
|
||||
| 01:57 | Edited CLAUDE.md | 2→3 lines | ~62 |
|
||||
| 01:58 | Edited CLAUDE.md | 2→3 lines | ~285 |
|
||||
| 01:58 | Edited .kiro/specs/scheduler-snapshot-job-allocation/brief.md | 1→2 lines | ~221 |
|
||||
| 01:59 | Edited .claude/skills/kiro-spec-design/rules/design-review-gate.md | expanded (+7 lines) | ~347 |
|
||||
| 01:59 | Edited .claude/skills/kiro-validate-design/rules/design-review.md | expanded (+6 lines) | ~291 |
|
||||
| 02:00 | Edited .claude/skills/kiro-spec-design/rules/design-review-gate.md | expanded (+6 lines) | ~300 |
|
||||
| 02:01 | Edited .claude/skills/kiro-validate-design/rules/design-review.md | 7→12 lines | ~439 |
|
||||
|
||||
## Session: 2026-08-27 02:05
|
||||
|
||||
| Time | Action | File(s) | Outcome | ~Tokens |
|
||||
|------|--------|---------|---------|--------|
|
||||
| 02:07 | Created .kiro/specs/scheduler-snapshot-job-allocation/spec.json | — | ~134 |
|
||||
| 02:08 | Created .kiro/specs/scheduler-snapshot-job-allocation/requirements.md | — | ~720 |
|
||||
| 02:30 | Edited .kiro/steering/security.md | modified gap() | ~387 |
|
||||
| 02:32 | Edited .kiro/specs/scheduler-snapshot-job-allocation/brief.md | modified survey() | ~303 |
|
||||
| 02:33 | Edited .kiro/specs/scheduler-snapshot-job-allocation/brief.md | modified fixed() | ~1112 |
|
||||
| 02:33 | Edited .kiro/specs/scheduler-snapshot-job-allocation/brief.md | 24→29 lines | ~1395 |
|
||||
| 02:34 | Edited .kiro/specs/scheduler-snapshot-job-allocation/brief.md | 5→6 lines | ~446 |
|
||||
| 02:34 | Edited .kiro/specs/scheduler-snapshot-job-allocation/requirements.md | 1→3 lines | ~736 |
|
||||
| 02:37 | Edited .kiro/specs/scheduler-snapshot-job-allocation/requirements.md | modified mechanism() | ~1705 |
|
||||
| 02:37 | Edited .kiro/specs/scheduler-snapshot-job-allocation/spec.json | 8→8 lines | ~55 |
|
||||
| 02:50 | Edited .kiro/specs/scheduler-snapshot-job-allocation/requirements.md | reduced (-6 lines) | ~370 |
|
||||
| 02:50 | Edited .kiro/specs/scheduler-snapshot-job-allocation/brief.md | modified note() | ~347 |
|
||||
| 02:50 | Edited .kiro/specs/scheduler-snapshot-job-allocation/spec.json | inline fix | ~12 |
|
||||
| 03:05 | Edited .kiro/specs/scheduler-snapshot-job-allocation/spec.json | 4→4 lines | ~22 |
|
||||
| 03:11 | Created .kiro/specs/scheduler-snapshot-job-allocation/research.md | — | ~3491 |
|
||||
| 03:14 | Created .kiro/specs/scheduler-snapshot-job-allocation/design.md | — | ~7090 |
|
||||
| 03:14 | Edited .kiro/specs/scheduler-snapshot-job-allocation/design.md | 2→2 lines | ~54 |
|
||||
| 03:14 | Edited .kiro/specs/scheduler-snapshot-job-allocation/design.md | inline fix | ~28 |
|
||||
| 03:15 | Edited .kiro/specs/scheduler-snapshot-job-allocation/spec.json | 12→12 lines | ~74 |
|
||||
| 03:26 | Edited .kiro/specs/scheduler-snapshot-job-allocation/design.md | modified comment() | ~200 |
|
||||
| 03:26 | Edited .kiro/specs/scheduler-snapshot-job-allocation/design.md | inline fix | ~231 |
|
||||
| 03:26 | Edited .kiro/specs/scheduler-snapshot-job-allocation/design.md | inline fix | ~107 |
|
||||
| 03:27 | Edited .kiro/specs/scheduler-snapshot-job-allocation/design.md | 4→5 lines | ~412 |
|
||||
| 03:27 | Edited .kiro/specs/scheduler-snapshot-job-allocation/design.md | 6→7 lines | ~244 |
|
||||
| 03:28 | Edited .kiro/specs/scheduler-snapshot-job-allocation/design.md | inline fix | ~94 |
|
||||
| 03:28 | Edited .kiro/specs/scheduler-snapshot-job-allocation/design.md | inline fix | ~186 |
|
||||
| 03:28 | Edited .kiro/specs/scheduler-snapshot-job-allocation/research.md | 1→3 lines | ~494 |
|
||||
| 03:29 | Edited .kiro/specs/scheduler-snapshot-job-allocation/design.md | 4→5 lines | ~200 |
|
||||
| 03:30 | Edited .kiro/specs/scheduler-snapshot-job-allocation/design.md | 7→12 lines | ~727 |
|
||||
| 03:30 | Edited .kiro/specs/scheduler-snapshot-job-allocation/spec.json | inline fix | ~12 |
|
||||
| 03:34 | Edited .kiro/specs/scheduler-snapshot-job-allocation/spec.json | 4→4 lines | ~20 |
|
||||
| 03:37 | Created .kiro/specs/scheduler-snapshot-job-allocation/tasks.md | — | ~2371 |
|
||||
| 03:43 | Edited .kiro/specs/scheduler-snapshot-job-allocation/tasks.md | 2→2 lines | ~38 |
|
||||
| 03:44 | Edited .kiro/specs/scheduler-snapshot-job-allocation/tasks.md | 6→7 lines | ~149 |
|
||||
| 03:44 | Edited .kiro/specs/scheduler-snapshot-job-allocation/tasks.md | 25→29 lines | ~761 |
|
||||
| 03:48 | Edited .kiro/specs/scheduler-snapshot-job-allocation/spec.json | 17→17 lines | ~94 |
|
||||
| 04:23 | Edited .kiro/specs/scheduler-snapshot-job-allocation/spec.json | 6→6 lines | ~31 |
|
||||
| 04:33 | Edited src/lib/testsuite/test_xcealloc.c | 8→13 lines | ~93 |
|
||||
| 04:36 | Edited src/lib/testsuite/test_xcealloc.c | modified test_dup_job_alloc_single_host_single_resource() | ~3933 |
|
||||
| 04:36 | Edited src/include/xcealloc.h | 6→7 lines | ~120 |
|
||||
| 04:37 | Edited src/lib/xcealloc.c | added 14 condition(s) | ~938 |
|
||||
| 04:46 | Edited src/lib/testsuite/test_xcealloc.c | 2→7 lines | ~107 |
|
||||
| 05:07 | Edited .kiro/specs/scheduler-snapshot-job-allocation/tasks.md | inline fix | ~20 |
|
||||
| 05:14 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | modified make_job() | ~330 |
|
||||
| 05:15 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | modified test_snapshot_load_builds_schedobj_for_primary_compound_and_alter_resreq() | ~1821 |
|
||||
| 05:15 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | 4→8 lines | ~82 |
|
||||
| 05:21 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | expanded (+8 lines) | ~194 |
|
||||
| 05:24 | Edited src/scheduler/src/snapshot_load.c | added 4 condition(s) | ~1197 |
|
||||
| 05:25 | Edited src/scheduler/src/snapshot_load.c | added 2 condition(s) | ~247 |
|
||||
| 05:30 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | expanded (+19 lines) | ~380 |
|
||||
| 05:30 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | 7→9 lines | ~66 |
|
||||
| 06:15 | Edited .kiro/specs/scheduler-snapshot-job-allocation/tasks.md | inline fix | ~26 |
|
||||
| 06:24 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | 4→5 lines | ~28 |
|
||||
| 06:25 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | modified test_snapshot_save_populates_allocs_for_running_job_with_multi_host_allocation() | ~1937 |
|
||||
| 06:26 | Edited src/scheduler/src/snapshot_wire.c | 4→5 lines | ~28 |
|
||||
| 06:27 | Edited src/scheduler/src/snapshot_wire.c | added 2 condition(s) | ~292 |
|
||||
| 06:30 | Edited src/scheduler/src/snapshot_wire.c | modified if() | ~77 |
|
||||
| 06:44 | Edited .kiro/specs/scheduler-snapshot-job-allocation/tasks.md | inline fix | ~17 |
|
||||
| 07:02 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | modified test_snapshot_load_restores_allocs_and_accounts_for_running_job_across_cores() | ~2723 |
|
||||
| 07:02 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | 3→7 lines | ~83 |
|
||||
| 07:04 | Edited src/scheduler/src/snapshot_load.c | 6→7 lines | ~39 |
|
||||
| 07:04 | Edited src/scheduler/src/snapshot_load.c | added 2 condition(s) | ~556 |
|
||||
| 07:08 | Edited src/scheduler/src/snapshot_load.c | removed 31 lines | ~10 |
|
||||
| 07:13 | Edited src/scheduler/src/snapshot_load.c | added 2 condition(s) | ~390 |
|
||||
| 07:20 | Edited src/scheduler/src/snapshot_load.c | modified if() | ~145 |
|
||||
| 07:26 | Edited .kiro/specs/scheduler-snapshot-job-allocation/tasks.md | inline fix | ~21 |
|
||||
| 07:49 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | expanded (+19 lines) | ~551 |
|
||||
| 07:49 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | modified kill_sched_job() | ~502 |
|
||||
| 07:50 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | added 5 condition(s) | ~6402 |
|
||||
| 07:51 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | 3→5 lines | ~62 |
|
||||
| 08:13 | Edited src/scheduler/src/snapshot_load.c | modified if() | ~126 |
|
||||
| 08:14 | Edited src/scheduler/src/snapshot_load.c | inline fix | ~10 |
|
||||
| 08:19 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | 6→10 lines | ~161 |
|
||||
| 08:20 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | reason() → itself() | ~174 |
|
||||
| 08:21 | Edited .kiro/specs/scheduler-snapshot-job-allocation/tasks.md | inline fix | ~36 |
|
||||
| 08:27 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | modified 2() | ~2140 |
|
||||
| 08:27 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | 3→5 lines | ~61 |
|
||||
| 08:30 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | 4→4 lines | ~45 |
|
||||
| 08:30 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | 6→6 lines | ~49 |
|
||||
| 08:30 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | modified test_snapshot_load_leaves_running_job_without_allocation_at_zero_resinuse() | ~25 |
|
||||
| 08:38 | Edited src/scheduler/src/snapshot_load.c | modified if() | ~82 |
|
||||
| 08:42 | Edited .kiro/specs/scheduler-snapshot-job-allocation/tasks.md | inline fix | ~18 |
|
||||
| 08:54 | Created src/scheduler/testsuite/test_snapshot_load_allocator_regression.c | — | ~4447 |
|
||||
| 08:54 | Edited src/scheduler/testsuite/meson.build | modified 1() | ~556 |
|
||||
| 08:54 | Edited src/scheduler/testsuite/meson.build | 3→6 lines | ~85 |
|
||||
| 08:56 | Edited src/scheduler/testsuite/test_snapshot_load_allocator_regression.c | 4→5 lines | ~53 |
|
||||
| 08:56 | Edited src/scheduler/testsuite/test_snapshot_load_allocator_regression.c | inline fix | ~14 |
|
||||
| 09:01 | Edited src/scheduler/testsuite/test_snapshot_load_allocator_regression.c | modified build_pend_only_job() | ~449 |
|
||||
| 09:01 | Edited src/scheduler/testsuite/test_snapshot_load_allocator_regression.c | added 1 condition(s) | ~1367 |
|
||||
| 09:01 | Edited src/scheduler/testsuite/test_snapshot_load_allocator_regression.c | 1→2 lines | ~24 |
|
||||
| 09:02 | Edited src/scheduler/testsuite/test_snapshot_load_allocator_regression.c | expanded (+9 lines) | ~587 |
|
||||
| 09:05 | Edited src/scheduler/testsuite/test_snapshot_load_allocator_regression.c | 4→4 lines | ~71 |
|
||||
| 09:14 | Edited src/scheduler/src/snapshot_load.c | modified if() | ~82 |
|
||||
| 09:16 | Edited src/scheduler/testsuite/test_snapshot_load_allocator_regression.c | 6→3 lines | ~55 |
|
||||
| 09:17 | Edited src/scheduler/src/snapshot_load.c | modified if() | ~80 |
|
||||
| 09:17 | Edited src/scheduler/testsuite/test_snapshot_load_allocator_regression.c | 3→6 lines | ~89 |
|
||||
| 09:31 | Edited .kiro/specs/scheduler-snapshot-job-allocation/tasks.md | inline fix | ~33 |
|
||||
| 09:58 | Edited src/scheduler/testsuite/test_sim_e2e_original_motivation.c | modified below() | ~687 |
|
||||
| 10:00 | Edited src/scheduler/testsuite/test_sim_e2e_original_motivation.c | expanded (+9 lines) | ~114 |
|
||||
| 10:02 | Edited src/scheduler/testsuite/test_sim_e2e_original_motivation.c | expanded (+25 lines) | ~282 |
|
||||
| 10:04 | Edited src/scheduler/testsuite/test_sim_e2e_original_motivation.c | added 2 condition(s) | ~1076 |
|
||||
| 10:05 | Edited src/scheduler/testsuite/test_sim_e2e_original_motivation.c | modified Prelude() | ~726 |
|
||||
| 10:05 | Edited src/scheduler/testsuite/test_sim_e2e_original_motivation.c | expanded (+11 lines) | ~314 |
|
||||
| 10:06 | Edited src/scheduler/testsuite/test_sim_e2e_original_motivation.c | modified 3() | ~1194 |
|
||||
|
||||
## Session: 2026-08-28 21:01
|
||||
|
||||
| Time | Action | File(s) | Outcome | ~Tokens |
|
||||
|------|--------|---------|---------|--------|
|
||||
| 21:10 | Edited src/scheduler/testsuite/test_sim_e2e_original_motivation.c | modified here() | ~403 |
|
||||
| 21:12 | Edited src/scheduler/src/snapshot_load.c | modified if() | ~44 |
|
||||
| 21:13 | Edited src/scheduler/src/snapshot_load.c | modified if() | ~26 |
|
||||
| 21:25 | Edited .kiro/specs/scheduler-snapshot-job-allocation/tasks.md | inline fix | ~26 |
|
||||
| 21:26 | Edited .kiro/specs/scheduler-snapshot-job-allocation/tasks.md | modified finding() | ~224 |
|
||||
|
||||
## Session: 2026-08-28 21:29
|
||||
|
||||
| Time | Action | File(s) | Outcome | ~Tokens |
|
||||
|------|--------|---------|---------|--------|
|
||||
| 21:47 | Edited .kiro/specs/scheduler-snapshot-job-allocation/tasks.md | inline fix | ~27 |
|
||||
| 21:49 | Edited src/scheduler/src/snapshot.c | expanded (+23 lines) | ~697 |
|
||||
| 21:55 | Edited .kiro/specs/scheduler-snapshot-job-allocation/tasks.md | inline fix | ~22 |
|
||||
| 21:55 | Edited .kiro/specs/scheduler-snapshot-job-allocation/tasks.md | modified up() | ~250 |
|
||||
| 21:57 | Edited src/scheduler/src/snapshot.c | modified load() | ~159 |
|
||||
| 22:12 | Edited .kiro/specs/scheduler-snapshot-job-allocation/tasks.md | inline fix | ~22 |
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
Loading…
Add table
Add a link
Reference in a new issue