diff --git a/.kiro/specs/libxce-alloc-preason-equality/design.md b/.kiro/specs/libxce-alloc-preason-equality/design.md new file mode 100644 index 00000000..750c717d --- /dev/null +++ b/.kiro/specs/libxce-alloc-preason-equality/design.md @@ -0,0 +1,238 @@ +# Design Document + +## Overview + +**Purpose**: This feature delivers a single, shared implementation of structural equality for the scheduler's allocation and pending-reason data structures, published as public libxce API. + +**Users**: Scheduler engineers (today: `--sim-verify` replay checking and three scheduler test files) and, in future, controller engineers (`persist.c`, `resync.c`, `job.c` already carry these structs) who need to ask "do these two objects represent the same thing". + +**Impact**: Changes the current state by extracting four `static` comparison functions out of `src/scheduler/src/sim_verify.c` and into `src/lib/xcealloc.c`/`src/lib/preason.c`, and by switching the simulator and three white-box test consumers to the public API. The allocation comparators keep their documented `from`/`instid` exclusion; the pending-reason comparator becomes a full structural match (all fields, detail lists included). The decision-level dispatcher (`sim_decision_eq`) and mismatch reporting (`sim_log_mismatch`) stay in the scheduler, and `sim_decision_eq`'s PEND branch keeps a simulator-local main-reason-only comparison (bug-1845). No wire format or struct layout changes. + +### Goals +- Publish `alloc_eq`, `host_alloc_eq`, `job_alloc_eq` in `xcealloc.h` and `job_preason_eq` in `preason.h`. +- Make `job_preason_eq` a full structural match (main reason + host-based reasons + resource-based reasons, in order). +- Make the existing consumers use the public API, removing the white-box `#include` dependency on `sim_verify.c` for the type-level comparisons. +- Preserve `--sim-verify` behavior bit-for-bit (no new false/missed mismatches), which means the simulator's PEND check remains main-reason-only. +- Add libxce unit tests that pin the allocation-exclusion, full-match preason, and NULL-handling semantics. + +### Non-Goals +- Do not move `sim_decision_eq`, `sim_log_mismatch`, or the `describe_*` helpers (simulator-specific; depend on capnp/replay types and `ERROR()`). +- Do not change the simulator's PEND outcome check from main-reason-only to full-match — that would reintroduce bug-1845's false mismatches on cache-hit PENDs (see Decision 3). +- Do not introduce any new comparison call site beyond the existing four consumers. +- Do not touch the controller's persistence/resync code paths. + +## Boundary Commitments + +### This Spec Owns +- The public equality functions `alloc_eq`, `host_alloc_eq`, `job_alloc_eq` (`xcealloc.{c,h}`) and `job_preason_eq` (`preason.{c,h}`), and their documented equality contracts. +- The shared NULL-safe string-equality helper both modules need (see Decision 5). +- The migration of `sim_verify.c` and the three white-box test files onto those functions. +- libxce unit tests for the new functions (`test_xcealloc.c`, `test_preason.c`). + +### Out of Boundary +- The decision-level comparison and mismatch reporting that remain in `sim_verify.c`, including the main-reason-only PEND outcome check (simulator-local). +- The `.replay` wire format and the `replay_decision_t`/`xce_decision_t` types. +- Any change to the controller's consumption of these structs. + +### Allowed Dependencies +- `src/include/xce/types.h` (the struct definitions — unchanged). +- `xcealloc.c`/`xcealloc.h`, `preason.c`/`preason.h`, and `utils.{c,h}` (the modules that already own construction of these structs, plus the shared string helper). +- `string.h` (`strcmp`) for the string-field comparisons. + +### Revalidation Triggers +- A change to the `xce_alloc_t`/`xce_host_alloc_t`/`xce_job_alloc_t`/`xce_job_preason_t` struct layouts must re-check the field lists in the equality functions and the exclusion documentation. +- A change to the pending-reason construction paths (cache-hit vs. full-computation) that alters whether detail lists are populated must re-check the simulator's main-reason-only PEND rationale (bug-1845) and the full-match `job_preason_eq` contract. + +## Architecture + +### Existing Architecture Analysis +`src/scheduler/src/sim_verify.c` (474 lines) owns two layers: +1. **Type-level comparators** — five `static` functions (`sim_verify_str_eq`, `sim_verify_alloc_eq`, `sim_verify_host_alloc_eq`, `sim_verify_job_alloc_eq`, `sim_verify_job_preason_eq`) that walk two libxce structs field-by-field. They have zero dependency on the scheduler, capnp, or the replay layer. Four of them are cleanly movable; `sim_verify_job_preason_eq` is movable but its current contract (main-reason-only) is wrong for a *public* equality function — it encodes a simulator replay asymmetry, not an object-identity property. +2. **Decision-level dispatcher + reporting** — `sim_decision_eq` (dispatches on `xce_decision_t.op` vs `ReplayDecision_body_which`), `sim_log_mismatch`, and two `describe_*` helpers. These depend on `replay_decision_t`, capnp enums, and `ERROR()`. + +The four allocation/pending-reason structs are defined in `src/include/xce/types.h` and already constructed/deep-copied by `src/lib/xcealloc.c` (`make_alloc`, `make_host_alloc`, `make_job_alloc`, `dup_job_alloc`) and `src/lib/preason.c` (`add_job_preason`, `make_pendreason`). The controller consumes these same structs in `persist.c`, `resync.c`, and `job.c` with no equality API available. + +Three scheduler test files already reuse two of the leaf comparators — but via white-box `#include "../src/sim_verify.c"` + direct calls to the `static` functions: `test_sim_equivalence.c` (lines 894, 902), `test_repl_equivalence.c` (lines 734, 742), and `test_repl_limit_table.c` (lines 2662, 2670). + +### Architecture Pattern & Boundary Map + +```mermaid +flowchart LR + subgraph scheduler["src/scheduler"] + SV["sim_verify.c\n(sim_decision_eq, sim_log_mismatch, describe_*,\nmain-reason-only PEND outcome check)"] + TE["test_sim_equivalence.c / test_repl_equivalence.c / test_repl_limit_table.c"] + TSV["test_sim_verify.c"] + end + subgraph libxce["src/lib + src/include"] + XA["xcealloc.{c,h}\nalloc_eq / host_alloc_eq / job_alloc_eq"] + PR["preason.{c,h}\njob_preason_eq (full match)"] + UT["utils.{c,h}\nstr_eq (NULL-safe)"] + TY["xce/types.h\n(struct defs, unchanged)"] + end + SV -->|"DISPATCH: job_alloc_eq (public API)"| XA + TE -.->|"calls (replaces white-box static calls)"| XA + TE -.->|"calls"| PR + TSV -->|"exercises via sim_decision_eq"| SV + XA --> UT + PR --> UT + XA --> TY + PR --> TY +``` + +**Architecture Integration**: +- Selected pattern: co-locate equality with the module that already owns each struct's construction and deep-copy — `alloc` family in `xcealloc.c` (mirrors `dup_job_alloc`), `job_preason_eq` in `preason.c` (mirrors `add_job_preason`). No new module is introduced. +- Domain boundary: the library owns *struct-level* equality (full structural match); the simulator owns *decision-level* comparison and reporting, including its one deliberate loosening (main-reason-only PEND). This is the clean seam: everything below it is libxce-typed and identity-honest, everything above it touches `replay_decision_t`/capnp and may narrow for replay reasons. +- Existing patterns preserved: public headers in `src/include/`, implementations in `src/lib/`, `const`-qualified read-only parameters, lowercase unprefixed function names (`make_*`/`dup_*`/`erase_*`). +- Steering compliance: no changes to `types.h` layouts, no new persistence surface, no HA/RBAC/persistence semantics touched. + +### Technology Stack + +| Layer | Choice / Version | Role in Feature | Notes | +|-------|------------------|-----------------|-------| +| Language | C (gnu17) | equality functions + tests | unchanged from libxce | +| Build | Meson via `xce-build` | compile libxce + run `src/lib/testsuite` | unchanged | +| Test | Unity | `test_xcealloc.c`, `test_preason.c` | existing libxce test harness | + +## File Structure Plan + +### Modified Files +- `src/lib/xcealloc.c` — add `alloc_eq`, `host_alloc_eq`, `job_alloc_eq` (using the shared `str_eq` helper). +- `src/include/xcealloc.h` — declare the three allocation equality functions. +- `src/lib/preason.c` — add `job_preason_eq` (full match, using the shared `str_eq` helper). +- `src/include/preason.h` — declare `job_preason_eq`. +- `src/lib/utils.c` / `src/include/utils.h` — add a NULL-safe `str_eq(const char*, const char*)` helper (see Decision 5). +- `src/scheduler/src/sim_verify.c` — delete the five `static` comparators; have `sim_decision_eq` call `job_alloc_eq` for DISPATCH and keep a simulator-local `static` main-reason-only helper for PEND. Keep `sim_log_mismatch` + describe helpers. +- `src/lib/testsuite/test_xcealloc.c` — add allocation equality unit tests. +- `src/lib/testsuite/test_preason.c` — add `job_preason_eq` full-match unit tests. +- `src/scheduler/testsuite/test_sim_equivalence.c`, `test_repl_equivalence.c`, `test_repl_limit_table.c` — replace direct calls to `sim_verify_job_alloc_eq`/`sim_verify_job_preason_eq` with `job_alloc_eq`/`job_preason_eq`; add `#include "xcealloc.h"`/`#include "preason.h"`. +- `src/scheduler/testsuite/test_sim_verify.c` — remove the DISPATCH alloc-level unit tests that now belong to libxce; keep the PEND/decision-level tests (they exercise `sim_decision_eq`'s simulator-local main-reason-only behavior) and the integration-level tests. + +## System Flows + +The comparison flow is unchanged from today — this feature only relocates the leaf logic. `sim_decision_eq` remains the entry point: + +```mermaid +flowchart TD + A["sim_decision_eq(expected, actual)"] --> B{"actual->op"} + B -->|DISPATCH| C["job_alloc_eq(expected->body.dispatch->alloc, actual->u.alloc)"] + B -->|PEND| D["jobid equal AND main_reason equal (sim-local, NOT job_preason_eq)"] + B -->|PREEMPT| E["log + report mismatch (unchanged)"] + C --> F["host_alloc_eq per host"] --> G["alloc_eq per resource (excl. from/instid)"] + D --> H["main-reason only, detail ignored (bug-1845)"] +``` + +## Requirements Traceability + +| Requirement | Summary | Components | Interfaces | +|-------------|---------|------------|------------| +| 1.1–1.3 | allocation equality for `xce_alloc_t`/`xce_host_alloc_t`/`xce_job_alloc_t` | `xcealloc.c` | `alloc_eq`, `host_alloc_eq`, `job_alloc_eq` | +| 1.4 | exclude `from`/`instid` | `xcealloc.c` | `alloc_eq` field list | +| 1.5–1.6 | NULL handling + read-only | all four functions | const params | +| 2.1–2.3 | pending-reason full structural equality | `preason.c` | `job_preason_eq` | +| 3.1–3.2, 3.5 | simulator behavior preserved | `sim_verify.c` | `sim_decision_eq` | +| 3.3 | simulator uses libxce allocation comparison | `sim_verify.c` | `job_alloc_eq` | +| 3.4 | simulator-local main-reason-only PEND check | `sim_verify.c` | local static helper | +| 4.1–4.2 | single shared implementation | `sim_verify.c` + 3 test files | public API | +| 4.3–4.4 | libxce unit tests + no regressions | `test_xcealloc.c`, `test_preason.c` | — | + +## Components and Interfaces + +### libxce + +#### Allocation equality (`xcealloc.c`) + +| Field | Detail | +|-------|--------| +| Intent | Public structural equality for the allocation structs | +| Requirements | 1.1–1.6 | + +```c +/* src/include/xcealloc.h */ +extern int alloc_eq(const xce_alloc_t* a, const xce_alloc_t* b); +extern int host_alloc_eq(const xce_host_alloc_t* a, const xce_host_alloc_t* b); +extern int job_alloc_eq(const xce_job_alloc_t* a, const xce_job_alloc_t* b); +``` + +- **Preconditions**: none (NULL-safe). +- **Postconditions**: returns non-zero iff the two arguments are equal per the documented contract; does not mutate either argument. +- **Invariants**: `alloc_eq(a, b)` ignores `from` and `instid`; `host_alloc_eq`/`job_alloc_eq` are order-sensitive (both sides come from the same deterministic construction, so array order is expected to match, not just multiset). + +#### Pending-reason equality (`preason.c`) + +| Field | Detail | +|-------|--------| +| Intent | Public full structural equality for `xce_job_preason_t` | +| Requirements | 2.1–2.3 | + +```c +/* src/include/preason.h */ +extern int job_preason_eq(const xce_job_preason_t* a, const xce_job_preason_t* b); +``` + +- **Postconditions**: returns non-zero iff `main_reason` matches, the host-based reason list matches in order (each entry by host name + reason), and the resource-based reason list matches in order (each entry by resource name + host name + reason); NULL-safe per the shared convention. +- **Invariants**: this is a full match — it is the honest "are these two pending reasons identical" predicate, not the simulator's replay loosening (see Decision 3). + +#### Shared string helper (`utils.c`) + +```c +/* src/include/utils.h */ +extern int str_eq(const char* a, const char* b); +``` + +- **Postconditions**: returns non-zero iff `a` and `b` are both NULL, or both non-NULL and `strcmp(a, b) == 0`; zero otherwise. + +## Data Models + +No struct is added or modified. The equality contract itself is the only new "model", captured in the field lists below. + +- `alloc_eq` compares: `value` (exact `==`, documented bit-exact-safe because both sides are exact copies of the same `double`), `duration`, `resname`, `reserve`. **Excluded**: `from` (wall-clock `time(NULL)` stamp), `instid` (per-process `nextid++` counter) — both runtime bookkeeping, not identity. +- `host_alloc_eq` compares: `host`, then `allocs[i]` in order for `i < nallocs`. +- `job_alloc_eq` compares: `jobid`, then `allocs[i]` in order for `i < nhosts`, then `allocs_shared[i]` in order for `i < nres_shared`. +- `job_preason_eq` compares: `main_reason`; then `host_based_reasons[i]` in order for `i < nhost_reasons`, each entry by `host` and `reason`; then `res_based_reasons[i]` in order for `i < nres_reasons`, each entry by `resource`, `host`, and `reason`. **Nothing is excluded** — this is a full structural match. + +## Error Handling + +These are pure read-only functions with no allocation, no I/O, and no failure path. The only "error" case is NULL input, which is handled by the shared convention (identical pointers including both-NULL → equal; exactly one NULL → not-equal) rather than an error return. No logging is added. + +## Testing Strategy + +- **Unit (libxce)**: `test_xcealloc.c` — match/mismatch on each compared field, the `from`/`instid` exclusion, ordered-list sensitivity, and NULL handling at top and nested levels; `test_preason.c` — full-match: main-reason match/mismatch, host-based reason list match/mismatch (per field + order), resource-based reason list match/mismatch (per field + order), and NULL handling. +- **Unit (scheduler)**: `test_sim_verify.c` keeps `sim_decision_eq`-level tests (DISPATCH/PEND match and mismatch, kind mismatch, PREEMPT always-false, NULL defense) and drops the DISPATCH alloc-level tests that now belong to libxce. The PEND tests stay because they pin the simulator-local main-reason-only behavior. +- **Integration (scheduler)**: `test_sim_verify.c`'s `--sim-verify` round-trips (zero-mismatch self-replay, exact-mismatch altered replay, reconcile cases) continue to assert unchanged behavior, proving the relocation is behavior-preserving. +- **Regression**: `test_sim_equivalence.c`, `test_repl_equivalence.c`, `test_repl_limit_table.c` must stay green after migrating to the public API. + +## Design Decisions + +### Decision 1: Co-locate equality with construction +- **Context**: where to put the four functions. +- **Alternatives**: a single new `xce_equality.{c,h}` module; or one function per struct's existing owner module. +- **Selected**: co-locate — `alloc` family in `xcealloc.c`, `job_preason_eq` in `preason.c`. +- **Rationale**: matches the existing precedent (`dup_job_alloc` lives beside `make_*` in `xcealloc.c`; `add_job_preason`/`make_pendreason` in `preason.c`); avoids a new module for four small functions. +- **Trade-offs**: none material. + +### Decision 2: Naming (`alloc_eq`, not `xce_alloc_eq`) +- **Context**: public API names. +- **Alternatives**: `xce_`-prefixed names; `_equals` suffix. +- **Selected**: `alloc_eq`/`host_alloc_eq`/`job_alloc_eq`/`job_preason_eq`. +- **Rationale**: follows the existing unprefixed lowercase convention (`make_alloc`, `dup_job_alloc`, `erase_job_preason`), with the `_eq` suffix as the standard C equality idiom. The `job_alloc`/`job_preason` stems match the struct names. +- **Trade-offs**: `alloc_eq` is slightly generic, but it is unambiguous in a header (`xcealloc.h`) that already owns the unprefixed `alloc` family. + +### Decision 3: `job_preason_eq` is a full structural match; the simulator keeps its own main-reason-only PEND check +- **Context**: the current `sim_verify_job_preason_eq` compares only `main_reason`, deliberately (bug-1845): the simulator has no alloc cache, so its full-detail PEND can never match a recorded cache-hit's main-reason-only shape, and requiring detail to match produced systematic false mismatches. That narrowing is a property of *recorded-vs-live replay comparison*, not of "are these two pending-reason objects identical". +- **Alternatives**: (a) publish a full-deep `job_preason_eq` and keep the main-reason-only comparison as a simulator-local static helper (selected); (b) publish main-reason-only `job_preason_eq` (rejected — leaks a replay artifact into a general-purpose equality API and contradicts the identity semantics a controller engineer would expect). +- **Selected**: `job_preason_eq(a, b)` returns non-zero iff `main_reason` matches and both detail lists match in order, field-for-field. The simulator's `sim_decision_eq` PEND branch keeps a small `static` main-reason-only helper (documented with the bug-1845 rationale) instead of calling `job_preason_eq`. +- **Rationale**: a public "same pending reason" predicate must be identity-honest — a controller comparing two pending reasons will want the detail to matter. The replay asymmetry is a scheduler-only concern and stays where it belongs. +- **Trade-offs**: two PEND-comparison shapes now exist (full-match in libxce, main-reason-only in sim_verify.c). This is intentional: they answer different questions ("identical objects" vs. "same replay outcome"). The sim-local helper is `static` and small, so the duplication is negligible and clearly scoped. +- **Follow-up**: none expected; the sim-local helper is not exported. + +### Decision 4: Preserve NULL and order semantics exactly +- **Context**: NULL handling and list-order sensitivity. +- **Selected**: keep today's behavior — identical pointer (including both NULL) → equal; exactly one NULL → not-equal; ordered (not multiset) list comparison. +- **Rationale**: the point of the relocation is zero behavioral change; both sides of every current comparison come from the same deterministic construction, so order-sensitivity is correct. The full-match `job_preason_eq` uses the same convention. + +### Decision 5: Promote the NULL-safe string comparison to a shared `str_eq` in `utils.{c,h}` +- **Context**: with `job_preason_eq` now a full match, both `xcealloc.c` (host/resname/reserve strings) and `preason.c` (host/resource strings) need NULL-safe string comparison. Keeping it `static` in one file forces the other to duplicate it. +- **Alternatives**: (a) shared `str_eq` in `utils.{c,h}` (selected); (b) duplicate a `static` helper in each of the two files. +- **Selected**: add `int str_eq(const char* a, const char* b)` to `utils.{c,h}`, beside the existing `float_equal` helper. +- **Rationale**: one consumer each would have justified a `static`; two consumers and a third on the horizon (controller) justify a shared helper. `utils.h` already centralizes small generic C helpers (`fmt_jobid`, `float_equal`). +- **Trade-offs**: a new public symbol in `utils.h`; negligible surface, and it will be reused by the controller when it adopts these comparators. diff --git a/.kiro/specs/libxce-alloc-preason-equality/requirements.md b/.kiro/specs/libxce-alloc-preason-equality/requirements.md new file mode 100644 index 00000000..6acf8d1c --- /dev/null +++ b/.kiro/specs/libxce-alloc-preason-equality/requirements.md @@ -0,0 +1,51 @@ +# Requirements Document + +## Introduction + +Engineers maintaining the XCE scheduler and controller — anyone who needs to determine whether two scheduler allocation or pending-reason objects represent the same thing — currently have no shared way to do so. The structural equality comparators for the shared data structures `xce_alloc_t`, `xce_host_alloc_t`, `xce_job_alloc_t`, and `xce_job_preason_t` (all defined in libxce's `src/include/xce/types.h`) live as `static` functions inside the scheduler's simulation-verification module `src/scheduler/src/sim_verify.c`, written for `--sim-verify` replay checking. Three other scheduler test files already reuse two of these comparators (`sim_verify_job_alloc_eq`/`sim_verify_job_preason_eq`) — but only by white-box `#include "../src/sim_verify.c"` and calling the `static` functions directly, a fragile mechanism that drags the entire simulator translation unit into each consumer. Meanwhile the controller already consumes these same shared structs (`persist.c`, `resync.c`, `job.c`) with no equality API available at all. This should change so that the type-level equality comparators for these shared structures become public libxce API — `alloc_eq`/`host_alloc_eq`/`job_alloc_eq` in `xcealloc.{c,h}` (the module that already constructs and deep-copies them) and `job_preason_eq` in `preason.{c,h}` (the module that already owns pending-reason construction) — so any component or test can compare these shared objects by linking libxce, while the decision-level dispatcher (`sim_decision_eq`) and mismatch reporting (`sim_log_mismatch`) that are genuinely simulator-specific stay in `sim_verify.c`. + +## Boundary Context + +- **In scope**: extracting the four type-level comparators (`xce_alloc_t`, `xce_host_alloc_t`, `xce_job_alloc_t`, `xce_job_preason_t`) from the scheduler simulator into public libxce API; giving the allocation comparators a documented "identity" contract (excluding `from`/`instid`) and the pending-reason comparator a full structural-match contract (all fields, detail lists included); migrating every in-repo consumer of these comparators (the simulator plus the scheduler test files that currently reuse the static functions) to the public API; libxce unit tests for the new functions. +- **Out of scope**: the decision-level comparison dispatcher (`sim_decision_eq`) and mismatch reporting (`sim_log_mismatch` and its describe helpers), which depend on `replay_decision_t`, capnp enums, and `ERROR()` logging and remain in the scheduler; any change to the *semantics* of what is compared (the deliberate field exclusions are preserved, not re-litigated); any new comparison call site beyond the existing consumers; any change to the controller's persistence or resync behavior; any change to the `.replay` wire format. +- **Adjacent expectations**: the `.replay` wire format and the `replay_decision_t`/`xce_decision_t` decision types are unchanged; libxce's existing constructors and deep-copy helpers (`make_alloc`, `make_host_alloc`, `make_job_alloc`, `dup_job_alloc`, `make_pendreason`) and the `xce/types.h` struct layouts are unchanged; the controller's Cap'n Proto persistence round-trips for these structures are unchanged. + +## Requirements + +### Requirement 1: Shared equality API for allocation structures +**Objective:** As a scheduler or controller engineer who needs to compare two allocation objects, I want a public libxce equality function for each allocation structure, so that I can compare them correctly by linking libxce rather than reimplementing the comparison or white-box-including scheduler source. + +#### Acceptance Criteria +1. The library shall provide a public equality function for `xce_alloc_t` that reports equal if and only if the two records have the same resource name, reserve, value, and duration. +2. The library shall provide a public equality function for `xce_host_alloc_t` that reports equal if and only if the two records have the same host name and, in order, the same per-resource `xce_alloc_t` records. +3. The library shall provide a public equality function for `xce_job_alloc_t` that reports equal if and only if the two records have the same job id, the same per-host `xce_host_alloc_t` records in order, and the same shared-resource `xce_alloc_t` records in order. +4. The allocation equality functions shall not consider a record's reservation start time (`from`) or allocation-instance id (`instid`) when determining equality, because these are runtime bookkeeping values, not part of an allocation's identity. +5. When the two arguments to an allocation equality function are the identical pointer (including two NULL pointers), the function shall report equal; when exactly one of the two arguments is NULL at the top level or at any nested level, the function shall report not-equal without crashing. +6. The allocation equality functions shall not modify either argument. + +### Requirement 2: Shared equality API for the pending-reason structure +**Objective:** As a scheduler or controller engineer who needs to compare two pending-reason objects, I want a public libxce equality function that reports a full structural match, so that I can determine whether two pending reasons are identical by linking libxce rather than reimplementing the comparison. + +#### Acceptance Criteria +1. The library shall provide a public equality function for `xce_job_preason_t` that reports equal if and only if the two records have the same main reason code, the same host-based reason entries in order, and the same resource-based reason entries in order. +2. The pending-reason equality function shall compare each host-based reason entry by its host name and reason code, and each resource-based reason entry by its resource name, host name, and reason code. +3. The pending-reason equality function shall handle NULL arguments the same way as the allocation equality functions (identical pointers including both NULL report equal; exactly one NULL at the top level or at any nested level reports not-equal) and shall not modify either argument. + +### Requirement 3: Simulator verification behavior is preserved +**Objective:** As an operator relying on `--sim-verify` replay checking, I want the simulator's decision verification to behave exactly as it did before this change, so that no new false mismatch or missed mismatch is introduced. + +#### Acceptance Criteria +1. The simulator shall continue to compare a recorded decision against a live decision using the same equality semantics as before this change. +2. The simulator shall continue to report a verification mismatch with the same logical-time and decision-detail context as before this change. +3. The simulator shall obtain its allocation comparison from the libxce public API rather than defining it itself. +4. The simulator shall keep its pending-decision comparison as a simulator-local main-reason-only check, because a recorded cache-hit pending reason carries no host/resource detail while a live replayed one always does — this documented asymmetry is unchanged by this feature and remains a simulator concern, not a property of the full-match libxce `job_preason_eq`. +5. Replaying a recording against its own previously-recorded output with verification enabled shall continue to report zero mismatches. + +### Requirement 4: Single shared implementation across all consumers +**Objective:** As a maintainer, I want the allocation and pending-reason comparisons to exist exactly once, in libxce, so that future changes to their semantics are made in one place and every consumer picks them up. + +#### Acceptance Criteria +1. Every in-repo consumer of these comparisons — the simulator and the scheduler test files that currently reuse the static comparators — shall obtain the allocation comparison and the full pending-reason comparison from the libxce public API; the simulator's main-reason-only pending-decision check is the single documented simulator-local exception. +2. A consumer that needs only the type-level comparisons shall be able to obtain them without white-box-including the simulator's verification source. +3. The library shall provide unit test coverage for each new equality function, covering a match, a mismatch on each compared field, and the NULL-handling cases. +4. Adding these functions shall not alter the behavior of any existing libxce function. diff --git a/.kiro/specs/libxce-alloc-preason-equality/research.md b/.kiro/specs/libxce-alloc-preason-equality/research.md new file mode 100644 index 00000000..d96f95b3 --- /dev/null +++ b/.kiro/specs/libxce-alloc-preason-equality/research.md @@ -0,0 +1,74 @@ +# Research & Design Decisions + +## Summary +- **Feature**: `libxce-alloc-preason-equality` +- **Discovery Scope**: Extension (relocate existing comparison logic, add public API, migrate consumers) +- **Key Findings**: + 1. The type-level equality comparators in `sim_verify.c` operate purely on libxce structs with zero scheduler/capnp/replay dependency — they are cleanly movable; the decision-level dispatcher and mismatch reporting are not. + 2. The comparators are already shared across four translation units, but via white-box `#include "../src/sim_verify.c"` + direct calls to `static` functions (`test_sim_equivalence.c`, `test_repl_equivalence.c`, `test_repl_limit_table.c`). + 3. The controller (`persist.c`, `resync.c`, `job.c`) already consumes `xce_job_alloc_t`/`xce_host_alloc_t`/`xce_job_preason_t`/`xce_alloc_t` with no equality API — confirming these structs are genuinely cross-component, not scheduler-only. + 4. One deliberate field exclusion is baked into the alloc comparators (`from`/`instid`), which generalizes cleanly into a shared identity contract. The preason comparator's main-reason-only narrowing is *not* an identity property — it is a replay artifact (bug-1845) that must stay in the simulator while the shared `job_preason_eq` becomes a full match. + +## Research Log + +### Where do the equality checks live? +- **Context**: locate all comparison logic in the scheduler simulator. +- **Sources Consulted**: `src/scheduler/src/sim_verify.c` (full read); `src/scheduler/src/sim_loop.c`, `sim_alloc.c`, `sim_picker.c`, `sim_limit.c` (grep for `_eq`/`_equal`/`compare`). +- **Findings**: `sim_verify.c` (474 lines) is the only file with equality logic. It holds five `static` leaf comparators plus `sim_decision_eq`, `sim_log_mismatch`, and two `describe_*` helpers. No other sim file compares these structs. +- **Implications**: the split point is exactly between the five `static` leaf functions and `sim_decision_eq`. + +### Who consumes these comparators, and how? +- **Context**: understand the "future sharing" concern. +- **Sources Consulted**: grep for `sim_verify_job_alloc_eq`/`sim_verify_job_preason_eq`/`sim_verify_alloc_eq` across `src/scheduler/testsuite/`. +- **Findings**: three test files call the two leaf comparators directly after white-box-including `sim_verify.c`: `test_sim_equivalence.c:894,902`, `test_repl_equivalence.c:734,742`, `test_repl_limit_table.c:2662,2670`. Each builds its own `replay_decision_content_eq`-style helper on top of the two leaves. +- **Implications**: moving the leaves to libxce turns a fragile include-and-call-static hack into a normal public API; these three files must migrate their call sites. + +### Are these structs shared beyond the scheduler? +- **Context**: justify libxce as the destination rather than "scheduler convenience". +- **Sources Consulted**: grep for the four struct names across `src/controller`, `src/worker`, `src/harness`, `src/lib`. +- **Findings**: `xce_job_alloc_t`/`xce_host_alloc_t`/`xce_job_preason_t`/`xce_alloc_t` are consumed by `src/controller/persist.c` (Cap'n Proto encode/decode), `src/controller/resync.c`, and `src/controller/job.c`, plus the scheduler. No equality API exists anywhere. +- **Implications**: the controller is a real future consumer of equality; the API belongs in libxce. + +### Where do the structs' construction/dup helpers already live? +- **Context**: choose the destination module. +- **Sources Consulted**: `src/lib/xcealloc.c` (read full), `src/include/xcealloc.h`, `src/lib/preason.c`, `src/include/preason.h`. +- **Findings**: `make_alloc`/`make_host_alloc`/`make_job_alloc`/`dup_job_alloc`/`make_job_account` in `xcealloc.c`; `erase_job_preason`/`add_job_preason`/`make_pendreason` in `preason.c`. No existing `_eq`/`_equal` convention in libxce (only `float_equal` in `utils.h`); no NULL-safe string-equality helper exists. +- **Implications**: co-location with construction is the natural home; the string helper is new and only needed by the alloc family today. + +## Architecture Pattern Evaluation + +| Option | Description | Strengths | Risks / Limitations | Notes | +|--------|-------------|-----------|---------------------|-------| +| Co-locate in existing modules | equality beside `make_*`/`dup_*`/`erase_*` | matches precedent, no new module | none material | selected | +| New `xce_equality.{c,h}` | one module for all four | single discoverable location | new module for 4 small functions | rejected (YAGNI) | +| Leave in scheduler, expose via header | promote the statics to scheduler-external | minimal | controller can't use them; wrong ownership | rejected | + +## Design Decisions + +(Full decisions live in `design.md`; the significant trade-offs are recapped here.) + +### Decision: `job_preason_eq` is a full structural match; the simulator keeps its own main-reason-only PEND check +- **Context**: `sim_verify_job_preason_eq` compares only `main_reason`, deliberately (bug-1845) — sim has no alloc cache, so its full detail can never match a recorded cache-hit's main-reason-only shape; requiring detail to match produced systematic false mismatches. That narrowing answers "same replay outcome", not "identical pending-reason objects". +- **Alternatives Considered**: + 1. Publish full-deep `job_preason_eq` + keep a `static` main-reason-only helper in `sim_verify.c` for the PEND branch (selected). + 2. Publish `job_preason_eq` as main-reason-only (rejected — leaks a replay artifact into a general-purpose equality API). +- **Selected Approach**: `job_preason_eq(a, b)` returns non-zero iff `main_reason` matches and both detail lists (host-based, resource-based) match in order field-for-field. `sim_decision_eq`'s PEND branch uses a small simulator-local `static` main-reason-only comparison instead of `job_preason_eq`. +- **Rationale**: a public "same pending reason" predicate must be identity-honest; the replay asymmetry is a scheduler-only concern and stays where it belongs. +- **Trade-offs**: two PEND-comparison shapes exist (full-match in libxce, main-reason-only in sim_verify.c) — intentional, they answer different questions, and the sim-local helper is `static` and small. +- **Follow-up**: none expected; the sim-local helper is not exported. + +### Decision: preserve NULL and order semantics exactly +- **Context**: the current leaf comparators use "identical pointer (including both NULL) → equal; exactly one NULL → not-equal", and ordered list comparison. +- **Selected Approach**: reproduce exactly in libxce. +- **Rationale**: the relocation is required to be zero-behavioral-change (Requirement 3); ordered comparison is correct because both sides come from the same deterministic construction. + +## Risks & Mitigations +- Risk: a consumer silently relies on the excluded fields (`from`/`instid`, preason detail) → Mitigation: explicit doc contract on each function + revalidation triggers; no behavior change for existing consumers. +- Risk: migration of the three white-box test files breaks their `replay_decision_content_eq` helpers → Mitigation: mechanical call-site rename, verified by the full scheduler suite (task 7). +- Risk: duplicate-symbol collisions if a test both white-box-includes `sim_verify.c` and links libxce after the statics are removed → Mitigation: the statics are deleted from `sim_verify.c` (task 4) before the test files migrate (task 5), so the names resolve only against libxce. + +## References +- `src/scheduler/src/sim_verify.c` — current home of the comparators (source of truth for field lists and exclusion rationale). +- `src/include/xce/types.h` — the four struct definitions. +- `src/lib/xcealloc.c`, `src/lib/preason.c` — destination modules and their construction/dup conventions. +- `.kiro/specs/scheduler-snapshot-job-allocation/` — precedent for a scheduler-driven spec adding libxce helpers (`dup_job_alloc`) to `xcealloc.c`. diff --git a/.kiro/specs/libxce-alloc-preason-equality/spec.json b/.kiro/specs/libxce-alloc-preason-equality/spec.json new file mode 100644 index 00000000..0e043321 --- /dev/null +++ b/.kiro/specs/libxce-alloc-preason-equality/spec.json @@ -0,0 +1,22 @@ +{ + "feature_name": "libxce-alloc-preason-equality", + "created_at": "2026-09-24T04:42:20Z", + "updated_at": "2026-09-24T05:12:00Z", + "language": "en", + "phase": "tasks-generated", + "approvals": { + "requirements": { + "generated": true, + "approved": false + }, + "design": { + "generated": true, + "approved": false + }, + "tasks": { + "generated": true, + "approved": false + } + }, + "ready_for_implementation": false +} diff --git a/.kiro/specs/libxce-alloc-preason-equality/tasks.md b/.kiro/specs/libxce-alloc-preason-equality/tasks.md new file mode 100644 index 00000000..8da64778 --- /dev/null +++ b/.kiro/specs/libxce-alloc-preason-equality/tasks.md @@ -0,0 +1,69 @@ +# Implementation Plan + +## Task Format + +Work is sequenced test-first where the deliverable is new library API (TDD: red → green → refactor), then the scheduler-side migration is a behavior-preserving refactor verified by the existing `--sim-verify` tests. + +--- + +- [ ] 1. Add a shared NULL-safe string helper to libxce (`utils.c`/`utils.h`) + - Add `int str_eq(const char* a, const char* b)` beside the existing `float_equal` helper: returns non-zero iff both arguments are NULL, or both non-NULL and `strcmp(a, b) == 0`. + - Document that it is the shared NULL-safe string-equality helper for the equality functions (and any future controller consumer). + - _Requirements: 1.1–1.6, 2.1–2.3 (supporting)_ + - _Boundary: utils.c, utils.h_ + +- [ ] 2. Add allocation equality API to libxce (`xcealloc.c`/`xcealloc.h`) + - Implement `alloc_eq(const xce_alloc_t*, const xce_alloc_t*)` comparing `value`, `duration`, `resname`, `reserve` (excluding `from`/`instid`), with the shared NULL convention. + - Implement `host_alloc_eq(const xce_host_alloc_t*, const xce_host_alloc_t*)` comparing `host` then `allocs[i]` in order. + - Implement `job_alloc_eq(const xce_job_alloc_t*, const xce_job_alloc_t*)` comparing `jobid`, `allocs[i]` in order, then `allocs_shared[i]` in order. + - Use `str_eq` from `utils.h` for the string fields. + - Declare all three in `src/include/xcealloc.h` with a per-function doc comment capturing the equality contract (including the `from`/`instid` exclusion and NULL/order semantics). + - _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6_ + - _Boundary: xcealloc.c, xcealloc.h_ + - _Depends: 1_ + +- [ ] 3. Add pending-reason equality API to libxce (`preason.c`/`preason.h`) + - Implement `job_preason_eq(const xce_job_preason_t*, const xce_job_preason_t*)` as a full structural match: `main_reason`, then `host_based_reasons[i]` in order (each by `host` via `str_eq` + `reason`), then `res_based_reasons[i]` in order (each by `resource` + `host` via `str_eq` + `reason`), with the shared NULL convention. + - Declare it in `src/include/preason.h` with a doc comment capturing the full-match contract (nothing excluded) and the NULL/order semantics. + - _Requirements: 2.1, 2.2, 2.3_ + - _Boundary: preason.c, preason.h_ + - _Depends: 1_ + +- [ ] 4. Add libxce unit tests for the new equality functions (TDD, before relying on them) + - `test_xcealloc.c`: `alloc_eq` match/mismatch per field (value, duration, resname, reserve), `from`/`instid` exclusion, `host_alloc_eq` host-name and order sensitivity, `job_alloc_eq` jobid + ordered hosts + ordered shared, NULL at top and nested levels. + - `test_preason.c`: `job_preason_eq` full match — main-reason match/mismatch, host-based list match/mismatch per field and order, resource-based list match/mismatch per field and order, NULL handling; plus `str_eq` NULL/NULL and NULL/non-NULL cases. + - Confirm `src/lib/testsuite` builds and passes (`/xce-build lib`). + - _Requirements: 4.3, 4.4_ + - _Boundary: test_xcealloc.c, test_preason.c_ + - _Depends: 2, 3_ + +- [ ] 5. Migrate the simulator to the public API (`sim_verify.c`) + - Delete the five `static` comparators (`sim_verify_str_eq`, `sim_verify_alloc_eq`, `sim_verify_host_alloc_eq`, `sim_verify_job_alloc_eq`, `sim_verify_job_preason_eq`). + - Add a new `static` simulator-local `sim_verify_pend_outcome_eq(const xce_job_preason_t*, const xce_job_preason_t*)` (or an inline check) that compares `main_reason` only, documented with the bug-1845 rationale. + - In `sim_decision_eq`, replace the DISPATCH payload comparison with `job_alloc_eq(...)` and the PEND payload comparison with `jobid` equality plus the local main-reason-only check. + - Add `#include "xcealloc.h"` and `#include "preason.h"`; keep `sim_log_mismatch` and the `describe_*` helpers unchanged. + - _Requirements: 3.1, 3.2, 3.3, 3.4_ + - _Boundary: sim_verify.c_ + - _Depends: 2, 3_ + +- [ ] 6. Migrate the three white-box test consumers to the public API + - `test_sim_equivalence.c`, `test_repl_equivalence.c`, `test_repl_limit_table.c`: replace direct calls to `sim_verify_job_alloc_eq` → `job_alloc_eq` and `sim_verify_job_preason_eq` → `job_preason_eq`; add the two headers. + - Where a file no longer needs `sim_decision_eq`/`sim_log_mismatch`, drop its `#include "../src/sim_verify.c"`; otherwise keep it for the decision-level functions only. + - Note: these fixtures produce zero-detail PEND reasons on both sides, so the switch from main-reason-only to full-match `job_preason_eq` is behavior-preserving for them; verify with the full scheduler suite. + - _Requirements: 4.1, 4.2_ + - _Boundary: test_sim_equivalence.c, test_repl_equivalence.c, test_repl_limit_table.c_ + - _Depends: 5_ + +- [ ] 7. Reconcile `test_sim_verify.c` with the relocated comparators + - Remove the DISPATCH alloc-level unit tests that only re-prove the allocation comparators (identical-content match, per-field mismatch, `from`/`instid` exclusion) — these are now owned by libxce tests from task 4. + - Keep the PEND unit tests (identical-content match, jobid/main-reason mismatch, detail-ignored) — these now pin `sim_decision_eq`'s simulator-local main-reason-only behavior, not a libxce function. + - Keep the `sim_decision_eq`-level tests (kind mismatch, PREEMPT always-false, NULL defense) and all integration-level `--sim-verify` round-trip tests, which now prove the relocation is behavior-preserving. + - _Requirements: 3.1, 3.4, 4.3_ + - _Boundary: test_sim_verify.c_ + - _Depends: 5_ + +- [ ] 8. Full build + test verification + - `/xce-build lib` (debug + ASAN + tests) green. + - `/xce-build scheduler` (debug + ASAN + tests) green, including `test_sim_verify.c`, `test_sim_equivalence.c`, `test_repl_equivalence.c`, `test_repl_limit_table.c`. + - _Requirements: 3.4, 4.4_ + - _Depends: 4, 5, 6, 7_ diff --git a/.wolf/anatomy-index.json b/.wolf/anatomy-index.json index cf16bc57..df51b05c 100644 --- a/.wolf/anatomy-index.json +++ b/.wolf/anatomy-index.json @@ -1,13 +1,13 @@ { "version": 1, "meta": { - "lastScanned": "2026-09-23T02:33:52.263Z", - "fileCount": 683, + "lastScanned": "2026-09-24T08:19:48.605Z", + "fileCount": 697, "hits": 0, "misses": 0, - "renderedHash": "b057ab996dad903a1362cecdf33c671ae71dc917e01a294cf69f58208783b3fc", - "storeUpdatedAt": "2026-09-23T02:33:52.274Z", - "rootHash": "d5ef97d98b68db51" + "renderedHash": "874f53e000496ecda4b198ce3b58146a20309f7f3739dd5c8f2768596b12c1b4", + "storeUpdatedAt": "2026-09-24T08:19:48.613Z", + "rootHash": "a8b88b2c1c5b1022" }, "files": { ".agent-shell/transcripts/2026-08-11-11-39-59.md": { @@ -9809,11 +9809,11 @@ }, "src/scheduler/src/plugins/seqjob_allocator.c": { "description": "include ", - "tokens": 4054, - "hash": "178eeda9d459716f", - "size": 15201, - "mtimeMs": 1789799899379.4854, - "updatedAt": "2026-09-19T06:38:19.414Z", + "tokens": 4148, + "hash": "19ae618e97a55bd3", + "size": 15555, + "mtimeMs": 1790174753187.611, + "updatedAt": "2026-09-23T14:45:53.231Z", "source": "hook" }, "src/scheduler/src/plugins/parjob_allocator.c": { @@ -9953,11 +9953,11 @@ }, "src/scheduler/src/sim_loop.c": { "description": "include ", - "tokens": 23299, - "hash": "667da1ed4512bed0", - "size": 87369, - "mtimeMs": 1789990380544.1218, - "updatedAt": "2026-09-21T11:33:00.598Z", + "tokens": 24467, + "hash": "784a7c5665ef656b", + "size": 91751, + "mtimeMs": 1790175755405.0996, + "updatedAt": "2026-09-23T15:02:35.446Z", "source": "hook" }, "src/scheduler/testsuite/test_sim_reconcile_unaccounted.c": { @@ -10133,20 +10133,20 @@ }, ".claude/skills/xce-test-ha/run-ha-soak.sh": { "description": "run-ha-soak.sh — driver for /xce-test-ha: deploy a 3-node XCE HA cluster, then", - "tokens": 51606, - "hash": "b1e6808537161344", - "size": 196026, - "mtimeMs": 1790077156253.1445, - "updatedAt": "2026-09-22T11:39:16.297Z", + "tokens": 53533, + "hash": "3520b5a8a5caae10", + "size": 203338, + "mtimeMs": 1790235443238.6655, + "updatedAt": "2026-09-24T07:37:23.285Z", "source": "hook" }, ".claude/skills/xce-test-ha/SKILL.md": { "description": "XCE HA Failover Soak Test", - "tokens": 10827, - "hash": "f2649f6350937e0f", - "size": 43609, - "mtimeMs": 1790072383701.4255, - "updatedAt": "2026-09-22T10:19:43.755Z", + "tokens": 11215, + "hash": "86321b7a628581a6", + "size": 45162, + "mtimeMs": 1790237988560.2532, + "updatedAt": "2026-09-24T08:19:48.604Z", "source": "hook" }, ".claude/skills/xce-deploy-ha/SKILL.md": { @@ -10313,11 +10313,11 @@ }, "src/scheduler/src/handler.c": { "description": "include ", - "tokens": 5162, - "hash": "d149b77df5b2f981", - "size": 19355, - "mtimeMs": 1789460932712.6296, - "updatedAt": "2026-09-15T08:28:52.774Z", + "tokens": 5287, + "hash": "916b6635215e1bd1", + "size": 19826, + "mtimeMs": 1790225370209.1118, + "updatedAt": "2026-09-24T04:49:30.260Z", "source": "hook" }, "src/scheduler/src/driver.c": { @@ -11440,11 +11440,11 @@ }, "src/scheduler/src/plugins/limit.c": { "description": "include ", - "tokens": 6902, - "hash": "9697079f6613c566", - "size": 25880, - "mtimeMs": 1789905009654.844, - "updatedAt": "2026-09-20T11:50:09.699Z", + "tokens": 7158, + "hash": "1f708808ade18bfa", + "size": 26839, + "mtimeMs": 1790222899534.5757, + "updatedAt": "2026-09-24T04:08:19.576Z", "source": "hook" }, "src/scheduler/testsuite/test_limit_plugin.c": { @@ -11467,11 +11467,11 @@ }, "src/scheduler/testsuite/meson.build": { "description": "Declares vtask_entry", - "tokens": 32032, - "hash": "9cb6195a49fec0b7", - "size": 120125, - "mtimeMs": 1789828249059.953, - "updatedAt": "2026-09-19T14:30:49.107Z", + "tokens": 32247, + "hash": "2a299060b2ea97ec", + "size": 120931, + "mtimeMs": 1790150699438.6106, + "updatedAt": "2026-09-23T08:04:59.493Z", "source": "hook" }, "src/scheduler/src/repl.c": { @@ -11593,11 +11593,11 @@ }, "src/scheduler/src/main.c": { "description": "include ", - "tokens": 6375, - "hash": "7229a52c42d1a331", - "size": 23905, - "mtimeMs": 1789722185119.4016, - "updatedAt": "2026-09-18T09:03:05.155Z", + "tokens": 6480, + "hash": "f8ced02d6ffc2baa", + "size": 24298, + "mtimeMs": 1790225372395.1309, + "updatedAt": "2026-09-24T04:49:32.444Z", "source": "hook" }, "src/scheduler/testsuite/test_alloc_cache.c": { @@ -11656,11 +11656,11 @@ }, "src/scheduler/testsuite/test_snapshot_save_load.c": { "description": "", - "tokens": 29478, - "hash": "93bd3a220116e1f1", - "size": 110542, - "mtimeMs": 1789727790161.5889, - "updatedAt": "2026-09-18T10:36:30.206Z", + "tokens": 30184, + "hash": "7c91b437663314ea", + "size": 113190, + "mtimeMs": 1790160142535.5137, + "updatedAt": "2026-09-23T10:42:22.592Z", "source": "hook" }, "src/scheduler/testsuite/test_seqjob_rewrap_equivalence.c": { @@ -12219,20 +12219,20 @@ }, "src/scheduler/testsuite/test_sim_verify.c": { "description": "Declares mismatch", - "tokens": 19394, - "hash": "7fa136721cf382f0", - "size": 72727, - "mtimeMs": 1790130818284.3984, - "updatedAt": "2026-09-23T02:33:38.334Z", + "tokens": 20492, + "hash": "f691eb4dccf2e35b", + "size": 76842, + "mtimeMs": 1790159179243.261, + "updatedAt": "2026-09-23T10:26:19.295Z", "source": "hook" }, "src/scheduler/src/sim_verify.c": { "description": "include ", - "tokens": 4975, - "hash": "e1a321f2179b0f34", - "size": 18653, - "mtimeMs": 1790130791318.6594, - "updatedAt": "2026-09-23T02:33:11.369Z", + "tokens": 5772, + "hash": "efdc82215a4bbbbc", + "size": 21645, + "mtimeMs": 1790174047438.49, + "updatedAt": "2026-09-23T14:34:07.484Z", "source": "hook" }, "src/scheduler/src/core.c": { @@ -12369,6 +12369,132 @@ "mtimeMs": 1790072281493.8599, "updatedAt": "2026-09-22T10:18:01.537Z", "source": "hook" + }, + "src/scheduler/src/recording_session.c": { + "description": "include ", + "tokens": 3559, + "hash": "85b39da9cf68e84d", + "size": 13344, + "mtimeMs": 1790150526198.198, + "updatedAt": "2026-09-23T08:02:06.245Z", + "source": "hook" + }, + "src/scheduler/include/recording.h": { + "description": "if !defined(_RECORDING_H_)", + "tokens": 6233, + "hash": "b1bc8141f990c30b", + "size": 23373, + "mtimeMs": 1790150567535.4485, + "updatedAt": "2026-09-23T08:02:47.581Z", + "source": "hook" + }, + "src/scheduler/testsuite/test_record_lifecycle.c": { + "description": "", + "tokens": 16263, + "hash": "0c4e5d5c65a4a335", + "size": 60983, + "mtimeMs": 1790236420330.391, + "updatedAt": "2026-09-24T07:53:40.374Z", + "source": "hook" + }, + "src/scheduler/src/snapshot_load.c": { + "description": "include ", + "tokens": 6646, + "hash": "5b0971eea3c6834b", + "size": 24920, + "mtimeMs": 1790231697348.8787, + "updatedAt": "2026-09-24T06:34:57.383Z", + "source": "hook" + }, + "src/scheduler/testsuite/test_limit_host_dedup.c": { + "description": "Declares char", + "tokens": 2231, + "hash": "5845a2556678e293", + "size": 8365, + "mtimeMs": 1790222853646.2876, + "updatedAt": "2026-09-24T04:07:33.691Z", + "source": "hook" + }, + ".kiro/specs/libxce-alloc-preason-equality/spec.json": { + "description": "", + "tokens": 133, + "hash": "476d73dec2289f2c", + "size": 463, + "mtimeMs": 1790231805754.1677, + "updatedAt": "2026-09-24T06:36:45.793Z", + "source": "hook" + }, + ".kiro/specs/libxce-alloc-preason-equality/requirements.md": { + "description": "Requirements Document", + "tokens": 2086, + "hash": "f457b66f320e827e", + "size": 8360, + "mtimeMs": 1790231529002.555, + "updatedAt": "2026-09-24T06:32:09.049Z", + "source": "hook" + }, + ".kiro/specs/libxce-alloc-preason-equality/design.md": { + "description": "Design Document", + "tokens": 4885, + "hash": "ac3b7f48d69ade69", + "size": 19617, + "mtimeMs": 1790231696452.8503, + "updatedAt": "2026-09-24T06:34:56.506Z", + "source": "hook" + }, + ".kiro/specs/libxce-alloc-preason-equality/tasks.md": { + "description": "Implementation Plan", + "tokens": 1445, + "hash": "16639cfbd9c66d25", + "size": 5797, + "mtimeMs": 1790231768606.78, + "updatedAt": "2026-09-24T06:36:08.649Z", + "source": "hook" + }, + ".kiro/specs/libxce-alloc-preason-equality/research.md": { + "description": "Research & Design Decisions", + "tokens": 1953, + "hash": "cdfa2fd8174acc8c", + "size": 7841, + "mtimeMs": 1790231733112.0073, + "updatedAt": "2026-09-24T06:35:33.164Z", + "source": "hook" + }, + "src/scheduler/testsuite/test_config_push_deferred.c": { + "description": "", + "tokens": 1903, + "hash": "3e0a32478f0e1d11", + "size": 7136, + "mtimeMs": 1790225291386.73, + "updatedAt": "2026-09-24T04:48:11.422Z", + "source": "hook" + }, + "src/lib/client_failover.c": { + "description": "define _GNU_SOURCE", + "tokens": 4507, + "hash": "388237d7bfd5d5c5", + "size": 18404, + "mtimeMs": 1790233455309.3013, + "updatedAt": "2026-09-24T07:04:15.467Z", + "source": "hook" + }, + "src/controller/testsuite/test_worker_join.c": { + "description": "include ", + "tokens": 15943, + "hash": "c2f86b96064af183", + "size": 60445, + "mtimeMs": 1790235651106.0027, + "updatedAt": "2026-09-24T07:40:51.143Z", + "source": "hook" + }, + "src/controller/worker.c": { + "description": "include ", + "tokens": 10434, + "hash": "8753ac8ab104f5b8", + "size": 39134, + "mtimeMs": 1790235703661.0974, + "updatedAt": "2026-09-24T07:41:43.705Z", + "source": "hook" } } } \ No newline at end of file diff --git a/.wolf/anatomy.md b/.wolf/anatomy.md index ce96f65c..2ca41fc3 100644 --- a/.wolf/anatomy.md +++ b/.wolf/anatomy.md @@ -1,7 +1,7 @@ # anatomy.md -> Auto-maintained by OpenWolf. Last scanned: 2026-09-23T02:33:52.263Z -> Files: 683 tracked | Anatomy hits: 0 | Misses: 0 +> Auto-maintained by OpenWolf. Last scanned: 2026-09-24T08:19:48.605Z +> Files: 697 tracked | Anatomy hits: 0 | Misses: 0 ## ./ @@ -35,8 +35,8 @@ ## .claude/skills/xce-test-ha/ -- `run-ha-soak.sh` — run-ha-soak.sh — driver for /xce-test-ha: deploy a 3-node XCE HA cluster, then (~51606 tok) -- `SKILL.md` — XCE HA Failover Soak Test (~10827 tok) +- `run-ha-soak.sh` — run-ha-soak.sh — driver for /xce-test-ha: deploy a 3-node XCE HA cluster, then (~53533 tok) +- `SKILL.md` — XCE HA Failover Soak Test (~11215 tok) - `soak-verdict-selftest.sh` — soak-verdict-selftest.sh — deterministic unit test for the verdict parser in (~2867 tok) - `soak-verdict.sh` — soak-verdict.sh — pure (side-effect-free) verdict helpers for run-ha-soak.sh. (~3084 tok) @@ -356,6 +356,14 @@ - `spec.json` (~444 tok) - `tasks.md` — Implementation Plan: job-array-single-wal-record (~15580 tok) +## .kiro/specs/libxce-alloc-preason-equality/ + +- `design.md` — Design Document (~4885 tok) +- `requirements.md` — Requirements Document (~2086 tok) +- `research.md` — Research & Design Decisions (~1953 tok) +- `spec.json` (~133 tok) +- `tasks.md` — Implementation Plan (~1445 tok) + ## .kiro/specs/libxce-failover/ - `brief.md` — Brief: libxce-failover (~2238 tok) @@ -899,6 +907,7 @@ - `meson.build` (~3405 tok) - `persist.c` — define _GNU_SOURCE (~109748 tok) - `resync.c` — here: construction (~11276 tok) +- `worker.c` — include (~10434 tok) ## src/controller/testsuite/ @@ -912,6 +921,7 @@ - `test_ha_role.c` — include (~13998 tok) - `test_ha_vep_teardown.c` (~8206 tok) - `test_rcfile_integration.sh` — test_rcfile_integration.sh — xce-test-rc-file integration test, shell-driven (~4117 tok) +- `test_worker_join.c` — include (~15943 tok) - `test_worker_resync.c` — Declares that (~56586 tok) ## src/harness/ @@ -942,6 +952,7 @@ ## src/lib/ +- `client_failover.c` — define _GNU_SOURCE (~4507 tok) - `meson.build` (~2428 tok) - `misc.c` — include (~486 tok) - `mylock.c` — include (~1632 tok) @@ -975,6 +986,7 @@ - `core.h` — if !defined(_CORE_H_) (~11793 tok) - `plugin.h` — if !defined(_PLUGIN_H_) (~5194 tok) +- `recording.h` — if !defined(_RECORDING_H_) (~6233 tok) - `scheduler.h` — if !defined(_SCHEDULER_H_) (~6140 tok) - `sim.h` — if !defined(_SIM_H_) (~4579 tok) @@ -990,10 +1002,11 @@ - `core_events.c` — include (~16984 tok) - `core.c` — include (~4495 tok) - `driver.c` — include (~11968 tok) -- `handler.c` — include (~5162 tok) +- `handler.c` — include (~5287 tok) - `host_iter.c` — Declares entry (~1718 tok) -- `main.c` — include (~6375 tok) +- `main.c` — include (~6480 tok) - `plugin.c` — include (~4692 tok) +- `recording_session.c` — include (~3559 tok) - `repl_break.c` — include (~9921 tok) - `repl_fmt.c` — include (~3489 tok) - `repl_info.c` — include (~21762 tok) @@ -1005,20 +1018,22 @@ - `scheduler.c` — include (~17581 tok) - `sim_alloc.c` — include (~2585 tok) - `sim_limit.c` — include (~2102 tok) -- `sim_loop.c` — include (~23299 tok) -- `sim_verify.c` — include (~4975 tok) +- `sim_loop.c` — include (~24467 tok) +- `sim_verify.c` — include (~5772 tok) - `sim.c` — include (~7432 tok) +- `snapshot_load.c` — include (~6646 tok) ## src/scheduler/src/plugins/ -- `limit.c` — include (~6902 tok) +- `limit.c` — include (~7158 tok) - `parjob_allocator.c` — include (~8014 tok) -- `seqjob_allocator.c` — include (~4054 tok) +- `seqjob_allocator.c` — include (~4148 tok) ## src/scheduler/testsuite/ -- `meson.build` — Declares vtask_entry (~32032 tok) +- `meson.build` — Declares vtask_entry (~32247 tok) - `test_alloc_cache.c` — Declares char (~6169 tok) +- `test_config_push_deferred.c` (~1903 tok) - `test_core_events.c` — Declares and (~19790 tok) - `test_core_isolation.c` — Declares char (~4492 tok) - `test_driver_translate.c` — Declares field (~11757 tok) @@ -1026,11 +1041,13 @@ - `test_job_array_expand.c` (~3393 tok) - `test_job_array_wire_e2e.c` — Declares vtask_entry (~4026 tok) - `test_job_update.c` — Declares char (~3637 tok) +- `test_limit_host_dedup.c` — Declares char (~2231 tok) - `test_limit_plugin.c` — Declares heap (~4475 tok) - `test_parjob_combinatorial.c` — Declares of (~12416 tok) - `test_picker_config_fields.c` — Declares char (~3358 tok) - `test_picker_core_loop_boundary.c` — Declares body (~2731 tok) - `test_plugin_hook_dispatch.c` — Declares char (~3531 tok) +- `test_record_lifecycle.c` (~16263 tok) - `test_repl_breakpoints_inspection.c` — Declares and (~23533 tok) - `test_repl_equivalence.c` (~9975 tok) - `test_repl_help.c` — Declares error (~6047 tok) @@ -1055,8 +1072,8 @@ - `test_sim_resource_contention.c` — Declares is (~10149 tok) - `test_sim_round_boundary_reconciliation.c` (~9980 tok) - `test_sim_session_ordering.c` (~10531 tok) -- `test_sim_verify.c` — Declares mismatch (~19394 tok) -- `test_snapshot_save_load.c` (~29478 tok) +- `test_sim_verify.c` — Declares mismatch (~20492 tok) +- `test_snapshot_save_load.c` (~30184 tok) - `test.c` — define UNIT_TEST 1 (~7044 tok) ## src/schema/ diff --git a/.wolf/memory.md b/.wolf/memory.md index 5627ae39..2c0930bd 100644 --- a/.wolf/memory.md +++ b/.wolf/memory.md @@ -5730,3 +5730,115 @@ | 19:33 | Edited src/scheduler/testsuite/test_sim_verify.c | 2→3 lines | ~57 | | 19:33 | Edited src/scheduler/testsuite/test_sim_live_allocation.c | modified entirely() | ~259 | | 20:24 | Session end: 4 writes across 3 files (sim_verify.c, test_sim_verify.c, test_sim_live_allocation.c) | 2 reads | ~12989 tok | + +## Session: 2026-09-23 00:14 + +| Time | Action | File(s) | Outcome | ~Tokens | +|------|--------|---------|---------|--------| +| 00:24 | Edited src/scheduler/src/sim_loop.c | added 2 condition(s) | ~224 | +| 00:25 | Edited src/scheduler/src/sim_loop.c | added 2 condition(s) | ~315 | +| 00:25 | Edited src/scheduler/src/sim_loop.c | modified if() | ~206 | +| 00:25 | Edited src/scheduler/src/sim_loop.c | modified for() | ~237 | +| 00:36 | Edited src/scheduler/src/sim_loop.c | modified for() | ~142 | +| 00:36 | Edited src/scheduler/src/sim_loop.c | modified if() | ~105 | +| 01:00 | Edited src/scheduler/src/recording_session.c | expanded (+20 lines) | ~422 | +| 01:01 | Edited src/scheduler/src/recording_session.c | 4→5 lines | ~27 | +| 01:01 | Edited src/scheduler/src/recording_session.c | modified if() | ~216 | +| 01:01 | Edited src/scheduler/src/recording_session.c | added 1 condition(s) | ~481 | +| 01:02 | Edited src/scheduler/src/recording_session.c | 2→3 lines | ~16 | +| 01:02 | Edited src/scheduler/include/recording.h | expanded (+9 lines) | ~316 | +| 01:02 | Edited src/scheduler/include/recording.h | 3→5 lines | ~84 | +| 01:02 | Edited src/scheduler/include/recording.h | expanded (+8 lines) | ~202 | +| 01:04 | Edited src/scheduler/testsuite/meson.build | expanded (+12 lines) | ~270 | +| 01:06 | Edited src/scheduler/testsuite/test_record_lifecycle.c | added 2 condition(s) | ~407 | +| 01:07 | Edited src/scheduler/testsuite/test_record_lifecycle.c | modified cause() | ~1170 | +| 01:07 | Edited src/scheduler/testsuite/test_record_lifecycle.c | 2→3 lines | ~41 | +| 02:25 | Edited src/scheduler/src/sim_verify.c | added 1 condition(s) | ~265 | +| 02:25 | Edited src/scheduler/src/sim_loop.c | added 2 condition(s) | ~331 | +| 02:26 | Edited src/scheduler/src/sim_loop.c | added 1 condition(s) | ~162 | +| 02:34 | Edited src/scheduler/src/sim_verify.c | modified sim_verify_alloc_eq() | ~95 | +| 02:35 | Edited src/scheduler/src/sim_verify.c | modified excluded() | ~560 | +| 02:36 | Edited src/scheduler/testsuite/test_sim_verify.c | modified test_dispatch_decisions_differing_instid_still_match() | ~506 | +| 02:36 | Edited src/scheduler/testsuite/test_sim_verify.c | 1→2 lines | ~39 | +| 03:24 | Edited src/scheduler/src/sim_verify.c | modified sim_verify_job_preason_eq() | ~546 | +| 03:25 | Edited src/scheduler/testsuite/test_sim_verify.c | modified test_pend_decisions_differing_reason_detail_still_match() | ~588 | +| 03:26 | Edited src/scheduler/testsuite/test_sim_verify.c | 1→2 lines | ~37 | +| 03:28 | Edited src/scheduler/src/snapshot_load.c | modified snapshot_build_primary_schedobj() | ~431 | +| 03:29 | Edited src/scheduler/src/snapshot_load.c | 1846 → 1772 | ~19 | +| 03:41 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | modified fix() | ~806 | +| 03:42 | Edited src/scheduler/testsuite/test_snapshot_save_load.c | 1→2 lines | ~43 | +| 07:03 | Edited src/scheduler/src/sim_loop.c | added 4 condition(s) | ~463 | +| 07:04 | Edited src/scheduler/src/sim_loop.c | added 2 condition(s) | ~444 | +| 07:04 | Edited src/scheduler/src/sim_loop.c | added 1 condition(s) | ~244 | +| 07:11 | Edited src/scheduler/src/sim_loop.c | 6→8 lines | ~139 | +| 07:18 | Edited src/scheduler/src/sim_loop.c | get_resvals() → zarray_get() | ~188 | +| 07:18 | Edited src/scheduler/src/sim_loop.c | removed 3 lines | ~5 | +| 07:26 | Edited src/scheduler/src/sim_verify.c | added 1 condition(s) | ~258 | +| 07:33 | Edited src/scheduler/src/sim_verify.c | modified if() | ~172 | +| 07:34 | Edited src/scheduler/src/sim_verify.c | modified for() | ~268 | +| 07:45 | Edited src/scheduler/src/plugins/seqjob_allocator.c | modified if() | ~197 | +| 07:45 | Edited src/scheduler/src/plugins/seqjob_allocator.c | 5→6 lines | ~94 | +| 08:02 | Edited src/scheduler/src/sim_loop.c | modified for() | ~165 | + +## Session: 2026-09-24 20:41 + +| Time | Action | File(s) | Outcome | ~Tokens | +|------|--------|---------|---------|--------| + +## Session: 2026-09-24 12:06 + +| Time | Action | File(s) | Outcome | ~Tokens | +|------|--------|---------|---------|--------| +| 21:07 | Created src/scheduler/testsuite/test_limit_host_dedup.c | — | ~2231 | +| 21:08 | Edited src/scheduler/src/plugins/limit.c | 7→7 lines | ~49 | +| 21:08 | Edited src/scheduler/src/plugins/limit.c | added 1 condition(s) | ~282 | + +## Session: 2026-09-24 12:13 + +| Time | Action | File(s) | Outcome | ~Tokens | +|------|--------|---------|---------|--------| +| 21:30 | bug-1846 part 1: dedup host-limit vres in match_job_host_limit_ (hlimits accumulated 1 copy per attempt → N-fold limit charge, live+sim); new sched_limit_host_dedup_test; truncated live rounds (part 2) still open | limit.c, test_limit_host_dedup.c, meson.build, buglog.json | 86/86 scheduler tests green | ~0 | +| 12:42 | Created .kiro/specs/libxce-alloc-preason-equality/spec.json | — | ~132 | +| 12:43 | Created .kiro/specs/libxce-alloc-preason-equality/requirements.md | — | ~2063 | +| 12:44 | Created .kiro/specs/libxce-alloc-preason-equality/design.md | — | ~4125 | +| 12:45 | Created .kiro/specs/libxce-alloc-preason-equality/tasks.md | — | ~1189 | +| 12:46 | Created .kiro/specs/libxce-alloc-preason-equality/research.md | — | ~1950 | +| 12:46 | Created .kiro/specs/libxce-alloc-preason-equality/spec.json | — | ~133 | +| 21:48 | Created src/scheduler/testsuite/test_config_push_deferred.c | — | ~1903 | +| 21:49 | Edited src/scheduler/src/handler.c | expanded (+6 lines) | ~199 | +| 21:49 | Edited src/scheduler/src/main.c | modified while() | ~124 | +| 22:10 | bug-1846 part 2: config push deferred to round edge (handler stashes via sched_stash_config, main loop applies via sched_apply_pending_config) -- mid-round rejoin config push was resetting fcfs cursor and truncating live rounds | handler.c, main.c, scheduler.c/.h, test_config_push_deferred.c, test_handlers.c (+3 stub files), meson.build, buglog.json | 87/87 scheduler tests green, uncommitted | ~0 | +| 14:32 | Edited .kiro/specs/libxce-alloc-preason-equality/requirements.md | inline fix | ~157 | +| 14:32 | Edited .kiro/specs/libxce-alloc-preason-equality/requirements.md | modified that() | ~308 | +| 14:32 | Edited .kiro/specs/libxce-alloc-preason-equality/requirements.md | 5→6 lines | ~248 | +| 14:32 | Edited .kiro/specs/libxce-alloc-preason-equality/requirements.md | 5→5 lines | ~208 | +| 23:33 | Edited src/scheduler/src/snapshot_load.c | modified if() | ~179 | +| 14:34 | Created .kiro/specs/libxce-alloc-preason-equality/design.md | — | ~5210 | +| 23:34 | Edited src/scheduler/src/snapshot_load.c | 3→4 lines | ~75 | +| 14:35 | Edited .kiro/specs/libxce-alloc-preason-equality/research.md | inline fix | ~95 | +| 14:35 | Edited .kiro/specs/libxce-alloc-preason-equality/research.md | 9→9 lines | ~398 | +| 14:36 | Created .kiro/specs/libxce-alloc-preason-equality/tasks.md | — | ~1542 | +| 14:36 | Edited .kiro/specs/libxce-alloc-preason-equality/spec.json | inline fix | ~12 | +| 22:45 | open-issues pass: config push now recorded (bug-1847), hlimits list leak fixed (bug-1848), benign picker WARN->TRACE (bug-1849), ~100s reconnect trigger analysed (bug-1850, open, libxce failover) | scheduler.c, schedobj.c, core_events.c, snapshot_load.c, types.h, tests, buglog | 87/87 green, uncommitted | ~0 | +| 00:04 | Edited src/lib/client_failover.c | 2→5 lines | ~88 | +| 00:04 | Session end: 9 writes across 7 files (test_limit_host_dedup.c, limit.c, test_config_push_deferred.c, handler.c, main.c) | 10 reads | ~5495 tok | + +## Session: 2026-09-24 00:04 + +| Time | Action | File(s) | Outcome | ~Tokens | +|------|--------|---------|---------|--------| +| 00:28 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | modified gate() | ~275 | +| 00:28 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | expanded (+10 lines) | ~194 | +| 00:28 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | modified restart_scheduler_for_new_primary() | ~182 | +| 00:28 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | 4→5 lines | ~65 | +| 00:28 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | modified sched_extra_args() | ~1124 | +| 00:28 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | 2→6 lines | ~78 | +| 00:28 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | 4→8 lines | ~69 | +| 00:28 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | expanded (+6 lines) | ~152 | +| 00:28 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | expanded (+6 lines) | ~135 | +| 00:29 | Edited .claude/skills/xce-test-ha/SKILL.md | expanded (+22 lines) | ~289 | +| 00:37 | Edited .claude/skills/xce-test-ha/run-ha-soak.sh | 2→5 lines | ~80 | +| 00:40 | Edited src/controller/testsuite/test_worker_join.c | added 2 condition(s) | ~1429 | +| 00:41 | Edited src/controller/worker.c | added 4 condition(s) | ~297 | +| 00:53 | Edited src/scheduler/testsuite/test_record_lifecycle.c | added 1 condition(s) | ~1470 | +| 01:19 | Edited .claude/skills/xce-test-ha/SKILL.md | expanded (+6 lines) | ~158 |