chore(wolf): update session tracking state
anatomy.md/cerebrum.md/memory.md/token-ledger.json/hooks state covering the design-review fix pass (tlog activation, core-isolation audit, schedAdmin admission gate, resource-value-verification doc/comment gaps) and task 12.5's fresh completion verification.
This commit is contained in:
parent
aa42752d35
commit
ffa8bc110f
5 changed files with 5156 additions and 29 deletions
|
|
@ -1,7 +1,7 @@
|
|||
# anatomy.md
|
||||
|
||||
> Auto-maintained by OpenWolf. Last scanned: 2026-08-19T09:46:55.696Z
|
||||
> Files: 586 tracked | Anatomy hits: 0 | Misses: 0
|
||||
> Auto-maintained by OpenWolf. Last scanned: 2026-08-19T10:58:57.650Z
|
||||
> Files: 592 tracked | Anatomy hits: 0 | Misses: 0
|
||||
|
||||
## ../../../../../../tmp/
|
||||
|
||||
|
|
@ -585,7 +585,7 @@
|
|||
|
||||
## .kiro/specs/resource-value-verification/
|
||||
|
||||
- `design.md` — Design Document (~11019 tok)
|
||||
- `design.md` — Design Document (~11716 tok)
|
||||
- `requirements.md` — Requirements Document (~2797 tok)
|
||||
- `research.md` — Research & Design Decisions (~5967 tok)
|
||||
- `spec.json` (~133 tok)
|
||||
|
|
@ -616,11 +616,11 @@
|
|||
## .kiro/specs/scheduler-core/
|
||||
|
||||
- `brief.md` — Brief: scheduler-core (~2035 tok)
|
||||
- `design.md` — Technical Design: scheduler-core (~17063 tok)
|
||||
- `requirements.md` — Requirements Document (~26234 tok)
|
||||
- `design.md` — Technical Design: scheduler-core (~17349 tok)
|
||||
- `requirements.md` — Requirements Document (~26526 tok)
|
||||
- `research.md` — Research & Design Decisions: scheduler-core (~4398 tok)
|
||||
- `spec.json` (~200 tok)
|
||||
- `tasks.md` — Implementation Plan (~32131 tok)
|
||||
- `tasks.md` — Implementation Plan (~32783 tok)
|
||||
|
||||
## .kiro/specs/scheduler-core/.solomd/
|
||||
|
||||
|
|
@ -852,7 +852,7 @@
|
|||
|
||||
- `apiserver.c` — include <errno.h> (~6392 tok)
|
||||
- `CLAUDE.md` — XCE Controller (~4776 tok)
|
||||
- `cluster.c` — include "config.h"; task 10.2 adds XCE_TESTING-only schedadmin_set_timeout_ms_for_test seam (shrinks OPCode_schedAdmin's 5s forward-wait deadline for tests) (~26217 tok)
|
||||
- `cluster.c` — include "config.h" (~24998 tok)
|
||||
- `commit.c` — Declares is (~13741 tok)
|
||||
- `config_repl.c` — define _GNU_SOURCE (~36913 tok)
|
||||
- `config_repl.h` — pragma once (~6952 tok)
|
||||
|
|
@ -925,13 +925,14 @@
|
|||
- `test_ctrlstatus.c` — include <stdatomic.h> (~6287 tok)
|
||||
- `test_ha_rdhup_integration.c` — Declares char (~8362 tok)
|
||||
- `test_resval_typeresolve.c` — Declares override (~1118 tok)
|
||||
- `test_sched_admin_dispatch.c` (~7510 tok)
|
||||
- `test_sched_worker_change.c` — Declares bug (~3893 tok)
|
||||
- `test_term_safety_adapter.c` — define _GNU_SOURCE (~29920 tok)
|
||||
- `test.c` — include <argp.h> (~39556 tok)
|
||||
|
||||
## src/frontend/apiserver/backend/
|
||||
|
||||
- `handler.go` — Struct: DefaultHandler; NotificationHandler impl (OnJobCreated/Updated/Deleted, OnHostUpdated, OnClusterUpdated, OnResourceUpdated, OnResourceLocationUpdated). Task 4.2 (resource-value-verification): OnResourceLocationUpdated now emits `logger.Debug("resource-location stored", zap.String("resource",...), zap.String("host",...), <typed value field>)` right after store.ResourceLocations.Put (~1780 tok)
|
||||
- `handler.go` — Struct: DefaultHandler (~1883 tok)
|
||||
|
||||
## src/include/
|
||||
|
||||
|
|
@ -939,7 +940,8 @@
|
|||
|
||||
## src/scheduler/
|
||||
|
||||
- `meson.build` (~1183 tok)
|
||||
- `CLAUDE.md` — XCE Scheduler (~1188 tok)
|
||||
- `meson.build` (~1426 tok)
|
||||
|
||||
## src/scheduler/include/
|
||||
|
||||
|
|
@ -949,7 +951,7 @@
|
|||
## src/scheduler/src/
|
||||
|
||||
- `core_clock.c` — include <stdarg.h> (~3441 tok)
|
||||
- `core_events.c` — include <errno.h> (~10786 tok)
|
||||
- `core_events.c` — include <errno.h> (~10963 tok)
|
||||
- `core.c` — include <errno.h> (~4310 tok)
|
||||
- `driver.c` — include <errno.h> (~9230 tok)
|
||||
- `main.c` — CLI entry (argp table); bug-1585 added --record-snapshot-interval, wired to sched.record_snapshot_interval (~4650 tok)
|
||||
|
|
@ -961,9 +963,10 @@
|
|||
- `repl.c` — include <ctype.h>; task 12.5 added xce_repl_on_replay_complete(), replay_complete guard on step/continue, quote-stripping in repl_split_tokens() (bug-1594/1596) (~6611 tok)
|
||||
- `resource.c` — include <errno.h> (~4526 tok)
|
||||
- `scheduler.c` — include <errno.h> (~15773 tok)
|
||||
- `sim_loop.c` — include <errno.h>; task 12.5 added xce_repl_check_replay_complete() call after replay loop ends (bug-1594), 770/800 lines (~9386 tok)
|
||||
- `sim_loop.c` — include <errno.h> (~9686 tok)
|
||||
- `sim_picker.c` — include <errno.h> (~1374 tok)
|
||||
- `sim_verify.c` — include <stdio.h> (~4694 tok)
|
||||
- `sim.c` — include <errno.h> (~5100 tok)
|
||||
- `snapshot_load.c` — include <errno.h> (~4005 tok)
|
||||
- `snapshot_wire.c` — include <errno.h> (~2611 tok)
|
||||
|
||||
|
|
@ -973,17 +976,20 @@
|
|||
|
||||
## src/scheduler/testsuite/
|
||||
|
||||
- `meson.build` — Declares vtask_entry; registers sched_resval_ownership_test (bug-1585) (~15150 tok)
|
||||
- `meson.build` — Declares vtask_entry (~15137 tok)
|
||||
- `scratch_gen_resource_replay.c` (~2735 tok)
|
||||
- `test_clock.c` — Declares uint64_t (~5450 tok)
|
||||
- `test_core_events.c` — Declares and; real dup_val() reimpl added for link (bug-1585) (~7880 tok)
|
||||
- `test_core_isolation.c` — Declares char (~4069 tok)
|
||||
- `test_driver_translate.c` — Declares field; full_val/delta_val fixtures now set ->which=XCE_VAL_NUMERIC (bug-1585 dup_val fix exposed the gap) (~9580 tok)
|
||||
- `test_repl_breakpoints_inspection.c` — Declares and (~16735 tok)
|
||||
- `test_repl_equivalence.c` — Declares char (~7554 tok)
|
||||
- `test_repl_execution_control.c` — Declares directly (~8798 tok)
|
||||
- `test_resval_ownership.c` — bug-1585 regression: RESVALS_FULL/RESVALS_DELTA heap-use-after-free, with a REAL (non-stubbed) kill_sched_resval to actually catch it (~5471 tok)
|
||||
- `test_sim_alt_events.c` — Declares char (~4863 tok)
|
||||
- `test_sim_event_loop.c` — Declares char (~8156 tok)
|
||||
- `test_sim_event_loop.c` — Declares char (~9248 tok)
|
||||
- `test_sim_isolation.c` — Declares names (~8174 tok)
|
||||
- `test_sim_startup.c` — Declares char (~5535 tok)
|
||||
- `test_sim_verify.c` — Declares mismatch (~9312 tok)
|
||||
|
||||
## src/tools/
|
||||
|
|
|
|||
|
|
@ -27,6 +27,7 @@
|
|||
- **Fixing a busy-spin/livelock can unmask a latent notification gap (bug-1564)**: throttling an unbounded retry loop (bug-1563's fix) removed an *accidental* fast-repoll that had been quietly covering for `persist_flush()` never firing `on_durable_advance` (only the async `fsync_worker` did). A deterministic (3/3) new test failure appeared immediately after — always suspect "the old code accidentally covered for a real gap via sheer retry speed" when a correctness fix (removing a spin/race) makes a previously-green test start failing 100% of the time; that's a signal to look for a missing notification/wakeup, not to revert the fix.
|
||||
- **nix devShell Python packages are `PYTHONPATH` entries, not merged into `python3`'s own site-packages** (found building a standalone-installable xceadmin wheel): `python3 -m venv --system-site-packages` only inherits the base interpreter's OWN site-packages dir, which is empty in this project's nix shell (`python3Packages.click`/`pycapnp`/etc. are each a separate `/nix/store/...` path added via `$PYTHONPATH`, confirmed via `nix develop … --command sh -c 'echo $PYTHONPATH'`). A `--system-site-packages` venv therefore silently CANNOT see them (`ModuleNotFoundError` at runtime, discovered live). Fix: skip venv/pip entirely — extract the wheel directly (`python3 -m zipfile -e`) and bake the *resolved* `$PYTHONPATH` string into a wrapper script's `export PYTHONPATH=...` at generation time (expand now, not `${PYTHONPATH:-}` at runtime) — mirrors `src/xceadmin/meson-install.sh`'s existing nix-packaging wrapper, which already solved this correctly; I should have grepped for that pattern before reaching for venv the first time.
|
||||
- **`src/tools/xce-build.sh`'s `setup_build()` does `rm -rf "$BUILD_DIR"` for every single-component invocation** (bug-1565, FIXED): `/xce-build xceadmin` after `/xce-build controller` in the same dir used to silently delete `xce_controller`/`xce_test`/etc. Fixed by an opt-in `--incremental` flag (`meson setup --reconfigure` in place instead of wiping) — deliberately NOT the default, since this project's capnp-generation `custom_target` has known missing-dependency-edge races that a full wipe papers over as a loud, retry-fixable failure (skipping the wipe could turn the same gap into silent stale-generated-code instead). `/xce-build all` (build everything together) remains the simplest option when you don't specifically need incremental speed.
|
||||
- **A never-joined background thread reading a stack-local test fixture is a latent ASAN stack-use-after-return, exposed only by test ORDERING, not by the thread's own logic** (2026-08-19, scheduler-core design-review follow-up). `sched_sim_driver_run()` (sim_loop.c) spawns `sim_picker_thread_fn` on a real pthread that is, by this codebase's own documented design, "never joined or cancelled... no sim-driver teardown/fini function exists yet" — it loops forever on `cbuf_get(&sim->core.wbuf, ...)`. Every existing white-box test in `test_sim_event_loop.c` already called this with a stack-local `xce_sched_sim_driver_t sim;`, which was ALWAYS a latent stack-use-after-return once the test function returned — it just hadn't been caught, because no test's stack frame had yet been reused by a later frame in a way ASAN's shadow-memory detector caught. Simply INSERTING one new test between two existing ones (no logic change to the existing tests) shifted the stack-reuse pattern enough to make it fire deterministically (reproduced twice). Fix: change the fixture from stack-local to `static` storage — safe here specifically because (a) `sched_sim_driver_start()` unconditionally `memset()`s the whole struct as its first action so no stale-data leak across runs, and (b) each affected test function is called exactly once per process (verified via grep, no re-entrancy) so a second call's memset can never stomp a still-live zombie thread's cbuf mutex/condvar state from a first call — `static` would be UNSAFE to apply to a helper called more than once while its first zombie thread might still be alive. General lesson: when a background thread spawned by code-under-test is documented as "never joined," any test giving that code a stack-local fixture is a ticking ASAN time bomb regardless of whether it has fired yet — check for exactly this "never joined/cancelled" phrasing in the target function's own comments before adding a new test that changes stack layout/ordering in the same file.
|
||||
- **A pipeline into `head -N` can SIGPIPE-kill the upstream script early** under `set -o pipefail` (bit twice this session: once with `timeout N cmd | head -N` leaving a stale background process running for hours after the harness reported the task "complete", once with `xce-build.sh ... | head -3` silently truncating a build before it finished writing its wheel). If you need to sanity-check just the first few lines of a long-running command's output, redirect to a file and `head` the file afterward instead of piping directly into `head`.
|
||||
- **`sim_loop.c` (and its `*_isolation`-audited siblings) genuinely has zero line-budget headroom** — its own file header says so explicitly. When a fix needs new logic there, split a new sibling file (bug-1567's `sim_picker.c`, following the exact precedent chain `sim_loop.c`→`sim_record.c`→`repl.c`→`repl_break.c`/`repl_info.c`→`repl_params.c`) rather than growing it even slightly. This requires wiring the new file into: `scheduler_srcs` AND `sched_sim_isolation_lib` (both `src/scheduler/meson.build`), a new `XCE_SIM_*_PATH` macro (`testsuite/meson.build`) consumed by `test_sim_isolation.c`'s 3 per-file text-scan functions, AND every white-box test TU that `#include`s `sim_loop.c` directly (9 files this session — grep `#include "../src/sim_loop.c"` in `testsuite/` to find them all) needs the new file `#include`d too or the link fails on the new extern. Mechanical but real work — budget for it.
|
||||
- **`controller_unit_test`'s intermittent SIGSEGV (bug-1553, ~1-in-3 full runs) is a known pre-existing flake**, not a new regression — reproduce standalone (`./src/build.dbg/controller/controller_unit_test`, no `meson test` wrapper) a few times before suspecting your own change; `meson test`'s `MALLOC_PERTURB_` randomization per invocation is what makes it flaky, not deterministic per-build.
|
||||
|
|
@ -37,6 +38,7 @@
|
|||
- **apiserver's `main.go` only calls `RegisterToBackend` when `BackendYamlConfig.Address != ""` — checked BEFORE `Peers` is ever consulted** (`cmd/main.go:58`). Setting `app.yaml`'s `backend.peers` to a fully-populated HA candidate list while leaving `backend.address`/`backend.port` blank (reasoning "Peers takes priority inside RegisterToBackend anyway, per `backend/config.go`'s own doc comment") is NOT sufficient — the apiserver silently boots in permanent "stub mode" (`logger.Info("Backend not configured")`) and never even attempts a connection, `peers` or not. Caught live while building `xce-deploy-ha`'s new apiserver step (resource-value-verification task 3.1): always populate `address`/`port` too (e.g. the first `--peers` candidate's host:port) even when also setting `peers` — belt-and-suspenders, not redundant, because the two fields gate two different things (whether to connect at all, vs. which candidate(s) to try).
|
||||
- **resource-value-verification task 4.1's scheduler trace-log exact format** (`xce_core_push_events()`'s `XCE_CORE_EVENT_RESVALS_DELTA` case, `src/scheduler/src/core_events.c`, right after `zarray_put(&(w->resvals), res->idx, rv->val)`): `TRACE("resval applied host=%s resource=%s value=%f", ...)` for numeric (`Value_u_fvalue`) resources, or `value=%s` for `Value_u_svalue` ones — live-confirmed via `xce-deploy-ha` + `xce_test` worker-context `join`/`resupdate -n r15m -v 12345.5` producing exactly `resval applied host=task41-worker resource=r15m value=12345.500000` in `scheduler.log` at `-vvv`. Task 6.3 (scheduler-side log-tail check) MUST grep for `resval applied host=<H> resource=<R> value=<V>` — labeled `key=value` fields, no other fixed text to depend on. `%f` on a normal resource value produces trailing zeros (e.g. `12345.500000`), not the input's original decimal precision — task 6.3's comparison should parse-and-compare as float, not do a literal string match against the injected value string.
|
||||
- **bug-1572 (NOT FIXED, pre-existing, discovered live task 4.1 round-2): `xce_scheduler` double-frees on a worker's SECOND periodic resource report for the same resource(s), silently dying with no ASAN output** (backgrounded via `setsid nohup ... >/dev/null 2>&1`, so the crash report is lost -- only visible as scheduler.log stopping mid-stream + `handle_scheduler_leave` in the controller log ~1s later). Root cause: `core_events.c`'s `XCE_CORE_EVENT_RESVALS_DELTA` case does `zarray_put(&(w->resvals), res->idx, rv->val)` with NO `MOVE_PTR` (unlike its sibling `core_apply_config_resvals()`, which does `MOVE_PTR(val, nv->val)` first) -- the same `xce_val_t*` stays aliased in the decoded request. `driver.c`'s `release_translated_payload()` then unconditionally frees that request's resvals list even when `applied=1` (its `XCE_SCHED_OP_UPDATE_RESVALS` case has no `if (!applied)` guard, unlike the sibling `XCE_SCHED_OP_UPDATE_CONFIG` case which explicitly skips the free when applied). Net effect: `w->resvals[idx]` is left dangling immediately after every successful RESVALS_DELTA batch; the SECOND update to the same index triggers zarray's killer to free the already-freed pointer. Confirms identically on round-1's code (unrelated to the enum-branch bug fixed in that round) -- a live smoke test that only runs one resupdate quickly after deploy will NOT hit this; anything that waits past one real worker's report interval (xcecd_virtual reports ~every 15s) will. When live-testing anything touching RESVALS_DELTA, either run fast (single update, no wait) or expect this crash and don't misattribute it to whatever fix is actually under test.
|
||||
- **`src/frontend/Makefile`'s `build-server` target is missing CGO_CFLAGS/CGO_LDFLAGS (bug-1597, NOT FIXED)**: unlike `test-backend` (bug-1558) and `apiserver/meson.build`, `build-server`'s recipe is a bare `go build` with no `-I`/`-L` for `client.h`/`controller.capnp.h`/`libxce.a`, so `nix develop ../ --command make build-server` fails with `fatal error: client.h: No such file or directory` regardless of source changes. Confirmed via git-stash A/B that this reproduces identically with or without an unrelated diff, so don't misattribute it to whatever change is under test. Workaround to actually verify compilation: reconstruct CGO_CFLAGS/CGO_LDFLAGS the same way `test-backend` does (pkg-config for librtti/c-capnproto/openssl/tinylog + `-I$BUILD_DIR/lib`/`-L$BUILD_DIR/lib -lxce`, `BUILD_DIR` defaulting to `src/xce.build`) and run `CGO_ENABLED=1 CGO_CFLAGS=... CGO_LDFLAGS=... go build -o /tmp/apiserver ./cmd/main.go` directly inside `nix develop ../` — see bug-1597 for the exact one-liner.
|
||||
- **`test.c`'s `set_resval()` leaves `xce_resval_t.name` (and `.val->which`) uninitialized for the `Value_u_svalue` fallback path** (`-n type`/`-n model`/explicit `-t string` on an unknown name) — its switch statement has no `case Value_u_svalue:` arm, and `vals[]` is a non-zeroed stack array reused across REPL invocations, so `.name` can be garbage, not NULL. Pre-existing (predates resource-value-verification's task 2.1/2.2), explicitly flagged as out-of-scope-to-fix in both tasks' code comments (Requirement 1.4's "preserve byte-for-byte" constraint). When building task 2.2's per-name verification loop on top of this, the safe pattern was to track the submitted name in a separate parallel array (`subnames[]`) captured directly from the getopt loop's own `resname` variable (always valid — that pointer is deliberately leaked/never freed, so it stays alive for the whole REPL call) rather than trusting `vals[i].name`.
|
||||
|
||||
- **Trap-based cleanup that must survive an async signal (SIGTERM/SIGINT) should combine multiple `pkill -9 -f` calls into ONE alternation pattern, not several sequential invocations, plus a retry-until-verified (`pgrep`) loop** (bug-1575, resource-value-verification task 5.1's `run-resource-verify.sh`). Four separate sequential `pkill -9 -f -- "pattern"` calls inside `cleanup()` worked reliably every time when reached via normal control flow (a direct `fail()`/`exit` call), but non-deterministically left 1-2 of the four target processes running when `cleanup()` instead ran from the EXIT trap fired by an externally-delivered SIGTERM translated via a separate `trap 'exit N' TERM` handler -- the exact failure point (which of the 4 pkill calls was "last to complete") varied run to run, consistent with a timing race in the trap-handler code path rather than a pattern-matching bug (each pattern was independently verified correct). Fix: one combined `pkill -9 -f -- "pat1|pat2|pat3|pat4"` call (fewer fork/exec round trips through the trap handler) wrapped in a loop that re-checks via `pgrep -f` and retries (up to ~2s total) until genuinely clean. Verified reliable across 5+ repeated live SIGTERM-mid-deployment interrupts after the fix (zero leftover processes every time), vs. non-deterministic leftovers before it. General lesson: don't trust a cleanup routine's reliability from its NORMAL-completion path alone -- explicitly live-test the signal-interrupted path too, since bash's async-signal-trap execution context can behave differently from the same code reached via ordinary top-to-bottom flow or a plain function call/exit.
|
||||
|
|
@ -72,6 +74,8 @@
|
|||
- **A dead `#if 0 /*FIXME*/ ... #endif` block is a signal the disabled code is unsafe as written, not merely "not yet wired in" — don't just uncomment it.** bug-1578: `sched_worker_change()`'s disabled block called `free_name_val_list(&w->host.resvals_to_sched, &w->host.nresvals_to_sched)` right before `xlist_fini(&resvalist)` — but `resvalist.data` and `w->host.resvals_to_sched` are the SAME pointer (the earlier line aliased one to the other's backing array), and `free_name_val_list()` already frees that array itself; uncommenting as-is would double-free. The safe fix null-and-reset the aliased fields directly (`w->host.resvals_to_sched = NULL; w->host.nresvals_to_sched = 0;`) and let `xlist_fini()` be the sole owner of the free — the pattern already proven correct in the sibling `apiserver_worker_change()` (apiserver.c). General lesson: when re-enabling dead code, first work out WHY it was disabled (git blame + read what it actually does to shared/aliased memory), don't assume `#if 0` means "harmless, just needs a flip." Other call sites still doing `free_name_val_list(&(w->resvals_to_sched), ...)` on this same field (dataex.c:435, opget.c:128, reply.c:66, apiserver.c:51) were NOT audited for the identical aliasing hazard as part of this fix — flagged, not yet checked.
|
||||
- **`xce_test`'s `!<shell-command>` escape (`preprocess_and_run()`, single call site in `main()`'s top-level REPL loop, `test.c` ~line 950-998, backed by a plain blocking `system()`) works IDENTICALLY in every REPL context, including `worker` — no `leave_ctx()`/disconnect happens, unlike moving between contexts via `..`/`cmd_up`.** This is the general pattern for "run an out-of-band check while a stateful REPL session must stay connected": bake a `!`-prefixed line into the script fed to the session (before its `quit`) that shells out to ANOTHER invocation of your own driver script in a hidden internal mode (e.g. `! bash "$0" --internal-<mode> <args>`), intercepted via a dispatch check placed BEFORE the normal arg-parsing entry point at the very bottom of the file. Used to fix bug-1579 (resource-value-verification): the controller-side resval check now runs nested inside the injecting worker session's own `!` escape instead of as a separate xce_test process launched after `quit` already disconnected the worker (which reset host-attrs resources to baseline first, an always-false MISMATCH). Gotchas: (1) the nested process must generally return exit 0 even on partial/expected failure conditions — a nonzero exit becomes the outer session's `g_last_status`/`$?`, and in non-interactive/batch mode `preprocess_and_run()` ABORTS the REPL loop before any later command (including `quit`) runs; surface real failures via a sentinel/marker file the OUTER script checks AFTER the whole session completes, not via the nested process's own exit code. (2) `system()`'s child inherits the invoking process's environment but NOT its parent bash's local (non-exported) shell variables — bake every value the nested invocation needs (paths, host lists, names) as literal, `printf '%q'`-quoted arguments directly into the `!` line's command string, don't rely on env-var passthrough through the `nix develop`/ASAN-wrapped intermediate hops. (3) since accept/reject-style bookkeeping computed by the REPL session itself is only visible as TEXT PRINTED to that session's own stdout — not accessible to a freshly-spawned nested process — have the nested step collect RAW, unfiltered data for every candidate (not just ones later known to be "accepted") and defer accept/reject filtering to the outer script, after it has parsed the whole session's captured output post-`quit`. (4) live-verified no orphan/zombie processes result even when a SIGTERM interrupts the outer driver script WHILE the nested `!`-escape's own sub-session is actively running — bash defers a pending trap until the current foreground command (the whole piped xce_test invocation, nested check included) returns, so the child process tree either completes and exits cleanly on its own or the trap fires right after; confirmed via a full `/proc/[0-9]*/cmdline` scan, not just `pgrep`.
|
||||
- **Piping scripted commands into `xce_test` (the controller REPL/testsuite binary) with default `ASAN_OPTIONS` loses ALL stdout output when redirected to a file/pipe.** `xce_test` is an ASAN build; `detect_leaks=1` is the default, and this binary has small pre-existing intentional leaks (e.g. `set_resval`'s `calloc`, `cmd_resupdate`'s `strdup` for `resname` — never freed by design, REPL-lifetime scratch data). LeakSanitizer's exit-time report appears to bypass/precede the normal stdio flush when stdout is non-interactive (fully block-buffered), so a redirected-output smoke test showed ONLY the ASAN leak report and none of the actual REPL command output (join/resupdate/resval results), even though the commands genuinely executed (confirmed via the leak stack trace itself). Fix: always run `xce_test` with `ASAN_OPTIONS=detect_leaks=0` (matching the meson test harness's own invocation, which already sets this) whenever piping/redirecting its stdout for a live smoke test — e.g. `nix develop ./src --command env ASAN_OPTIONS=detect_leaks=0 xce_test < cmds.txt`.
|
||||
- **A fork/subagent dispatched in a prior session that claimed "build/tests in progress, will report back" is NOT reachable in a new session — `ListAgents` will not show it, and its self-report (if logged to memory before the session ended) describes an *intent*, not a completed result.** 2026-08-19: a fork agent from the previous session was recorded as "actively writing tests... then building via xce-build skill and reporting final metrics," but the session ended before it reported. On resuming ("continue"), the scheduler test binaries in `src/xce.build` were still dated Aug 18 14:27 while the source edits were Aug 19 09:18-09:27 — proof the claimed build never actually happened in this environment. Always verify via binary/source `stat` mtime comparison (or just re-run the build) before trusting a cross-session "in progress"/"dispatched" claim, same discipline as the existing "stale/uncommitted tree GO claim" entry above but for cross-session fork continuity specifically, not just tree staleness.
|
||||
- **A REPL/replay test fixture that reuses the SAME jobid for both a "real" event (job push) and a "cheap padding" event (job state change, added purely to exercise a second event type) accidentally mutates the real job's state, not just decorates the log.** bug-1595 test regression (task 12.5, 2026-08-19): `test_step_echoes_events_before_session_decisions` pushed job 6161 then sent a JOBSTATECHANGE event for the SAME jobid 6161 (PEND->RUN) purely to get a second event type echoed — this genuinely transitioned the real job to RUNNING before the session ran, so the picker's `next_job()` never offered it and the session's own PEND decision (which the test also asserted on) was never produced. The sibling test `test_event_breakpoint_pauses_repl_at_matching_event` already documents the correct pattern in its own comment (`append_jobstatechange_event() establishes: core_events.c WARN+skips an unknown jobid rather than fail, so ... below needs no real job to exist`) — it deliberately used unrelated jobids (111/222) precisely to keep the padding event inert. When adding a "just need any event of type X to be logged" filler event to a fixture that also has a real job in flight, always target an unrelated/nonexistent jobid unless the state transition itself is the thing under test.
|
||||
- **`[ -n "${arr[key]+x}" ]` (associative-array key-EXISTENCE) is NOT a safe proxy for "the value is meaningful/reachable" when the producer unconditionally sets that key even on failure.** bug-1582 (resource-value-verification task 7.1, round-1 REJECTED, round-2 fixed): `main()`'s record-building loop derived `sched_ok`/`api_ok` from whether `SCHEDULER_OBSERVED[$vname]`/`APISERVER_OBSERVED[$vname]` existed — but `query_scheduler_resvals()`/`query_apiserver_resvals()` (already-approved tasks 6.3/6.4) always set that key, setting it to `""` on a genuine UNREACHABLE/timeout rather than leaving it unset. Key-existence was therefore always true, silently masking every real UNREACHABLE as a MISMATCH (comparing the injected value against an empty string). Fix: derive the "is this layer reachable" signal from the layer's own pre-classified status string (`SCHEDULER_CHECK_RESULTS[$vname]`/`APISERVER_CHECK_RESULTS[$vname]`, which DOES distinguish `UNREACHABLE:*` from `MATCH`/`MISMATCH:*`) instead of from a sibling data array's key presence. General lesson: before using `${arr[k]+x}`/`-v arr[k]` as a "was this checked / did this succeed" signal, verify the producer actually leaves the key UNSET on failure — if it sets it to an empty/placeholder value instead (a common pattern for "always populate every requested key so downstream code doesn't need existence-checks"), key-existence collapses to "was this key in the requested set" a wholly different, weaker signal than "did this succeed". Also: a synthetic test that only calls the pure aggregation function directly with hand-built records (task 8.2's original approach) cannot catch a bug living in the data-marshaling code that BUILDS those records — this class of bug needs the marshaling code itself extracted into a directly-callable function (`build_verdict_records_from_tsv()`) and exercised by its own test.
|
||||
|
||||
## Decision Log
|
||||
|
|
|
|||
|
|
@ -8,12 +8,12 @@
|
|||
"first_read": "2026-08-19T09:39:10.469Z"
|
||||
},
|
||||
"/tmp/claude-1000/-home-rshen-projects-xce-worktrees-scheduler-framework/0334ebbe-8d9c-4931-871d-87c678f83a79/scratchpad/build_output.log": {
|
||||
"count": 2,
|
||||
"count": 4,
|
||||
"tokens": 0,
|
||||
"first_read": "2026-08-19T09:41:20.109Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/src/sim_loop.c": {
|
||||
"count": 3,
|
||||
"count": 4,
|
||||
"tokens": 9386,
|
||||
"first_read": "2026-08-19T09:43:29.822Z"
|
||||
},
|
||||
|
|
@ -28,9 +28,149 @@
|
|||
"first_read": "2026-08-19T09:44:33.465Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/test_repl_breakpoints_inspection.c": {
|
||||
"count": 3,
|
||||
"tokens": 16580,
|
||||
"count": 4,
|
||||
"tokens": 16735,
|
||||
"first_read": "2026-08-19T09:44:42.168Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/.claude/skills/kiro-validate-design/rules/design-review.md": {
|
||||
"count": 1,
|
||||
"tokens": 1090,
|
||||
"first_read": "2026-08-19T09:54:41.844Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/.kiro/steering/product.md": {
|
||||
"count": 1,
|
||||
"tokens": 1509,
|
||||
"first_read": "2026-08-19T09:54:42.648Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/.kiro/steering/tech.md": {
|
||||
"count": 1,
|
||||
"tokens": 1500,
|
||||
"first_read": "2026-08-19T09:54:43.002Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/.kiro/steering/structure.md": {
|
||||
"count": 1,
|
||||
"tokens": 1348,
|
||||
"first_read": "2026-08-19T09:54:43.260Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/.kiro/specs/scheduler-core/design.md": {
|
||||
"count": 4,
|
||||
"tokens": 17283,
|
||||
"first_read": "2026-08-19T09:54:50.577Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/.kiro/specs/scheduler-core/requirements.md": {
|
||||
"count": 3,
|
||||
"tokens": 26234,
|
||||
"first_read": "2026-08-19T09:54:50.964Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/.kiro/specs/resource-value-verification/design.md": {
|
||||
"count": 2,
|
||||
"tokens": 11019,
|
||||
"first_read": "2026-08-19T09:56:07.049Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/.kiro/specs/resource-value-verification/requirements.md": {
|
||||
"count": 1,
|
||||
"tokens": 2797,
|
||||
"first_read": "2026-08-19T09:56:07.596Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/.kiro/specs/resource-value-verification/tasks.md": {
|
||||
"count": 1,
|
||||
"tokens": 8164,
|
||||
"first_read": "2026-08-19T09:56:07.761Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/.kiro/steering/ha-async-io.md": {
|
||||
"count": 1,
|
||||
"tokens": 4721,
|
||||
"first_read": "2026-08-19T10:06:35.141Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/src/controller/cluster.c": {
|
||||
"count": 3,
|
||||
"tokens": 26217,
|
||||
"first_read": "2026-08-19T10:07:05.618Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/src/core_clock.c": {
|
||||
"count": 1,
|
||||
"tokens": 3441,
|
||||
"first_read": "2026-08-19T10:10:55.791Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/src/sim.c": {
|
||||
"count": 1,
|
||||
"tokens": 0,
|
||||
"first_read": "2026-08-19T10:11:48.062Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/test_clock.c": {
|
||||
"count": 1,
|
||||
"tokens": 5450,
|
||||
"first_read": "2026-08-19T10:12:17.220Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/test_sim_startup.c": {
|
||||
"count": 6,
|
||||
"tokens": 5535,
|
||||
"first_read": "2026-08-19T10:12:31.555Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/test_sim_event_loop.c": {
|
||||
"count": 5,
|
||||
"tokens": 9082,
|
||||
"first_read": "2026-08-19T10:12:45.396Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/meson.build": {
|
||||
"count": 1,
|
||||
"tokens": 1183,
|
||||
"first_read": "2026-08-19T10:14:13.066Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/test_core_isolation.c": {
|
||||
"count": 1,
|
||||
"tokens": 0,
|
||||
"first_read": "2026-08-19T10:14:27.436Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/src/controller/testsuite/test_sched_admin_dispatch.c": {
|
||||
"count": 4,
|
||||
"tokens": 7491,
|
||||
"first_read": "2026-08-19T10:19:18.913Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/src/lib/vtaskmgr.c": {
|
||||
"count": 2,
|
||||
"tokens": 0,
|
||||
"first_read": "2026-08-19T10:19:43.824Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/src/core_events.c": {
|
||||
"count": 1,
|
||||
"tokens": 10786,
|
||||
"first_read": "2026-08-19T10:23:26.317Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/src/frontend/apiserver/backend/handler.go": {
|
||||
"count": 1,
|
||||
"tokens": 1780,
|
||||
"first_read": "2026-08-19T10:23:37.915Z"
|
||||
},
|
||||
"/home/claude/.claude/projects/-home-rshen-projects-xce-worktrees-scheduler-framework/0334ebbe-8d9c-4931-871d-87c678f83a79/tool-results/bzk7zzg3x.txt": {
|
||||
"count": 1,
|
||||
"tokens": 0,
|
||||
"first_read": "2026-08-19T10:25:28.269Z"
|
||||
},
|
||||
"/tmp/claude-1000/-home-rshen-projects-xce-worktrees-scheduler-framework/0334ebbe-8d9c-4931-871d-87c678f83a79/tasks/biipc8j9m.output": {
|
||||
"count": 1,
|
||||
"tokens": 0,
|
||||
"first_read": "2026-08-19T10:25:46.915Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/.kiro/specs/scheduler-core/tasks.md": {
|
||||
"count": 2,
|
||||
"tokens": 32131,
|
||||
"first_read": "2026-08-19T10:41:02.602Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/test_repl_snapshot_params.c": {
|
||||
"count": 1,
|
||||
"tokens": 0,
|
||||
"first_read": "2026-08-19T10:52:27.345Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/meson.build": {
|
||||
"count": 2,
|
||||
"tokens": 15137,
|
||||
"first_read": "2026-08-19T10:52:27.503Z"
|
||||
},
|
||||
"/home/rshen/projects/xce/worktrees/scheduler-framework/.kiro/specs/scheduler-core/spec.json": {
|
||||
"count": 1,
|
||||
"tokens": 200,
|
||||
"first_read": "2026-08-19T10:58:53.028Z"
|
||||
}
|
||||
},
|
||||
"files_written": [
|
||||
|
|
@ -39,14 +179,272 @@
|
|||
"action": "edit",
|
||||
"tokens": 225,
|
||||
"at": "2026-08-19T09:46:55.704Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/test_sim_startup.c",
|
||||
"action": "edit",
|
||||
"tokens": 576,
|
||||
"at": "2026-08-19T10:13:21.691Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/test_sim_startup.c",
|
||||
"action": "edit",
|
||||
"tokens": 63,
|
||||
"at": "2026-08-19T10:13:33.040Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/test_sim_event_loop.c",
|
||||
"action": "edit",
|
||||
"tokens": 684,
|
||||
"at": "2026-08-19T10:13:46.026Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/test_sim_event_loop.c",
|
||||
"action": "edit",
|
||||
"tokens": 76,
|
||||
"at": "2026-08-19T10:14:08.700Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/meson.build",
|
||||
"action": "edit",
|
||||
"tokens": 481,
|
||||
"at": "2026-08-19T10:15:19.532Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/meson.build",
|
||||
"action": "edit",
|
||||
"tokens": 190,
|
||||
"at": "2026-08-19T10:15:32.377Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/test_core_isolation.c",
|
||||
"action": "edit",
|
||||
"tokens": 1031,
|
||||
"at": "2026-08-19T10:15:59.949Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/test_core_isolation.c",
|
||||
"action": "edit",
|
||||
"tokens": 184,
|
||||
"at": "2026-08-19T10:16:06.812Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/test_core_isolation.c",
|
||||
"action": "edit",
|
||||
"tokens": 1370,
|
||||
"at": "2026-08-19T10:16:34.721Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/test_core_isolation.c",
|
||||
"action": "edit",
|
||||
"tokens": 146,
|
||||
"at": "2026-08-19T10:16:43.338Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/test_core_isolation.c",
|
||||
"action": "edit",
|
||||
"tokens": 143,
|
||||
"at": "2026-08-19T10:16:49.447Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/CLAUDE.md",
|
||||
"action": "edit",
|
||||
"tokens": 119,
|
||||
"at": "2026-08-19T10:17:01.549Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/src/sim.c",
|
||||
"action": "edit",
|
||||
"tokens": 352,
|
||||
"at": "2026-08-19T10:17:12.748Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/src/sim_loop.c",
|
||||
"action": "edit",
|
||||
"tokens": 416,
|
||||
"at": "2026-08-19T10:17:30.876Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/src/sim_loop.c",
|
||||
"action": "edit",
|
||||
"tokens": 128,
|
||||
"at": "2026-08-19T10:17:37.208Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/src/sim_loop.c",
|
||||
"action": "edit",
|
||||
"tokens": 57,
|
||||
"at": "2026-08-19T10:17:41.924Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/src/sim_loop.c",
|
||||
"action": "edit",
|
||||
"tokens": 62,
|
||||
"at": "2026-08-19T10:17:47.090Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/src/sim_loop.c",
|
||||
"action": "edit",
|
||||
"tokens": 83,
|
||||
"at": "2026-08-19T10:17:53.199Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/src/sim_loop.c",
|
||||
"action": "edit",
|
||||
"tokens": 24,
|
||||
"at": "2026-08-19T10:17:57.513Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/controller/cluster.c",
|
||||
"action": "edit",
|
||||
"tokens": 866,
|
||||
"at": "2026-08-19T10:18:52.483Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/controller/cluster.c",
|
||||
"action": "edit",
|
||||
"tokens": 96,
|
||||
"at": "2026-08-19T10:18:58.919Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/controller/cluster.c",
|
||||
"action": "edit",
|
||||
"tokens": 44,
|
||||
"at": "2026-08-19T10:19:05.082Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/controller/testsuite/test_sched_admin_dispatch.c",
|
||||
"action": "edit",
|
||||
"tokens": 837,
|
||||
"at": "2026-08-19T10:20:15.384Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/controller/testsuite/test_sched_admin_dispatch.c",
|
||||
"action": "edit",
|
||||
"tokens": 51,
|
||||
"at": "2026-08-19T10:20:43.404Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/.kiro/specs/scheduler-core/requirements.md",
|
||||
"action": "edit",
|
||||
"tokens": 367,
|
||||
"at": "2026-08-19T10:21:22.727Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/.kiro/specs/scheduler-core/design.md",
|
||||
"action": "edit",
|
||||
"tokens": 468,
|
||||
"at": "2026-08-19T10:21:44.124Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/.kiro/specs/scheduler-core/design.md",
|
||||
"action": "edit",
|
||||
"tokens": 92,
|
||||
"at": "2026-08-19T10:21:50.707Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/.kiro/specs/scheduler-core/design.md",
|
||||
"action": "edit",
|
||||
"tokens": 199,
|
||||
"at": "2026-08-19T10:22:03.641Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/.kiro/specs/resource-value-verification/design.md",
|
||||
"action": "edit",
|
||||
"tokens": 914,
|
||||
"at": "2026-08-19T10:23:01.897Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/.kiro/specs/resource-value-verification/design.md",
|
||||
"action": "edit",
|
||||
"tokens": 612,
|
||||
"at": "2026-08-19T10:23:15.150Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/src/core_events.c",
|
||||
"action": "edit",
|
||||
"tokens": 374,
|
||||
"at": "2026-08-19T10:23:48.111Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/frontend/apiserver/backend/handler.go",
|
||||
"action": "edit",
|
||||
"tokens": 290,
|
||||
"at": "2026-08-19T10:24:01.558Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/test_sim_event_loop.c",
|
||||
"action": "edit",
|
||||
"tokens": 426,
|
||||
"at": "2026-08-19T10:30:59.070Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/test_sim_event_loop.c",
|
||||
"action": "edit",
|
||||
"tokens": 190,
|
||||
"at": "2026-08-19T10:31:10.338Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/test_sim_event_loop.c",
|
||||
"action": "edit",
|
||||
"tokens": 116,
|
||||
"at": "2026-08-19T10:31:18.187Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/scratch_gen_resource_replay.c",
|
||||
"action": "create",
|
||||
"tokens": 2930,
|
||||
"at": "2026-08-19T10:54:45.882Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/meson.build",
|
||||
"action": "edit",
|
||||
"tokens": 249,
|
||||
"at": "2026-08-19T10:54:56.241Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/src/scheduler/testsuite/meson.build",
|
||||
"action": "edit",
|
||||
"tokens": 74,
|
||||
"at": "2026-08-19T10:56:30.335Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/.kiro/specs/scheduler-core/tasks.md",
|
||||
"action": "edit",
|
||||
"tokens": 1248,
|
||||
"at": "2026-08-19T10:58:45.163Z"
|
||||
},
|
||||
{
|
||||
"file": "/home/rshen/projects/xce/worktrees/scheduler-framework/.kiro/specs/scheduler-core/spec.json",
|
||||
"action": "edit",
|
||||
"tokens": 12,
|
||||
"at": "2026-08-19T10:58:57.657Z"
|
||||
}
|
||||
],
|
||||
"edit_counts": {
|
||||
"src/scheduler/testsuite/test_repl_breakpoints_inspection.c": 1
|
||||
"src/scheduler/testsuite/test_repl_breakpoints_inspection.c": 1,
|
||||
"src/scheduler/testsuite/test_sim_startup.c": 2,
|
||||
"src/scheduler/testsuite/test_sim_event_loop.c": 5,
|
||||
"src/scheduler/meson.build": 1,
|
||||
"src/scheduler/testsuite/meson.build": 3,
|
||||
"src/scheduler/testsuite/test_core_isolation.c": 5,
|
||||
"src/scheduler/CLAUDE.md": 1,
|
||||
"src/scheduler/src/sim.c": 1,
|
||||
"src/scheduler/src/sim_loop.c": 6,
|
||||
"src/controller/cluster.c": 3,
|
||||
"src/controller/testsuite/test_sched_admin_dispatch.c": 2,
|
||||
".kiro/specs/scheduler-core/requirements.md": 1,
|
||||
".kiro/specs/scheduler-core/design.md": 3,
|
||||
".kiro/specs/resource-value-verification/design.md": 2,
|
||||
"src/scheduler/src/core_events.c": 1,
|
||||
"src/frontend/apiserver/backend/handler.go": 1,
|
||||
"src/scheduler/testsuite/scratch_gen_resource_replay.c": 1,
|
||||
".kiro/specs/scheduler-core/tasks.md": 1,
|
||||
".kiro/specs/scheduler-core/spec.json": 1
|
||||
},
|
||||
"anatomy_hits": 4,
|
||||
"anatomy_misses": 2,
|
||||
"repeated_reads_warned": 7,
|
||||
"anatomy_hits": 24,
|
||||
"anatomy_misses": 10,
|
||||
"repeated_reads_warned": 34,
|
||||
"cerebrum_warnings": 0,
|
||||
"stop_count": 0
|
||||
"stop_count": 16
|
||||
}
|
||||
|
|
@ -2398,3 +2398,68 @@
|
|||
| Time | Action | File(s) | Outcome | ~Tokens |
|
||||
|------|--------|---------|---------|--------|
|
||||
| 02:46 | Edited src/scheduler/testsuite/test_repl_breakpoints_inspection.c | expanded (+8 lines) | ~210 |
|
||||
| 02:40 | Dispatched build agent: prior session's fork agent never reported back before session end; build binaries were stale (Aug 18 14:27) vs source edits (Aug 19 09:18-09:27) | src/xce.build/scheduler | scheduler asan build: 40/42 tests pass | ~1200 |
|
||||
| 02:47 | Root-caused and fixed test bug: test_step_echoes_events_before_session_decisions reused jobid 6161 for both JOBS_PUSH and JOBSTATECHANGE, transitioning the real pushed job PEND->RUN before the session ran so the picker never produced a PEND decision | src/scheduler/testsuite/test_repl_breakpoints_inspection.c | fixed, rebuilt, confirmed PASS (41/42, only pre-existing bug-1560 remains) | ~9000 |
|
||||
| 02:52 | Committed 5 logical commits per user request: worker bug-1589 fix, controller bug-1588 fix, scheduler task-12.5+related-bugs fix, xce-deploy-ha real-worker feature, wolf session-state chore | (see git log 70fe9e4..4dcfa80) | 5 commits, all builds/tests green (except pre-existing bug-1560) | ~2500 |
|
||||
| 03:07-03:30 | Ran retrospective design-quality review (kiro-validate-design) against scheduler-core and resource-value-verification design.md, both GO with tracked findings; user asked to fix all 5 findings | .kiro/specs/scheduler-core/{design,requirements}.md, .kiro/specs/resource-value-verification/design.md | both GO, 5 findings identified | ~15000 |
|
||||
| 03:30-04:10 | Fixed all 5 review findings: (1) tlog virtual-clock activation wired into sim.c/sim_loop.c with new tests; (2) core-isolation nm-audit extended to core_events.c/core_session.c/core_clock.c; (3) OPCode_schedAdmin now admission-gated via vtaskmgr_blocking_acquire/release, new fast-reject test; (4) resource-value-verification System Flows diagram + Error Handling doc gaps closed; (5) grep-pattern-fragility warning comments added at both log call sites | src/scheduler/{src,testsuite}/*, src/controller/cluster.c + testsuite, src/frontend/apiserver/backend/handler.go, 2 design.md files | scheduler 41/42, controller 1109/1111 (both only pre-existing known failures), apiserver fully green | ~85000 |
|
||||
| 04:10 | Found + fixed a genuine pre-existing ASAN stack-use-after-return (bug-1598) that my new tlog-deactivation test exposed: sim_picker_thread_fn is never joined, so a stack-local `sim` in test_sim_event_loop.c was a latent UAR; fixed via `static` storage on the 3 affected fixture declarations | src/scheduler/testsuite/test_sim_event_loop.c | confirmed fixed via 2 independent clean runs | ~12000 |
|
||||
| 04:35-05:05 | Ran /kiro-verify-completion against task 12.5 with fresh evidence (not trusting prior self-reports): fresh build+test, live subprocess smoke test of real xce_scheduler binary for gaps 1/2/3/5, direct source inspection of all 5 fixes. Returned NOT_VERIFIED solely on bookkeeping (tasks.md checkbox unchecked, no _Verified line) + gap 4 (info resource) unit-tested but not live-tested | .kiro/specs/scheduler-core/tasks.md | NOT_VERIFIED, 2 concrete gaps identified | ~35000 |
|
||||
| 05:05-05:20 | Closed both gaps: dispatched agent to hand-build a real .replay fixture with resource data and live-verify `info resource` against the real binary (confirmed all 3 assertions), then flipped task 12.5's checkbox to [x] with a full _Verified: line citing all evidence; bumped spec.json updated_at | .kiro/specs/scheduler-core/tasks.md, spec.json | task 12.5 now genuinely complete per spec convention | ~15000 |
|
||||
| 02:50 | Session end: 1 writes across 1 files (test_repl_breakpoints_inspection.c) | 6 reads | ~35060 tok |
|
||||
| 02:56 | Session end: 1 writes across 1 files (test_repl_breakpoints_inspection.c) | 15 reads | ~105784 tok |
|
||||
| 02:57 | Session end: 1 writes across 1 files (test_repl_breakpoints_inspection.c) | 15 reads | ~105784 tok |
|
||||
| 02:58 | Session end: 1 writes across 1 files (test_repl_breakpoints_inspection.c) | 15 reads | ~105784 tok |
|
||||
| 03:08 | Session end: 1 writes across 1 files (test_repl_breakpoints_inspection.c) | 17 reads | ~136722 tok |
|
||||
| 03:13 | Edited src/scheduler/testsuite/test_sim_startup.c | modified 1b() | ~538 |
|
||||
| 03:13 | Edited src/scheduler/testsuite/test_sim_startup.c | 2→3 lines | ~59 |
|
||||
| 03:13 | Edited src/scheduler/testsuite/test_sim_event_loop.c | modified returns() | ~639 |
|
||||
| 03:14 | Edited src/scheduler/testsuite/test_sim_event_loop.c | 4→5 lines | ~71 |
|
||||
| 03:15 | Edited src/scheduler/meson.build | modified 1() | ~449 |
|
||||
| 03:15 | Edited src/scheduler/testsuite/meson.build | expanded (+6 lines) | ~177 |
|
||||
| 03:15 | Edited src/scheduler/testsuite/test_core_isolation.c | modified up() | ~963 |
|
||||
| 03:16 | Edited src/scheduler/testsuite/test_core_isolation.c | modified XCE_CORE_C_PATH() | ~172 |
|
||||
| 03:16 | Edited src/scheduler/testsuite/test_core_isolation.c | modified test_core_source_paths_are_readable() | ~1279 |
|
||||
| 03:16 | Edited src/scheduler/testsuite/test_core_isolation.c | 9→10 lines | ~136 |
|
||||
| 03:16 | Edited src/scheduler/testsuite/test_core_isolation.c | 7→8 lines | ~134 |
|
||||
| 03:17 | Edited src/scheduler/CLAUDE.md | inline fix | ~111 |
|
||||
| 03:17 | Edited src/scheduler/src/sim.c | modified globally() | ~329 |
|
||||
| 03:17 | Edited src/scheduler/src/sim_loop.c | modified to() | ~388 |
|
||||
| 03:17 | Edited src/scheduler/src/sim_loop.c | modified if() | ~119 |
|
||||
| 03:17 | Edited src/scheduler/src/sim_loop.c | 5→6 lines | ~54 |
|
||||
| 03:17 | Edited src/scheduler/src/sim_loop.c | 4→5 lines | ~58 |
|
||||
| 03:17 | Edited src/scheduler/src/sim_loop.c | 9→10 lines | ~78 |
|
||||
| 03:17 | Edited src/scheduler/src/sim_loop.c | 4→6 lines | ~22 |
|
||||
| 03:18 | Edited src/controller/cluster.c | added 1 condition(s) | ~809 |
|
||||
| 03:18 | Edited src/controller/cluster.c | modified if() | ~90 |
|
||||
| 03:19 | Edited src/controller/cluster.c | 5→6 lines | ~41 |
|
||||
| 03:20 | Edited src/controller/testsuite/test_sched_admin_dispatch.c | modified 2b() | ~781 |
|
||||
| 03:20 | Edited src/controller/testsuite/test_sched_admin_dispatch.c | 2→3 lines | ~47 |
|
||||
| 03:21 | Edited .kiro/specs/scheduler-core/requirements.md | 2→5 lines | ~342 |
|
||||
| 03:21 | Edited .kiro/specs/scheduler-core/design.md | inline fix | ~437 |
|
||||
| 03:21 | Edited .kiro/specs/scheduler-core/design.md | inline fix | ~86 |
|
||||
| 03:22 | Edited .kiro/specs/scheduler-core/design.md | inline fix | ~186 |
|
||||
| 03:23 | Edited .kiro/specs/resource-value-verification/design.md | 28→33 lines | ~854 |
|
||||
| 03:23 | Edited .kiro/specs/resource-value-verification/design.md | 1→2 lines | ~571 |
|
||||
| 03:23 | Edited src/scheduler/src/core_events.c | modified spec() | ~349 |
|
||||
| 03:24 | Edited src/frontend/apiserver/backend/handler.go | modified 2() | ~271 |
|
||||
| 03:24 | Session end: 33 writes across 14 files (test_repl_breakpoints_inspection.c, test_sim_startup.c, test_sim_event_loop.c, meson.build, test_core_isolation.c) | 28 reads | ~192736 tok |
|
||||
| 03:25 | Session end: 33 writes across 14 files (test_repl_breakpoints_inspection.c, test_sim_startup.c, test_sim_event_loop.c, meson.build, test_core_isolation.c) | 29 reads | ~192736 tok |
|
||||
| 03:27 | Session end: 33 writes across 14 files (test_repl_breakpoints_inspection.c, test_sim_startup.c, test_sim_event_loop.c, meson.build, test_core_isolation.c) | 30 reads | ~192736 tok |
|
||||
| 10:28 | Verified comment-only change (handler.go OnResourceLocationUpdated) via nix build+test | apiserver/backend/handler.go | make build-server plain target pre-existing broken (bug-1597, missing CGO flags, confirmed unrelated via git-stash A/B); workaround build compiled clean (exit 0) and make test all green (store/utils/controllers/backend/backend-auth/xcecli all ok) | ~15k |
|
||||
| 03:28 | Session end: 33 writes across 14 files (test_repl_breakpoints_inspection.c, test_sim_startup.c, test_sim_event_loop.c, meson.build, test_core_isolation.c) | 30 reads | ~192736 tok |
|
||||
| 03:30 | Edited src/scheduler/testsuite/test_sim_event_loop.c | modified exists() | ~398 |
|
||||
| 03:31 | Edited src/scheduler/testsuite/test_sim_event_loop.c | modified returns() | ~178 |
|
||||
| 03:31 | Edited src/scheduler/testsuite/test_sim_event_loop.c | modified run_sim_and_capture_stdout() | ~109 |
|
||||
| 03:31 | Session end: 36 writes across 14 files (test_repl_breakpoints_inspection.c, test_sim_startup.c, test_sim_event_loop.c, meson.build, test_core_isolation.c) | 30 reads | ~193797 tok |
|
||||
| 03:34 | Session end: 36 writes across 14 files (test_repl_breakpoints_inspection.c, test_sim_startup.c, test_sim_event_loop.c, meson.build, test_core_isolation.c) | 30 reads | ~193797 tok |
|
||||
| 03:35 | Session end: 36 writes across 14 files (test_repl_breakpoints_inspection.c, test_sim_startup.c, test_sim_event_loop.c, meson.build, test_core_isolation.c) | 30 reads | ~193797 tok |
|
||||
| 03:41 | Session end: 36 writes across 14 files (test_repl_breakpoints_inspection.c, test_sim_startup.c, test_sim_event_loop.c, meson.build, test_core_isolation.c) | 31 reads | ~225928 tok |
|
||||
| 03:46 | Session end: 36 writes across 14 files (test_repl_breakpoints_inspection.c, test_sim_startup.c, test_sim_event_loop.c, meson.build, test_core_isolation.c) | 31 reads | ~225928 tok |
|
||||
| 03:50 | Session end: 36 writes across 14 files (test_repl_breakpoints_inspection.c, test_sim_startup.c, test_sim_event_loop.c, meson.build, test_core_isolation.c) | 31 reads | ~226083 tok |
|
||||
| 03:54 | Created src/scheduler/testsuite/scratch_gen_resource_replay.c | — | ~2735 |
|
||||
| 03:54 | Edited src/scheduler/testsuite/meson.build | expanded (+10 lines) | ~233 |
|
||||
| 03:56 | Edited src/scheduler/testsuite/meson.build | reduced (-10 lines) | ~69 |
|
||||
| 03:58 | Edited .kiro/specs/scheduler-core/tasks.md | 6→7 lines | ~1165 |
|
||||
| 03:58 | Edited .kiro/specs/scheduler-core/spec.json | inline fix | ~12 |
|
||||
| 03:59 | Session end: 41 writes across 17 files (test_repl_breakpoints_inspection.c, test_sim_startup.c, test_sim_event_loop.c, meson.build, test_core_isolation.c) | 34 reads | ~245953 tok |
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
Loading…
Add table
Add a link
Reference in a new issue