add weekly report and change logs
This commit is contained in:
parent
a221b6ae58
commit
a6b735b6f2
2 changed files with 373 additions and 0 deletions
252
CHANGES.md
252
CHANGES.md
|
|
@ -3,6 +3,258 @@
|
|||
Auto-generated by /xce-changes — component-grouped, newest week first.
|
||||
`.wolf/` bookkeeping commits are excluded.
|
||||
|
||||
## Aug 30, 2026 - Sep 05, 2026
|
||||
<!-- window: 2026-08-30..2026-09-05 -->
|
||||
|
||||
### controller
|
||||
- `12c6a59c` feat(eliminate-frontend-makefile): task 3.4 - update source code comments (2026-09-03)
|
||||
- `1c4cf8bd` fix(worker-rejoin-resync): task 7.5 - bound the outbound reconciliation query (2026-09-02)
|
||||
- `613b80e9` test(worker-rejoin-resync): task 7.4 - large job-count pagination correctness (2026-09-02)
|
||||
- `165fc158` test(worker-rejoin-resync): task 7.3 - burst recovery unbounded by count (2026-09-02)
|
||||
- `bceca0c9` test(worker-rejoin-resync): task 7.2 - failover-during-job recovery (2026-09-01)
|
||||
- `8650cfd9` test(worker-rejoin-resync): task 7.1 - full rejoin round trip (2026-09-01)
|
||||
- `9f655c61` feat(worker-rejoin-resync): task 6.2 - route job-state report to reconciliation (2026-09-01)
|
||||
- `a3d12d37` feat(worker-rejoin-resync): task 6.1 - trigger reconciliation on rejoin (2026-09-01)
|
||||
- `eec2596c` feat(worker-rejoin-resync): apply reported outcomes idempotently (task 3.2) (2026-09-01)
|
||||
- `442fcf45` feat(worker-rejoin-resync): reconciliation round lifecycle (task 3.1) (2026-09-01)
|
||||
- `ea0658ec` feat(worker-rejoin-resync): fail-fast disconnected-worker dispatch (task 2.3) (2026-09-01)
|
||||
- `ecfb4c94` feat(worker-rejoin-resync): reconcile unqueried job assignment (task 2.2) (2026-09-01)
|
||||
- `cfe66dde` feat(worker-rejoin-resync): reconcile unrecognized job to pending (task 2.1) (2026-09-01)
|
||||
- `026fe97b` fix(bug-1737): scheduler can never learn a job finished successfully (2026-08-31)
|
||||
- `0e96a69f` fix(bug-1429): persist_stop() before commit_engine_fini() in 21 fixtures (2026-08-30) — fixes a controller_unit_test hang/deadlock after HA quorum was lost during teardown
|
||||
- `5c2c6664` fix(bug-1736): capture memory_swap/tasks_current, compute peak_cpu_time (2026-08-30)
|
||||
- `d8a37fa2` fix(bug-1735): job memory/CPU usage always read 0 in scheduler replay (2026-08-30)
|
||||
- `a02d4777` fix(bug-1561): fix flaky test_conf_trim_keeps_only_n_versions/test_conf_get_saved_tag_no_config (2026-08-30)
|
||||
|
||||
### worker
|
||||
- `b6a2f5ec` fix(worker-rejoin-resync): task 4.4 - gate periodic load-indices send on resync_pending (2026-09-02)
|
||||
- `a3d12d37` feat(worker-rejoin-resync): task 6.1 - trigger reconciliation on rejoin (2026-09-01)
|
||||
- `5b699736` feat(worker-rejoin-resync): task 4.3 - wire query receipt and post-join gate (2026-09-01)
|
||||
- `cfe58c3d` feat(worker-rejoin-resync): task 4.2 - worker answers job-state query with extras (2026-09-01)
|
||||
- `e31012e8` feat(worker-rejoin-resync): task 4.1 - bounded finished-job retention on worker (2026-09-01)
|
||||
|
||||
### scheduler
|
||||
- `c009f108` test(scheduler-allocation): end-to-end multi-host parallel job proof (2026-09-05)
|
||||
- `e8c2087d` feat(scheduler-allocation): clear unsatisfiable-resreq cache per session (2026-09-05)
|
||||
- `5c022d0b` feat(scheduler-allocation): wire unsatisfiable-resreq cache into dispatch (2026-09-04)
|
||||
- `9d23060b` test(scheduler-allocation): prove parjob's shared resource commits once (2026-09-04)
|
||||
- `3d5e1c32` feat(scheduler-allocation): implement parjob's per-bucket allocation loop (2026-09-04)
|
||||
- `6ca717fb` feat(scheduler-allocation): implement parjob's combinatorial plan generation (2026-09-04)
|
||||
- `9d64ba0d` test(scheduler-allocation): prove seqjob rewrap preserves host selection (2026-09-04)
|
||||
- `b18c5f78` feat(scheduler-allocation): rewrap seqjob allocation onto shared iterators (2026-09-04)
|
||||
- `86c86352` feat(scheduler-allocation): implement the unsatisfiable-resreq cache (2026-09-04)
|
||||
- `d9831b0b` feat(scheduler-allocation): implement the shared host-candidate iterator (2026-09-04)
|
||||
- `d0024eb7` feat(scheduler-allocation): add per-host resource-debit reversal (2026-09-04)
|
||||
- `d8b5cf87` feat(scheduler-allocation): add forward-compatible allocation-mode field (2026-09-04)
|
||||
- `5a670676` feat(scheduler-allocation): add resreq fingerprinting for the cache (2026-09-04)
|
||||
- `d5d2c1ca` feat(scheduler-allocation): add lazy plan/host iterator contracts (2026-09-04)
|
||||
- `88b0ac99` fix(scheduler): close post-failover rejoin detection gap (bug-1760) (2026-09-04) — new primary's dispatched job stayed PEND 60s+ with zero scheduler activity
|
||||
- `9dcffce8` test(scheduler-sim-reconcile-terminal-guard): task 2.2 - final REPL-level verification (2026-09-03)
|
||||
- `b722de4f` fix(scheduler-sim-reconcile-terminal-guard): task 1.2 - bug-1764 terminal-state guard (2026-09-03) — info worker kept listing already-DONE/EXIT jobs as still assigned
|
||||
- `ff05c37b` test(scheduler-sim-reconcile-terminal-guard): task 1.1 - RED tests for terminal guard (2026-09-03)
|
||||
- `56e007cb` feat(scheduler-sim-step-verbosity): task 1.2 - implement preamble deduplication (2026-09-03)
|
||||
- `6a2aa47f` test(scheduler-sim-step-verbosity): task 1.1 - RED tests for preamble dedup (2026-09-03)
|
||||
- `bd973973` feat(scheduler-sim-repl-verbosity): task 9.2 - discoverable help and break event types (2026-09-02)
|
||||
- `bd77eea0` test(scheduler-sim-repl-verbosity): task 9.1 - RED tests for help/break discoverability (2026-09-02)
|
||||
- `9db99964` feat(scheduler-sim-repl-verbosity): task 8.2 - implement resource table rendering (2026-09-02)
|
||||
- `e48e222b` feat(scheduler-sim-repl-verbosity): task 8.1 - RED-phase resource table tests (2026-09-02)
|
||||
- `5ee1d6cb` feat(scheduler-sim-repl-verbosity): task 7.2 - implement resource formatting helpers (2026-09-02)
|
||||
- `11fe6194` feat(scheduler-sim-repl-verbosity): task 7.1 - RED-phase resource formatting tests (2026-09-02)
|
||||
- `da6ba96e` feat(scheduler-sim-repl-verbosity): task 6.2 - implement dynamic REPL prompt (2026-09-02)
|
||||
- `99fa45c4` feat(scheduler-sim-repl-verbosity): task 6.1 - RED-phase REPL prompt tests (2026-09-02)
|
||||
- `d9109b7c` feat(scheduler-sim-repl-verbosity): task 4.2 - retire redundant per-decision echo (2026-09-02)
|
||||
- `3d08a43a` feat(scheduler-sim-repl-verbosity): task 4.1 - RED-phase sim_print_decision Gate tests (2026-09-02)
|
||||
- `cc78d091` feat(scheduler-sim-repl-verbosity): task 3.2 - implement Event Echo Gate (2026-09-02)
|
||||
- `f587a441` feat(scheduler-sim-repl-verbosity): task 3.1 - RED-phase Event Echo Gate tests (2026-09-02)
|
||||
- `921c72bb` feat(scheduler-sim-repl-verbosity): task 2.2 - implement Round Rollup logic (2026-09-02)
|
||||
- `0c6ccb03` feat(scheduler-sim-repl-verbosity): task 2.1 - RED-phase Round Rollup tests (2026-09-02)
|
||||
- `140f0900` fix(scheduler): repl_cmd_delete() rejects non-numeric/invalid breakpoint ids (2026-09-02)
|
||||
- `b639fae1` feat(scheduler-sim-repl-verbosity): task 1.2 - implement job identifier range compression (2026-09-02)
|
||||
- `f165e3a7` feat(scheduler-sim-repl-verbosity): task 1.1 - RED-phase tests for job identifier range compression (2026-09-02)
|
||||
- `80d5a062` feat(worker-rejoin-resync): task 5 - exclude reconciling host from dispatch (2026-09-01)
|
||||
- `eb06f16d` fix(scheduler): wire mem reservation reconciliation into live rusage event path (2026-09-01)
|
||||
- `f4103e0e` fix(scheduler): charge unreserved jobs' real memory usage too (2026-08-31)
|
||||
- `87a2a46b` feat(scheduler): reconcile mem reservation against real usage (2026-08-31)
|
||||
- `fc559948` fix(bug-1727): 'info worker'/'info resource' double-counted used capacity (2026-08-31)
|
||||
- `0e78f863` fix(bug-1738,bug-1739): terminal-state regression + phantom RUN in JOBS_PUSH merge (2026-08-31) — phantom RUN entries exceeded the worker's real slot capacity
|
||||
- `026fe97b` fix(bug-1737): scheduler can never learn a job finished successfully (2026-08-31)
|
||||
- `b0476a84` fix(bug-1734): merge redelivered job state instead of discarding it (2026-08-30)
|
||||
- `dfcd345d` fix(bug-1560): symlink-escape test asserted the wrong error string (2026-08-30)
|
||||
- `38437e57` fix(scheduler): don't hand an orphaned duplicate job to the picker (2026-08-30)
|
||||
|
||||
### autoscaler
|
||||
- `0b1e3107` remove extralib (2026-09-04)
|
||||
- `68bc86bc` build: make Go and Python builds fully offline (hermetic Nix build) (2026-09-03)
|
||||
|
||||
### frontend
|
||||
- `6fabdba9` feat(eliminate-frontend-makefile): task 4.2 - delete src/frontend/Makefile (2026-09-03)
|
||||
- `12c6a59c` feat(eliminate-frontend-makefile): task 3.4 - update source code comments (2026-09-03)
|
||||
- `f5fbc6bc` feat(eliminate-frontend-makefile): task 3.3 - update live documentation (11 files) (2026-09-03)
|
||||
- `6b105125` feat(eliminate-frontend-makefile): task 3.2 - fix deploy-ha.sh/run.sh messages (2026-09-03)
|
||||
- `6c3ff201` feat(eliminate-frontend-makefile): task 1.1 - Meson backend CGO test target (2026-09-03)
|
||||
- `68bc86bc` build: make Go and Python builds fully offline (hermetic Nix build) (2026-09-03)
|
||||
- `728496be` fix(apiserver): populate hosts-list slot usage counts (2026-09-01)
|
||||
- `343639fc` fix(apiserver): stop job-control panic on int/int64 type mismatch (2026-09-01)
|
||||
|
||||
### xceadmin
|
||||
- `68bc86bc` build: make Go and Python builds fully offline (hermetic Nix build) (2026-09-03)
|
||||
- `ce3d391e` feat(worker-rejoin-resync): wire config fields into xceadmin build (task 1.5) (2026-09-01)
|
||||
|
||||
### lib
|
||||
- `5a670676` feat(scheduler-allocation): add resreq fingerprinting for the cache (2026-09-04)
|
||||
- `88b0ac99` fix(scheduler): close post-failover rejoin detection gap (bug-1760) (2026-09-04) — new primary's dispatched job stayed PEND 60s+ with zero scheduler activity
|
||||
- `0b1e3107` remove extralib (2026-09-04)
|
||||
- `9f3dfe79` feat(worker-rejoin-resync): add reconciliation wire codec helpers (task 1.4) (2026-09-01)
|
||||
- `026fe97b` fix(bug-1737): scheduler can never learn a job finished successfully (2026-08-31)
|
||||
|
||||
### include
|
||||
- `5a670676` feat(scheduler-allocation): add resreq fingerprinting for the cache (2026-09-04)
|
||||
- `88b0ac99` fix(scheduler): close post-failover rejoin detection gap (bug-1760) (2026-09-04) — new primary's dispatched job stayed PEND 60s+ with zero scheduler activity
|
||||
- `0b1e3107` remove extralib (2026-09-04)
|
||||
- `9f655c61` feat(worker-rejoin-resync): task 6.2 - route job-state report to reconciliation (2026-09-01)
|
||||
- `a3d12d37` feat(worker-rejoin-resync): task 6.1 - trigger reconciliation on rejoin (2026-09-01)
|
||||
- `9f3dfe79` feat(worker-rejoin-resync): add reconciliation wire codec helpers (task 1.4) (2026-09-01)
|
||||
- `1260d913` feat(worker-rejoin-resync): add worker reconciling status type (task 1.2) (2026-09-01)
|
||||
- `1ff57ef6` feat(worker-rejoin-resync): define reconciliation wire protocol (task 1.1) (2026-09-01)
|
||||
- `026fe97b` fix(bug-1737): scheduler can never learn a job finished successfully (2026-08-31)
|
||||
- `5c2c6664` fix(bug-1736): capture memory_swap/tasks_current, compute peak_cpu_time (2026-08-30)
|
||||
|
||||
### schema
|
||||
- `a3d12d37` feat(worker-rejoin-resync): task 6.1 - trigger reconciliation on rejoin (2026-09-01)
|
||||
- `1ff57ef6` feat(worker-rejoin-resync): define reconciliation wire protocol (task 1.1) (2026-09-01)
|
||||
- `026fe97b` fix(bug-1737): scheduler can never learn a job finished successfully (2026-08-31)
|
||||
- `5c2c6664` fix(bug-1736): capture memory_swap/tasks_current, compute peak_cpu_time (2026-08-30)
|
||||
|
||||
### docs
|
||||
- `f5fbc6bc` feat(eliminate-frontend-makefile): task 3.3 - update live documentation (11 files) (2026-09-03)
|
||||
|
||||
### specs
|
||||
- `ecfb0a3c` chore(scheduler-allocation): confirm HA soak regression (task 6.2) (2026-09-05)
|
||||
- `58e6a855` chore(scheduler-allocation): confirm full regression pass (task 6.1) (2026-09-05)
|
||||
- `c009f108` test(scheduler-allocation): end-to-end multi-host parallel job proof (2026-09-05)
|
||||
- `e8c2087d` feat(scheduler-allocation): clear unsatisfiable-resreq cache per session (2026-09-05)
|
||||
- `5c022d0b` feat(scheduler-allocation): wire unsatisfiable-resreq cache into dispatch (2026-09-04)
|
||||
- `9d23060b` test(scheduler-allocation): prove parjob's shared resource commits once (2026-09-04)
|
||||
- `3d5e1c32` feat(scheduler-allocation): implement parjob's per-bucket allocation loop (2026-09-04)
|
||||
- `6ca717fb` feat(scheduler-allocation): implement parjob's combinatorial plan generation (2026-09-04)
|
||||
- `9d64ba0d` test(scheduler-allocation): prove seqjob rewrap preserves host selection (2026-09-04)
|
||||
- `b18c5f78` feat(scheduler-allocation): rewrap seqjob allocation onto shared iterators (2026-09-04)
|
||||
- `86c86352` feat(scheduler-allocation): implement the unsatisfiable-resreq cache (2026-09-04)
|
||||
- `d9831b0b` feat(scheduler-allocation): implement the shared host-candidate iterator (2026-09-04)
|
||||
- `d0024eb7` feat(scheduler-allocation): add per-host resource-debit reversal (2026-09-04)
|
||||
- `d8b5cf87` feat(scheduler-allocation): add forward-compatible allocation-mode field (2026-09-04)
|
||||
- `5a670676` feat(scheduler-allocation): add resreq fingerprinting for the cache (2026-09-04)
|
||||
- `d5d2c1ca` feat(scheduler-allocation): add lazy plan/host iterator contracts (2026-09-04)
|
||||
- `d1a3e158` docs(scheduler-allocation): fix xce_host_iter_t.next() return type (2026-09-04)
|
||||
- `48c789df` docs(scheduler-allocation): approve requirements, add design and tasks (2026-09-04)
|
||||
- `375c49ec` docs(eliminate-frontend-makefile): mark task 4.2 and feature complete (2026-09-03)
|
||||
- `91d99134` docs(eliminate-frontend-makefile): task 4.1 - final go/no-go gate passed (2026-09-03)
|
||||
- `c554ed25` docs(eliminate-frontend-makefile): task 3.6 - live-reference sweep clean (2026-09-03)
|
||||
- `87707ff1` docs(eliminate-frontend-makefile): task 3.5 - verified regression suite parity (2026-09-03)
|
||||
- `12c6a59c` feat(eliminate-frontend-makefile): task 3.4 - update source code comments (2026-09-03)
|
||||
- `f5fbc6bc` feat(eliminate-frontend-makefile): task 3.3 - update live documentation (11 files) (2026-09-03)
|
||||
- `6b105125` feat(eliminate-frontend-makefile): task 3.2 - fix deploy-ha.sh/run.sh messages (2026-09-03)
|
||||
- `ff051e0e` feat(eliminate-frontend-makefile): task 3.1 - fix run-resource-verify.sh discovery (2026-09-03)
|
||||
- `3c199f37` feat(eliminate-frontend-makefile): task 2 - migrate xce-build.sh off Make (2026-09-03)
|
||||
- `54b11bc2` docs(eliminate-frontend-makefile): task 1.2 - verified real CGO path coverage (2026-09-03)
|
||||
- `6c3ff201` feat(eliminate-frontend-makefile): task 1.1 - Meson backend CGO test target (2026-09-03)
|
||||
- `be9a03a3` docs(eliminate-frontend-makefile): add approved spec (requirements/design/tasks) (2026-09-03)
|
||||
- `4a224e8c` Simplify AppImage build to nix build .#xce.appImage (2026-09-03)
|
||||
- `df4b386a` docs(specs): reconcile stale phase/checkbox metadata (3 more specs) (2026-09-03)
|
||||
- `8557b869` docs(xce-test-rc-file): reconcile stale phase/checkbox metadata (2026-09-03)
|
||||
- `1abab949` docs(specs): reconcile stale metadata found auditing stalled specs (2026-09-03)
|
||||
- `b14e53f5` docs(worker-rejoin-resync): reconcile stale phase metadata (2026-09-03)
|
||||
- `177e9b29` docs(xce-node-manager): add spec (requirements/design approved) (2026-09-03)
|
||||
- `6bb37cf4` docs(scheduler-sim-diagnostics): add draft spec (requirements only) (2026-09-03)
|
||||
- `0f7699de` docs(scheduler-primary-lifecycle): add superseded spec (requirements/design/tasks) (2026-09-03)
|
||||
- `9dcffce8` test(scheduler-sim-reconcile-terminal-guard): task 2.2 - final REPL-level verification (2026-09-03)
|
||||
- `4b26120f` docs(scheduler-sim-reconcile-terminal-guard): task 2.1 - validate no regression (2026-09-03)
|
||||
- `b722de4f` fix(scheduler-sim-reconcile-terminal-guard): task 1.2 - bug-1764 terminal-state guard (2026-09-03) — info worker kept listing already-DONE/EXIT jobs as still assigned
|
||||
- `ff05c37b` test(scheduler-sim-reconcile-terminal-guard): task 1.1 - RED tests for terminal guard (2026-09-03)
|
||||
- `f854d9fa` docs(scheduler-sim-reconcile-terminal-guard): add approved spec (requirements/design/tasks) (2026-09-03)
|
||||
- `56bd3103` docs(scheduler-sim-step-verbosity): task 2.2 - final regression and transcript verification (2026-09-03)
|
||||
- `19c9406d` docs(scheduler-sim-step-verbosity): task 2.1 - validate no out-of-scope changes (2026-09-03)
|
||||
- `56e007cb` feat(scheduler-sim-step-verbosity): task 1.2 - implement preamble deduplication (2026-09-03)
|
||||
- `6a2aa47f` test(scheduler-sim-step-verbosity): task 1.1 - RED tests for preamble dedup (2026-09-03)
|
||||
- `13745409` docs(scheduler-sim-step-verbosity): add approved spec (requirements/design/tasks) (2026-09-03)
|
||||
- `a51790dc` docs(scheduler-sim-repl-verbosity): fix two documentation gaps from final validation (2026-09-02)
|
||||
- `e87c0b3f` docs(scheduler-sim-repl-verbosity): task 10.2 - final regression and scale verification (2026-09-02)
|
||||
- `5ab129cd` docs(scheduler-sim-repl-verbosity): task 10.1 - validate no out-of-scope REPL regressions (2026-09-02)
|
||||
- `bd973973` feat(scheduler-sim-repl-verbosity): task 9.2 - discoverable help and break event types (2026-09-02)
|
||||
- `bd77eea0` test(scheduler-sim-repl-verbosity): task 9.1 - RED tests for help/break discoverability (2026-09-02)
|
||||
- `9db99964` feat(scheduler-sim-repl-verbosity): task 8.2 - implement resource table rendering (2026-09-02)
|
||||
- `e48e222b` feat(scheduler-sim-repl-verbosity): task 8.1 - RED-phase resource table tests (2026-09-02)
|
||||
- `5ee1d6cb` feat(scheduler-sim-repl-verbosity): task 7.2 - implement resource formatting helpers (2026-09-02)
|
||||
- `11fe6194` feat(scheduler-sim-repl-verbosity): task 7.1 - RED-phase resource formatting tests (2026-09-02)
|
||||
- `da6ba96e` feat(scheduler-sim-repl-verbosity): task 6.2 - implement dynamic REPL prompt (2026-09-02)
|
||||
- `99fa45c4` feat(scheduler-sim-repl-verbosity): task 6.1 - RED-phase REPL prompt tests (2026-09-02)
|
||||
- `d9109b7c` feat(scheduler-sim-repl-verbosity): task 4.2 - retire redundant per-decision echo (2026-09-02)
|
||||
- `3d08a43a` feat(scheduler-sim-repl-verbosity): task 4.1 - RED-phase sim_print_decision Gate tests (2026-09-02)
|
||||
- `cc78d091` feat(scheduler-sim-repl-verbosity): task 3.2 - implement Event Echo Gate (2026-09-02)
|
||||
- `f587a441` feat(scheduler-sim-repl-verbosity): task 3.1 - RED-phase Event Echo Gate tests (2026-09-02)
|
||||
- `921c72bb` feat(scheduler-sim-repl-verbosity): task 2.2 - implement Round Rollup logic (2026-09-02)
|
||||
- `0c6ccb03` feat(scheduler-sim-repl-verbosity): task 2.1 - RED-phase Round Rollup tests (2026-09-02)
|
||||
- `e6c67019` docs(scheduler-sim-repl-verbosity): expand spec with Requirements 5.5, 6, 7 (2026-09-02)
|
||||
- `b639fae1` feat(scheduler-sim-repl-verbosity): task 1.2 - implement job identifier range compression (2026-09-02)
|
||||
- `f165e3a7` feat(scheduler-sim-repl-verbosity): task 1.1 - RED-phase tests for job identifier range compression (2026-09-02)
|
||||
- `b6a2f5ec` fix(worker-rejoin-resync): task 4.4 - gate periodic load-indices send on resync_pending (2026-09-02)
|
||||
- `4140ecdb` docs(worker-rejoin-resync): add task 4.4 - close feature-level Req 3.8 gap (2026-09-02)
|
||||
- `a1dc2396` docs(worker-rejoin-resync): check off completed group headers in tasks.md (2026-09-02)
|
||||
- `1c4cf8bd` fix(worker-rejoin-resync): task 7.5 - bound the outbound reconciliation query (2026-09-02)
|
||||
- `613b80e9` test(worker-rejoin-resync): task 7.4 - large job-count pagination correctness (2026-09-02)
|
||||
- `165fc158` test(worker-rejoin-resync): task 7.3 - burst recovery unbounded by count (2026-09-02)
|
||||
- `bceca0c9` test(worker-rejoin-resync): task 7.2 - failover-during-job recovery (2026-09-01)
|
||||
- `8650cfd9` test(worker-rejoin-resync): task 7.1 - full rejoin round trip (2026-09-01)
|
||||
- `9f655c61` feat(worker-rejoin-resync): task 6.2 - route job-state report to reconciliation (2026-09-01)
|
||||
- `a3d12d37` feat(worker-rejoin-resync): task 6.1 - trigger reconciliation on rejoin (2026-09-01)
|
||||
- `80d5a062` feat(worker-rejoin-resync): task 5 - exclude reconciling host from dispatch (2026-09-01)
|
||||
- `5b699736` feat(worker-rejoin-resync): task 4.3 - wire query receipt and post-join gate (2026-09-01)
|
||||
- `cfe58c3d` feat(worker-rejoin-resync): task 4.2 - worker answers job-state query with extras (2026-09-01)
|
||||
- `e31012e8` feat(worker-rejoin-resync): task 4.1 - bounded finished-job retention on worker (2026-09-01)
|
||||
- `087b8c18` docs(worker-rejoin-resync): fold worker config-delivery gap into task 6.1 (2026-09-01)
|
||||
- `eec2596c` feat(worker-rejoin-resync): apply reported outcomes idempotently (task 3.2) (2026-09-01)
|
||||
- `442fcf45` feat(worker-rejoin-resync): reconciliation round lifecycle (task 3.1) (2026-09-01)
|
||||
- `ea0658ec` feat(worker-rejoin-resync): fail-fast disconnected-worker dispatch (task 2.3) (2026-09-01)
|
||||
- `ecfb4c94` feat(worker-rejoin-resync): reconcile unqueried job assignment (task 2.2) (2026-09-01)
|
||||
- `b7ed6de2` docs(worker-rejoin-resync): correct design.md's NULL-alloc encode claim (2026-09-01)
|
||||
- `cfe66dde` feat(worker-rejoin-resync): reconcile unrecognized job to pending (task 2.1) (2026-09-01)
|
||||
- `eda2eb1b` docs(worker-rejoin-resync): mark Foundation phase complete (task 1) (2026-09-01)
|
||||
- `ce3d391e` feat(worker-rejoin-resync): wire config fields into xceadmin build (task 1.5) (2026-09-01)
|
||||
- `9f3dfe79` feat(worker-rejoin-resync): add reconciliation wire codec helpers (task 1.4) (2026-09-01)
|
||||
- `9fbbd12b` docs(worker-rejoin-resync): add task 1.5, xceadmin config-build wiring (2026-09-01)
|
||||
- `25347648` docs(worker-rejoin-resync): close task 1.3, satisfied by pre-existing infra (2026-09-01)
|
||||
- `1260d913` feat(worker-rejoin-resync): add worker reconciling status type (task 1.2) (2026-09-01)
|
||||
- `1ff57ef6` feat(worker-rejoin-resync): define reconciliation wire protocol (task 1.1) (2026-09-01)
|
||||
- `6c908d50` docs(worker-rejoin-resync): fix design review findings; add tasks.md (2026-09-01)
|
||||
- `8a2e22f4` docs: mark scheduler-dispatch-commit tasks complete; add parked commit-batch-window spec (2026-09-01)
|
||||
- `f2ff45f1` docs(worker-rejoin-resync): add requirements, design, and research docs (2026-09-01)
|
||||
- `eb06f16d` fix(scheduler): wire mem reservation reconciliation into live rusage event path (2026-09-01)
|
||||
- `a6a0bdf7` docs(resource-accounting): explain why admission control doesn't query live (2026-08-31)
|
||||
- `f4103e0e` fix(scheduler): charge unreserved jobs' real memory usage too (2026-08-31)
|
||||
- `87a2a46b` feat(scheduler): reconcile mem reservation against real usage (2026-08-31)
|
||||
- `f59890ce` docs(scheduler-snapshot-job-allocation): sync spec.json phase to implementation-complete (2026-08-30)
|
||||
|
||||
### build
|
||||
- `0b1e3107` remove extralib (2026-09-04)
|
||||
- `6b105125` feat(eliminate-frontend-makefile): task 3.2 - fix deploy-ha.sh/run.sh messages (2026-09-03)
|
||||
- `3c199f37` feat(eliminate-frontend-makefile): task 2 - migrate xce-build.sh off Make (2026-09-03)
|
||||
- `4a224e8c` Simplify AppImage build to nix build .#xce.appImage (2026-09-03)
|
||||
- `68bc86bc` build: make Go and Python builds fully offline (hermetic Nix build) (2026-09-03)
|
||||
- `ce3d391e` feat(worker-rejoin-resync): wire config fields into xceadmin build (task 1.5) (2026-09-01)
|
||||
- `ff9de2e0` fix(bug-924): prune stale /tmp test-fixture dirs in xce-build.sh (2026-08-30)
|
||||
|
||||
### misc
|
||||
- `f5fbc6bc` feat(eliminate-frontend-makefile): task 3.3 - update live documentation (11 files) (2026-09-03)
|
||||
- `ff051e0e` feat(eliminate-frontend-makefile): task 3.1 - fix run-resource-verify.sh discovery (2026-09-03)
|
||||
- `4a224e8c` Simplify AppImage build to nix build .#xce.appImage (2026-09-03)
|
||||
- `651b5138` fix(xce-build skill): dispatch builds via Bash run_in_background, not Agent subagent (2026-09-03)
|
||||
- `68bc86bc` build: make Go and Python builds fully offline (hermetic Nix build) (2026-09-03)
|
||||
- `00fc0a43` test(xce-test-ha): add worker-rejoin-resync coverage to sequential/chaotic soak modes (2026-09-02)
|
||||
- `ea00879a` update README.md to the newest build guide (2026-09-02)
|
||||
- `87a2a46b` feat(scheduler): reconcile mem reservation against real usage (2026-08-31)
|
||||
- `63261097` update work status (2026-08-30)
|
||||
- `ccef5610` feat(skills): add xce-changes skill for component-grouped CHANGES.md (2026-08-30)
|
||||
|
||||
## Aug 24, 2026 - Aug 29, 2026
|
||||
<!-- window: 2026-08-24..2026-08-29 -->
|
||||
|
||||
|
|
|
|||
121
reports/aug30-sep05-2026.md
Normal file
121
reports/aug30-sep05-2026.md
Normal file
|
|
@ -0,0 +1,121 @@
|
|||
# XCE Project Status — Aug 30 – Sep 05, 2026
|
||||
|
||||
> Note: the calendar-default window (Aug 24–29) is already covered by
|
||||
> `aug23-aug29-2026.md`; this report covers the actual gap — the last
|
||||
> completed day before this report was generated.
|
||||
|
||||
## TL;DR
|
||||
|
||||
The single densest spec-throughput week on record: six full specs went
|
||||
requirements-to-implementation-complete inside seven days —
|
||||
`worker-rejoin-resync`, `scheduler-sim-repl-verbosity`,
|
||||
`eliminate-frontend-makefile`, `scheduler-sim-step-verbosity`,
|
||||
`scheduler-sim-reconcile-terminal-guard`, and — closing the week — the
|
||||
flagship `scheduler-allocation` feature (lazy iterator-driven allocation,
|
||||
parallel-job combinatorial planning, and the unsatisfiable-resreq cache),
|
||||
confirmed via full regression (`58e6a855`) and an HA soak (`ecfb0a3c`) on
|
||||
Saturday. Sunday and Monday cleared a dense backlog of live-deployment bugs
|
||||
(`bug-1429`, `bug-1560`, `bug-1727`, `bug-1734`–`bug-1739`) and closed both
|
||||
of last week's spec-metadata carry-over threads. The week's one unresolved
|
||||
thread, `bug-1714` (the scheduler UAF crash), had two more candidate causes
|
||||
ruled out but is still not root-caused after 11 sessions. Most consequential
|
||||
finding of the week: `scheduler-allocation`'s own closing commit documents
|
||||
that `parjob_allocator` — the entire multi-host allocation path just built
|
||||
and soak-validated — has **zero live production path** today, because
|
||||
`parse_resreq_primary()` hardcodes every job's task count to 1 (`bug-1770`);
|
||||
the feature is soak-clean but not yet "done" for real clusters.
|
||||
|
||||
## Commits by day
|
||||
|
||||
### Sun 08-30 — live-bug sweep + closing last week's metadata carry-overs
|
||||
|
||||
- **`0e96a69f`** — `bug-1429` (`controller_unit_test` 300s TIMEOUT, open since Jul 10, last week's oldest unresolved item): `persist_stop()` now runs before `commit_engine_fini()` in 21 test fixtures — test-fixture-only fix, no production code change.
|
||||
- **`dfcd345d`** — `bug-1560`: the symlink-escape test was asserting the wrong error string, not a real regression.
|
||||
- **`b0476a84`, `5c2c6664`, `d8a37fa2`** — `bug-1734` (merge redelivered job state instead of discarding it), `bug-1736` (capture `memory_swap`/`tasks_current`, compute `peak_cpu_time`), `bug-1735` (job memory/CPU usage always read 0 in scheduler replay) — a related trio of worker/controller resource-usage tracking gaps.
|
||||
- **`38437e57`** — scheduler no longer hands an orphaned duplicate job to the picker.
|
||||
- **`ff9de2e0`** — `bug-924`: prune stale `/tmp` test-fixture dirs in `xce-build.sh`.
|
||||
- **`a02d4777`** — `bug-1561`: fixed flaky `test_conf_trim_keeps_only_n_versions`/`test_conf_get_saved_tag_no_config`.
|
||||
- **`f59890ce`** — **closes last week's open thread #2 (half)**: `scheduler-snapshot-job-allocation`'s `spec.json` phase synced to `implementation-complete`, matching work already done.
|
||||
- **`ccef5610`** — new `xce-changes` skill for component-grouped `CHANGES.md` generation.
|
||||
- **`63261097`** — commits last week's `CHANGES.md` and `aug23-aug29-2026.md` report.
|
||||
|
||||
### Mon 08-31 — memory reservation-vs-usage reconciliation + a live 3-bug HA sweep
|
||||
|
||||
- **`87a2a46b`, `f4103e0e`, `a6a0bdf7`** — scheduler now reconciles memory reservation against real usage and charges unreserved jobs' real memory too, with a doc explaining why admission control deliberately doesn't query live usage.
|
||||
- **`fc559948`** — `bug-1727`: `info worker`/`info resource` was double-counting used capacity (`avail + used` instead of `avail` alone, since `avail` already included it).
|
||||
- **`0e78f863`, `026fe97b`** — a live 3-node HA cluster + 50-element array-job investigation surfaced and fixed `bug-1737` (scheduler could never learn a job finished successfully — EXIT vs. DONE confusion), `bug-1738` (a separate terminal-state regression), and `bug-1739` (phantom RUN jobs in `JOBS_PUSH` merge).
|
||||
|
||||
### Tue 09-01 — `worker-rejoin-resync`: Foundation through Integration in one day
|
||||
|
||||
- **`1ff57ef6`→`80d5a062`** (~20 commits) — wire protocol, worker reconciling status, reconciliation round lifecycle, idempotent outcome application, job-state query/answer, config-field wiring into xceadmin, and dispatch exclusion of a reconciling host — tasks 1.1 through 5 all landed the same day this large spec's requirements/design/tasks were also written (`f2ff45f1`, `6c908d50`).
|
||||
- **`8a2e22f4`** — **closes last week's open thread #2 (other half)**: `scheduler-dispatch-commit`'s stale 13/18 task count synced to the true 18/18; also adds the parked `commit-batch-window` spec (deliberately not being pursued — logged as a future cross-cutting WAL-commit-batching facility).
|
||||
- **`343639fc`, `728496be`, `eb06f16d`** — three more fixes: apiserver job-control panic on int/int64 mismatch (`bug-1742`, Gitea #45), hosts-list slot usage counts never populated (`bug-1743`, Gitea #46), and mem-reservation reconciliation wired into the live rusage event path (`bug-1740`).
|
||||
|
||||
### Wed 09-02 — `worker-rejoin-resync` closes out; `scheduler-sim-repl-verbosity` built end to end
|
||||
|
||||
- **`613b80e9`→`a1dc2396`** — `worker-rejoin-resync`'s remaining tasks (bounded pagination, burst/failover recovery, gating periodic reporting on `resync_pending`) land and all group headers get checked off — spec reaches `implementation-complete`.
|
||||
- **`f165e3a7`→`e6c67019`** (~20 commits) — the entire `scheduler-sim-repl-verbosity` feature (job-id range compression, round-boundary decision summaries, mode-aware event echo, dynamic REPL prompt, resource formatting/table rendering, discoverable help/break types) built and regression-verified in a single day, fixing `bug-1761` (`repl_cmd_delete()` rejecting invalid ids) along the way.
|
||||
- **`00fc0a43`** — `xce-test-ha` soak gains worker-rejoin-resync coverage in both sequential and chaotic modes.
|
||||
|
||||
### Thu 09-03 — busiest day: two more specs closed, a design pivot, and a hermetic build
|
||||
|
||||
- **`6c3ff201`→`375c49ec`** — `eliminate-frontend-makefile` built and closed end to end: Meson-based CGO test target, `xce-build.sh` migrated off Make, 11 doc files + 3 scripts + 2 C testsuite files + 2 Go source files updated, live-reference sweep clean, `src/frontend/Makefile` deleted.
|
||||
- **`6a2aa47f`→`9dcffce8`** — `scheduler-sim-step-verbosity` (preamble deduplication, fixing `bug-1762`/`bug-1763`) and `scheduler-sim-reconcile-terminal-guard` (terminal-state guard on job reconciliation, fixing `bug-1764`) both go from spec to `implementation-complete`.
|
||||
- **`0f7699de`, `177e9b29`** — `scheduler-primary-lifecycle` marked **superseded** by new spec `xce-node-manager` (a general per-node component-supervisor design, design-generated, tasks not yet drafted); `bug-1760`'s reactive-rejoin deadlock requirements carry forward into it.
|
||||
- **`68bc86bc`** — hermetic Nix build: vendors all Go modules and blocks PyPI/goproxy network access during build — the source of this week's outsized diff (see Stats).
|
||||
- **`df4b386a`, `8557b869`, `1abab949`, `b14e53f5`** — spec-metadata reconciliation sweep across `worker-rejoin-resync`, `xce-test-rc-file`, and 3 more specs.
|
||||
- **`4a224e8c`, `651b5138`** — AppImage build simplified to `nix build .#xce.appImage`; the `xce-build` skill now dispatches builds via `Bash run_in_background` instead of the `Agent` subagent tool.
|
||||
|
||||
### Fri 09-04 — `scheduler-allocation`: Foundation through cache-wiring in one day
|
||||
|
||||
- **`d5d2c1ca`→`5c022d0b`** (~14 commits) — the shared plan/host-iterator contracts, resreq fingerprinting, the unsatisfiable-resreq cache, seqjob's rewrap onto shared iterators, parjob's combinatorial plan generation and per-bucket allocation loop, and cache/dispatch wiring — tasks 1.1 through 5.1 of a 6-phase, 16-subtask spec, all in one day.
|
||||
- **`88b0ac99`** — `bug-1760` fixed: post-failover rejoin detection gap closed by adding passive, socket-free broken-connection detection (`xcecli_is_connection_broken()`) alongside the existing reactive path, live-verified against a real 3-node failover with zero jobs in flight.
|
||||
- **`1fc08d74`, `d912d33e`** — `bug-1714` forensics: ruled out the controller-side dual-push-path race and two more candidates; all known scheduler- and controller-side memory-ownership candidates are now exhausted with the stale second reader still unfound.
|
||||
- **`0b1e3107`** — removed an external "extralib" dependency, replaced with an in-tree `xcelist.c`/`xcelist.h` primitive, shrinking `flake.lock`.
|
||||
|
||||
### Sat 09-05 — `scheduler-allocation` closes the feature
|
||||
|
||||
- **`e8c2087d`** — cache cleared per scheduling session (task 5.2).
|
||||
- **`c009f108`** — end-to-end multi-host parallel job proof through the real dispatch path (task 5.3).
|
||||
- **`58e6a855`** — full existing scheduler regression pass, zero unexpected assertion changes (task 6.1).
|
||||
- **`ecfb0a3c`** — HA soak regression, zero new job loss/divergence/stall (task 6.2) — **feature complete**, with a standing caveat (see TL;DR and Open threads).
|
||||
|
||||
## Bugs opened/fixed this window
|
||||
|
||||
- **40 genuinely new bugs** (bug-1734 – bug-1774, one intentional ID gap at bug-1749), **22 fully fixed**, **3 partially fixed** (bug-1736, bug-1743, bug-1747 — each split into a fixed slice and an explicitly deferred remainder), **16 still open** at window end. One additional entry, `bug-1727`, was carried over from last week's open list and fixed this window (Mon, `fc559948`).
|
||||
- **Carry-over status from last week's report**: `bug-1429` — **fixed** (Sun, `0e96a69f`), closing the oldest open item. `bug-1714` — **still open**; two more candidates ruled out (controller dual-push race, redelivery-storm volume), all known static leads now exhausted. `bug-1630` housekeeping note — closed (fix field backfilled, re-verified zero collisions). `bug-1632`, `bug-1723`, `bug-1726`, `bug-1728` — no movement this window, still open.
|
||||
- Highest-severity fixes: **bug-1760** (HA availability — every job submitted after a controller failover could stay PEND forever until the scheduler was manually restarted), **bug-1737/1738/1739** (a live 3-node array-job run showing phantom RUN states and a scheduler that could never learn a job finished), **bug-1740** (memory-reservation reconciliation dead-wired to nothing).
|
||||
- ⚠️ **bug-1770**, discovered during `scheduler-allocation` task 5.3 (Sep 4): `parse_resreq_primary()` hardcodes every job's `minTasks`/`maxTasks` to 1, so `parjob_allocator` — the multi-host allocation path this window's flagship feature just built — has no live production path. Its `fix` field records a resolution dated **2026-09-06 (outside this window)**, by a separate feature; as of this window's close (Sep 5) it was still open. Not counted as fixed in this report's tallies. See Open threads.
|
||||
- **Integrity check**: of 41 unique bug IDs touched in-window, 1 (`bug-1727`) falls below the prior report's ID floor — a legitimate carry-over fix, not a duplicate-ID defect. The buglog counter otherwise stayed clean (40 new sequential-range IDs, one deliberate gap at bug-1749).
|
||||
|
||||
## Spec status changes
|
||||
|
||||
- **`scheduler-allocation`** — 0 → `implementation-complete` this window (all 16 subtasks, `spec.json` still reads `tasks-generated`/`ready_for_implementation: true` — a phase-sync pass is due). Standing caveat: soak-validated but not production-effective until `bug-1770` lands.
|
||||
- **`worker-rejoin-resync`** — `implementation-complete`, all tasks and group headers checked (Foundation through Validation, ~19 subtasks), spanning Tue–Wed.
|
||||
- **`scheduler-sim-repl-verbosity`** — `implementation-complete`, 10 major tasks / 23 subtasks, built and regression-verified in one day (Wed).
|
||||
- **`eliminate-frontend-makefile`** — tasks.md shows 14/14 complete and its own closing commit (`375c49ec`) declares the feature complete, but `spec.json` still reads phase `tasks-generated` / `ready_for_implementation: false` — a metadata/reality mismatch matching the pattern fixed for two other specs this same window.
|
||||
- **`scheduler-sim-step-verbosity`** and **`scheduler-sim-reconcile-terminal-guard`** — both `implementation-complete` (Thu), fixing bug-1762/1763 and bug-1764 respectively.
|
||||
- **`scheduler-dispatch-commit`** — task-count metadata lag from last week's report fixed (`8a2e22f4`): now correctly reads 18/18 matching its already-`implementation-complete` phase.
|
||||
- **`scheduler-snapshot-job-allocation`** — phase synced to `implementation-complete` (Sun), closing the other half of last week's metadata carry-over.
|
||||
- **`scheduler-primary-lifecycle`** — superseded by new spec **`xce-node-manager`** (design-generated; requirements/design approved, tasks not yet drafted) after a design pivot: node-level component supervision, not controller-internal scheduler spawning.
|
||||
- **`scheduler-sim-diagnostics`** — new draft spec added (requirements only, unapproved).
|
||||
- **`commit-batch-window`** — new spec created, explicitly parked/not pursued.
|
||||
|
||||
## Stats
|
||||
|
||||
- **160 commits** — **130 real** (feature/fix/docs/test), **30 `.wolf` bookkeeping**.
|
||||
- **3,565 non-`.wolf` files changed**, **+1,479,878 / −1,710** (non-wolf headline; raw incl. `.wolf`: +1,487,194 / −4,224). This figure is dominated by a single commit (`68bc86bc`, hermetic Nix build) that vendored ~3,177 third-party Go source files (~1.45M lines) — excluding that one-time vendor dump, the week's real hand-written churn is a small fraction of the headline number.
|
||||
- Busiest areas by files touched: `src/frontend` (3,188, ~3,177 of which are the vendor dump), `src/autoscaler` (151, also vendored), `src/scheduler` (78), `.kiro` (55), `src/controller` (35), `src/worker` (12), `src/lib` (10).
|
||||
- **6 specs shipped end-to-end** this window (`scheduler-allocation`, `worker-rejoin-resync`, `scheduler-sim-repl-verbosity`, `eliminate-frontend-makefile`, `scheduler-sim-step-verbosity`, `scheduler-sim-reconcile-terminal-guard`); **2 more** had metadata-only carry-over fixes (`scheduler-dispatch-commit`, `scheduler-snapshot-job-allocation`); **1 design pivot** (`scheduler-primary-lifecycle` → `xce-node-manager`); **3 new specs** opened.
|
||||
- **11 sessions** logged in `.wolf/memory.md` for Aug 30 – Sep 4 (no Sep 5 session flushed yet at report time); busiest logged days Sep 1 (396 actions, worker-rejoin-resync Foundation-to-Integration) and Sep 2 (331 actions, worker-rejoin-resync close-out + scheduler-sim-repl-verbosity).
|
||||
- Commits landed on all 7 days; peak day **Fri 09-04** for `scheduler-allocation`'s Foundation-through-cache-wiring push.
|
||||
|
||||
## Open threads / to-do heading into next session
|
||||
|
||||
1. **`bug-1714` still unresolved after 11 sessions.** All known scheduler-side memory-ownership candidates and the controller-side dual-push-path race are now ruled out; the stale second reader for `job_submit_data_t` remains unfound. Next step needs either a fresh live gdb-watchpoint campaign specifically instrumenting `core->jobs`'s own lifecycle (snapshot/recording code, REPL inspection commands, picker-plugin job-reference caching — none checked yet), or an explicit project-owner decision to accept this as a monitored low-frequency risk rather than continue speculative re-chasing.
|
||||
2. **`scheduler-allocation` is soak-validated but not production-effective.** `parjob_allocator`'s entire multi-host allocation path (tasks 4.1–4.3, 5.1–5.2) has zero live production path because `parse_resreq_primary()` (`src/lib/resreq.c:431-432`) hardcodes `minTasks=maxTasks=1` regardless of what a job actually requests, and the resreq grammar's own `span[ptile=N]` syntax is parsed then discarded. Logged as `bug-1770` (with sibling `bug-1771`, an unrelated fcfs NULL-queue-deref found while building the same test). This needs a dedicated, high-priority follow-up spec before the feature has any real-cluster effect.
|
||||
3. **`eliminate-frontend-makefile`'s `spec.json` metadata lag** — tasks.md and its own closing commit say the feature is done; phase still reads `tasks-generated` and `ready_for_implementation: false`. Same class of fix already applied to `scheduler-dispatch-commit` and `scheduler-snapshot-job-allocation` this window — needs the same treatment.
|
||||
4. **`scheduler-allocation`'s own `spec.json`** likewise needs its phase synced to `implementation-complete` now that all 16 subtasks and the HA soak are done.
|
||||
5. **Four lower-priority open bugs with zero movement this window**: `bug-1632`, `bug-1723`, `bug-1726`, `bug-1728` — still awaiting triage.
|
||||
6. **`xce-node-manager` needs `/kiro-spec-tasks`** — design is approved (GO, all findings fixed in place) but tasks aren't drafted yet; it's the successor to superseded `scheduler-primary-lifecycle` and carries forward `bug-1760`'s now-fixed rejoin-detection work as validated precedent.
|
||||
7. **A cluster of discovery-only bugs from this week's spec reviews remain open and unowned by any spec's current boundary**: `bug-1766`–`bug-1769`, `bug-1771`–`bug-1774` (compound/alter-resreq rollback and resource-accounting gaps, a heap-OOB type confusion, snapshot/schedobj duplication patterns) — worth a triage pass to decide which need dedicated follow-up specs.
|
||||
Loading…
Add table
Add a link
Reference in a new issue