update work status
This commit is contained in:
parent
ccd7fc2184
commit
6326109798
2 changed files with 485 additions and 0 deletions
374
CHANGES.md
Normal file
374
CHANGES.md
Normal file
|
|
@ -0,0 +1,374 @@
|
||||||
|
# XCE Change Log
|
||||||
|
|
||||||
|
Auto-generated by /xce-changes — component-grouped, newest week first.
|
||||||
|
`.wolf/` bookkeeping commits are excluded.
|
||||||
|
|
||||||
|
## Aug 24, 2026 - Aug 29, 2026
|
||||||
|
<!-- window: 2026-08-24..2026-08-29 -->
|
||||||
|
|
||||||
|
### controller
|
||||||
|
- `67009815` fix(bug-1731): fire HOOK_DETAIL_JOB_FAIL on WAL_JOB_FAIL apply (2026-08-29)
|
||||||
|
- `ebe691c3` fix(bug-1731): fire HOOK_DETAIL_JOB_START/JOB_EXIT on WAL apply (2026-08-29)
|
||||||
|
- `47532f0a` fix(ha): first-boot promotion evaluated eligibility before its own WAL recovery ran (bug-1696) (2026-08-27)
|
||||||
|
- `f886c809` fix(controller): standby dispatch-gate rejections now redirect to a known primary (bug-1700, bug-1701) (2026-08-27)
|
||||||
|
- `ac1e312e` fix(controller): job.start_time never set -- dispatch/WAL_JOB_START RUN-state collision (bug-1697) (2026-08-27)
|
||||||
|
- `7880ab42` chore(xce-test-rc-file): pass --norc from existing xce_test automation (2026-08-27)
|
||||||
|
- `19bc8192` test(xce-test-rc-file): task 3.2 end-to-end rc-file failure handling (2026-08-26)
|
||||||
|
- `d258aa3d` test(xce-test-rc-file): task 3.1 end-to-end rc-file discovery/loading/--norc (2026-08-26)
|
||||||
|
- `e863d699` feat(xce-test-rc-file): task 2.2 wire rc-file loading into xce_test startup (2026-08-26)
|
||||||
|
- `68fcd657` feat(xce-test-rc-file): task 2.1 add --norc flag to xce_test (2026-08-26)
|
||||||
|
- `77aba21f` feat(xce-test-rc-file): task 1.2 rc-file line-loading via pluggable runner (2026-08-26)
|
||||||
|
- `9de4db95` feat(xce-test-rc-file): task 1.1 rc-file path resolution with XDG precedence (2026-08-26)
|
||||||
|
- `ac46bfa6` fix(controller): restored-but-stale raft term permanently blocked re-election after restart (bug-1695) (2026-08-25)
|
||||||
|
- `b4eb9ec8` fix(controller): offload checkpoint_task_fn off the delaymgr thread (bug-1694) (2026-08-25)
|
||||||
|
- `3eba7c16` fix(controller): fail loud on scheduler/worker disconnect instead of relying on the reply-wait timeout (bug-1693) (2026-08-25)
|
||||||
|
- `d0e5f85a` fix(controller): WAL_JOB_PREASON loses job.pend_reasons on replay (bug-1692) (2026-08-25)
|
||||||
|
- `97760761` fix(controller): WAL_JOB_CKPT loses job.ckptinfo on replay (bug-1691) (2026-08-25)
|
||||||
|
- `f7b732a3` fix(controller): WAL_JOB_RUSAGE loses job.run_usage on replay (bug-1690) (2026-08-25)
|
||||||
|
- `c2690cbf` fix(controller): WAL_JOB_FINISH loses exit_info/finish_time on replay -- and exit_info was never set at all in production (bug-1689) (2026-08-25)
|
||||||
|
- `955efecd` fix(controller): WAL_JOB_START loses pidinfo/start_time on restart replay (bug-1687) (2026-08-25)
|
||||||
|
- `e5e6cd18` fix(controller): raft_cb_persist must deny vote and roll back state on persist failure (bug-1684) (2026-08-25)
|
||||||
|
- `8d6f610e` test(controller): cover live dispatch's apiserver notification (bug-1679) (2026-08-25)
|
||||||
|
- `da7813bf` fix(controller): notify apiserver synchronously on live dispatch (bug-1679) (2026-08-25)
|
||||||
|
- `f8851353` test(controller): cover WAL_JOB_DISPATCH allocation persistence (bug-1678) (2026-08-25)
|
||||||
|
- `db142d41` fix(controller): persist dispatch-time resource allocation into WAL_JOB_DISPATCH (bug-1678, gitea #43) (2026-08-25)
|
||||||
|
|
||||||
|
### worker
|
||||||
|
- `347e996e` feat(worker): distinguish signal-killed job exits from real crashes in logs (2026-08-29)
|
||||||
|
|
||||||
|
### scheduler
|
||||||
|
- `14f95b06` fix(scheduler): re-derive worker/shared resource used-totals on snapshot load (2026-08-29)
|
||||||
|
- `ebe691c3` fix(bug-1731): fire HOOK_DETAIL_JOB_START/JOB_EXIT on WAL apply (2026-08-29)
|
||||||
|
- `b1323f8d` fix(bug-1730): free job accounts/allocs on JOB_UPDATE terminal transition (2026-08-29)
|
||||||
|
- `5948e079` docs(scheduler-snapshot-job-allocation): close final invariant-audit follow-up, feature complete (2026-08-28)
|
||||||
|
- `93d791c6` docs(scheduler-snapshot-job-allocation): correct snapshot.c scope-decision documentation (2026-08-28)
|
||||||
|
- `2654e03f` test(scheduler-snapshot-job-allocation): close originally-reported symptom end to end (2026-08-28)
|
||||||
|
- `de6ad514` test(scheduler-snapshot-job-allocation): regression test for allocator crash (2026-08-27)
|
||||||
|
- `f0fbebde` test(scheduler-snapshot-job-allocation): backward-compat for allocation-less snapshots (2026-08-27)
|
||||||
|
- `7481cedb` test(scheduler-snapshot-job-allocation): full cross-instance round-trip proof (2026-08-27)
|
||||||
|
- `182cb555` feat(scheduler-snapshot-job-allocation): restore allocs/accounts on load (2026-08-27)
|
||||||
|
- `6fa2d970` feat(scheduler-snapshot-job-allocation): populate allocs on snapshot save (2026-08-27)
|
||||||
|
- `22353849` feat(scheduler-snapshot-job-allocation): construct schedobj for restored jobs (2026-08-27)
|
||||||
|
- `0ebf3a81` fix(scheduler): repeat WORKERS_PUSH with a NULL resval clobbers good values (bug-1688) (2026-08-27)
|
||||||
|
- `ab077916` test(scheduler-sim-eager-reconciliation): task 3.2 sim-verify regression (2026-08-27)
|
||||||
|
- `c971f7dc` test(scheduler-sim-eager-reconciliation): task 3.1 resource-contention proof (2026-08-27)
|
||||||
|
- `c67382e6` test(scheduler-sim-eager-reconciliation): task 1.2 capture-safety unit coverage (2026-08-27)
|
||||||
|
- `0893fd50` test(scheduler-sim-eager-reconciliation): task 2.1 resInUse deferral proof (2026-08-27)
|
||||||
|
- `e2bb98c8` feat(scheduler-sim-eager-reconciliation): task 1.1 state-independent dispatch capture (2026-08-27)
|
||||||
|
- `bc3f76b4` fix(scheduler): invalidate stale builtin_fcfs picker cursors across calls (bug-1686) (2026-08-25)
|
||||||
|
- `b040dfc0` fix(scheduler): remove double-free in limit plugin config update (bug-1685) (2026-08-25)
|
||||||
|
- `6400f96d` fix(scheduler): mirror plugin_begin_session/plugin_end_session into sim-mode driver (bug-1681) (2026-08-25)
|
||||||
|
|
||||||
|
### frontend
|
||||||
|
- `ebe691c3` fix(bug-1731): fire HOOK_DETAIL_JOB_START/JOB_EXIT on WAL apply (2026-08-29)
|
||||||
|
- `9a230f08` fix(apiserver): FilterRows must not route non-bitmask operators through bitmask matching (bug-1680) (2026-08-25)
|
||||||
|
- `1a61b04a` fix(apiserver): thread from_host through the CGO conversion layer (bug-1676, gitea #44) (2026-08-24)
|
||||||
|
- `fe8818b1` fix(apiserver): pass client-asserted FromHost through unchanged (bug-1676, gitea #44) (2026-08-24)
|
||||||
|
- `4fad7884` fix(xcecli): assert own hostname as FROM_HOST at submission time (bug-1676, gitea #44) (2026-08-24)
|
||||||
|
- `fe87dafc` fix(apiserver): fall back to default HostConfig for unmatched workers (bug-1673, gitea #40) (2026-08-24)
|
||||||
|
- `88c7c05d` fix(apiserver): jobs list -l/-o validates against the wrong struct (bug-1674, gitea #41) (2026-08-24)
|
||||||
|
- `a58c2a19` fix(xcecli): friendly message for connection-refused instead of raw OS error (bug-1675, gitea #42) (2026-08-24)
|
||||||
|
|
||||||
|
### xceadmin
|
||||||
|
- `6193fa7e` test(admin-config-tool): update standby-redirect E2E for bug-1700/1701 (2026-08-29)
|
||||||
|
- `cfd7db8c` fix(xceadmin): retry transient scheduler-admin failures after a redirect (bug-1713, bug-1715) (2026-08-27)
|
||||||
|
- `3891bd90` test(xceadmin): add env-var override for live E2E binary discovery (2026-08-27)
|
||||||
|
|
||||||
|
### lib
|
||||||
|
- `42e1eeaa` fix(bug-1733): fix heap-use-after-free race in vtask_enq() (2026-08-29)
|
||||||
|
- `097e33ae` feat(scheduler-snapshot-job-allocation): add dup_job_alloc deep-copy primitive (2026-08-27)
|
||||||
|
- `00207503` fix(lib): heap buffer overflow in make_job_account() for multi-host jobs (bug-1722) (2026-08-27)
|
||||||
|
- `363709b0` fix(lib): thread from_host through the wire format and the job-submit interning object (bug-1676, gitea #44) (2026-08-24)
|
||||||
|
|
||||||
|
### include
|
||||||
|
- `097e33ae` feat(scheduler-snapshot-job-allocation): add dup_job_alloc deep-copy primitive (2026-08-27)
|
||||||
|
- `ac1e312e` fix(controller): job.start_time never set -- dispatch/WAL_JOB_START RUN-state collision (bug-1697) (2026-08-27)
|
||||||
|
- `b4eb9ec8` fix(controller): offload checkpoint_task_fn off the delaymgr thread (bug-1694) (2026-08-25)
|
||||||
|
- `d0e5f85a` fix(controller): WAL_JOB_PREASON loses job.pend_reasons on replay (bug-1692) (2026-08-25)
|
||||||
|
- `97760761` fix(controller): WAL_JOB_CKPT loses job.ckptinfo on replay (bug-1691) (2026-08-25)
|
||||||
|
- `f7b732a3` fix(controller): WAL_JOB_RUSAGE loses job.run_usage on replay (bug-1690) (2026-08-25)
|
||||||
|
- `c2690cbf` fix(controller): WAL_JOB_FINISH loses exit_info/finish_time on replay -- and exit_info was never set at all in production (bug-1689) (2026-08-25)
|
||||||
|
- `955efecd` fix(controller): WAL_JOB_START loses pidinfo/start_time on restart replay (bug-1687) (2026-08-25)
|
||||||
|
- `da7813bf` fix(controller): notify apiserver synchronously on live dispatch (bug-1679) (2026-08-25)
|
||||||
|
- `db142d41` fix(controller): persist dispatch-time resource allocation into WAL_JOB_DISPATCH (bug-1678, gitea #43) (2026-08-25)
|
||||||
|
- `363709b0` fix(lib): thread from_host through the wire format and the job-submit interning object (bug-1676, gitea #44) (2026-08-24)
|
||||||
|
|
||||||
|
### schema
|
||||||
|
- `f7b732a3` fix(controller): WAL_JOB_RUSAGE loses job.run_usage on replay (bug-1690) (2026-08-25)
|
||||||
|
- `363709b0` fix(lib): thread from_host through the wire format and the job-submit interning object (bug-1676, gitea #44) (2026-08-24)
|
||||||
|
|
||||||
|
### specs
|
||||||
|
- `6193fa7e` test(admin-config-tool): update standby-redirect E2E for bug-1700/1701 (2026-08-29)
|
||||||
|
- `42fdeb66` docs: save scheduler-config-version-tag spec (requirements/design/tasks) (2026-08-29)
|
||||||
|
- `b64682dc` docs: save scheduler-snapshot-job-allocation and xce-test-cli-ux spec docs, sync OpenWolf state (2026-08-28)
|
||||||
|
- `5948e079` docs(scheduler-snapshot-job-allocation): close final invariant-audit follow-up, feature complete (2026-08-28)
|
||||||
|
- `93d791c6` docs(scheduler-snapshot-job-allocation): correct snapshot.c scope-decision documentation (2026-08-28)
|
||||||
|
- `9988e714` test(scheduler-snapshot-job-allocation): confirm no regression to reconciliation or snapshot format (2026-08-28)
|
||||||
|
- `2654e03f` test(scheduler-snapshot-job-allocation): close originally-reported symptom end to end (2026-08-28)
|
||||||
|
- `de6ad514` test(scheduler-snapshot-job-allocation): regression test for allocator crash (2026-08-27)
|
||||||
|
- `f0fbebde` test(scheduler-snapshot-job-allocation): backward-compat for allocation-less snapshots (2026-08-27)
|
||||||
|
- `7481cedb` test(scheduler-snapshot-job-allocation): full cross-instance round-trip proof (2026-08-27)
|
||||||
|
- `182cb555` feat(scheduler-snapshot-job-allocation): restore allocs/accounts on load (2026-08-27)
|
||||||
|
- `6fa2d970` feat(scheduler-snapshot-job-allocation): populate allocs on snapshot save (2026-08-27)
|
||||||
|
- `22353849` feat(scheduler-snapshot-job-allocation): construct schedobj for restored jobs (2026-08-27)
|
||||||
|
- `097e33ae` feat(scheduler-snapshot-job-allocation): add dup_job_alloc deep-copy primitive (2026-08-27)
|
||||||
|
- `d32f06e5` docs: formalize save/restore field-completeness as a standing invariant (2026-08-27)
|
||||||
|
- `5e68fcc0` docs(spec): promote eliminate-frontend-makefile backlog item to a brief (2026-08-27)
|
||||||
|
- `13246a89` docs: OpenWolf pointer cleanup + redirect-hint checklist item (bug-1700 lesson) (2026-08-27)
|
||||||
|
- `446437bf` test(scheduler-sim-eager-reconciliation): task 3.3 full regression confirmation (2026-08-27)
|
||||||
|
- `ab077916` test(scheduler-sim-eager-reconciliation): task 3.2 sim-verify regression (2026-08-27)
|
||||||
|
- `c971f7dc` test(scheduler-sim-eager-reconciliation): task 3.1 resource-contention proof (2026-08-27)
|
||||||
|
- `c67382e6` test(scheduler-sim-eager-reconciliation): task 1.2 capture-safety unit coverage (2026-08-27)
|
||||||
|
- `0893fd50` test(scheduler-sim-eager-reconciliation): task 2.1 resInUse deferral proof (2026-08-27)
|
||||||
|
- `e2bb98c8` feat(scheduler-sim-eager-reconciliation): task 1.1 state-independent dispatch capture (2026-08-27)
|
||||||
|
- `8fc86ef4` test(xce-test-rc-file): task 3.3 confirm existing automation unaffected (2026-08-26)
|
||||||
|
- `19bc8192` test(xce-test-rc-file): task 3.2 end-to-end rc-file failure handling (2026-08-26)
|
||||||
|
- `d258aa3d` test(xce-test-rc-file): task 3.1 end-to-end rc-file discovery/loading/--norc (2026-08-26)
|
||||||
|
- `e863d699` feat(xce-test-rc-file): task 2.2 wire rc-file loading into xce_test startup (2026-08-26)
|
||||||
|
- `68fcd657` feat(xce-test-rc-file): task 2.1 add --norc flag to xce_test (2026-08-26)
|
||||||
|
- `77aba21f` feat(xce-test-rc-file): task 1.2 rc-file line-loading via pluggable runner (2026-08-26)
|
||||||
|
- `9de4db95` feat(xce-test-rc-file): task 1.1 rc-file path resolution with XDG precedence (2026-08-26)
|
||||||
|
- `71a814f3` docs(steering): add field-complete apply-parity invariant; extend xce-audit-invariants (2026-08-25)
|
||||||
|
- `5c82ad0b` chore(specs): reconcile stale phase/checkbox metadata across 4 specs (2026-08-24)
|
||||||
|
- `0c736c79` chore(specs): add scheduler-sim-eager-reconciliation spec, weekly report, cross-spec sequencing notes (2026-08-24)
|
||||||
|
|
||||||
|
### build
|
||||||
|
- `83cb660b` fix(bug-1732): build xcecd_virtual in xce-build.sh's worker target (2026-08-29)
|
||||||
|
- `faedc6fc` fix(deploy-ha): guard against double-launching a live HA deploy (2026-08-27)
|
||||||
|
|
||||||
|
### misc
|
||||||
|
- `f0a3729f` ignore daemon.log token-ledger (2026-08-28)
|
||||||
|
- `1b473736` docs: general design-time cross-check against project invariant dimensions (2026-08-27)
|
||||||
|
- `d32f06e5` docs: formalize save/restore field-completeness as a standing invariant (2026-08-27)
|
||||||
|
- `13246a89` docs: OpenWolf pointer cleanup + redirect-hint checklist item (bug-1700 lesson) (2026-08-27)
|
||||||
|
- `7880ab42` chore(xce-test-rc-file): pass --norc from existing xce_test automation (2026-08-27)
|
||||||
|
- `5441d1b8` docs(steering): add Decision Log fidelity as controller audit dimension 8 (2026-08-25)
|
||||||
|
- `71a814f3` docs(steering): add field-complete apply-parity invariant; extend xce-audit-invariants (2026-08-25)
|
||||||
|
- `643741fa` fix(gitea-sync): unwrap buglog.json's {version,bugs} wrapper before iterating (bug-1677) (2026-08-24)
|
||||||
|
- `8fd232c1` chore: gitignore generated capnp-c codegen output (2026-08-24)
|
||||||
|
- `f2b5f9c9` chore: untrack .wolf/dashboard-token, gitignore it going forward (2026-08-24)
|
||||||
|
- `0c736c79` chore(specs): add scheduler-sim-eager-reconciliation spec, weekly report, cross-spec sequencing notes (2026-08-24)
|
||||||
|
|
||||||
|
## Aug 17, 2026 - Aug 23, 2026
|
||||||
|
<!-- window: 2026-08-17..2026-08-23 -->
|
||||||
|
|
||||||
|
### controller
|
||||||
|
- `a9cfee5f` test(scheduler-dispatch-commit): fix last stale pinned assertion, full regression clean (task 5.4, final) (2026-08-23)
|
||||||
|
- `bea95699` test(scheduler-dispatch-commit): terminal dispatch failure e2e coverage (task 5.3) (2026-08-23)
|
||||||
|
- `16d90319` test(scheduler-dispatch-commit): retryable and bounded-retry dispatch failure e2e coverage (task 5.2) (2026-08-23)
|
||||||
|
- `7b3dfed5` feat(scheduler-dispatch-commit): handle_jobdispatch no longer duplicates dispatch-outcome reporting (task 4.1) (2026-08-23)
|
||||||
|
- `05c45ef3` feat(scheduler-dispatch-commit): controller resolves retryable dispatch failures with bounded escalation (task 3.3) (2026-08-23)
|
||||||
|
- `8a4266ad` feat(scheduler-dispatch-commit): controller resolves terminal dispatch failures to EXIT (task 3.2) (2026-08-23)
|
||||||
|
- `d9aedc4b` feat(scheduler-dispatch-commit): controller commits alloc+RUN transition together on successful dispatch (task 3.1) (2026-08-23)
|
||||||
|
- `19bbac6a` feat(scheduler-dispatch-commit): add dispatch-fail retry counter to job_t (task 1.3) (2026-08-23)
|
||||||
|
- `8eea7490` feat(scheduler-dispatch-commit): add durable WAL_JOB_DISPATCH fact type (task 1.2) (2026-08-23)
|
||||||
|
- `5724d1e9` feat(scheduler-dispatch-commit): add controller dispatch-failure classification (task 1.1) (2026-08-23)
|
||||||
|
- `1c906e95` fix(worker): job ids print human-readable format via fmt_jobid() (bug-1588) (2026-08-22)
|
||||||
|
- `fd62e26b` feat(dirty-content-fingerprint): controller/scheduler show fingerprint in --version (2026-08-21)
|
||||||
|
- `88649a04` feat(build-time-versioning): controller/scheduler --version and startup log (2026-08-20)
|
||||||
|
- `5147396d` fix(controller): job start_time/finish_time/run_time never set by apply-pump (2026-08-20)
|
||||||
|
- `e52c68f4` fix(controller): admission-gate OPCode_schedAdmin's blocking wait (Req 5 AC 10) (2026-08-19)
|
||||||
|
- `515f4ca0` fix(controller): job.c TRACE lines print human-readable jobid (bug-1588) (2026-08-19)
|
||||||
|
- `b406dd76` fix(controller): sched_worker_change() use-after-free on worker disconnect (bug-1578) (2026-08-18)
|
||||||
|
- `f5390c14` feat(resource-value-verification): per-name accept/reject verification in resupdate (task 2.2) (2026-08-18)
|
||||||
|
- `058335d2` feat(resource-value-verification): wire resolver into resupdate, add -t flag (task 2.1) (2026-08-17)
|
||||||
|
- `98b8c34b` feat(resource-value-verification): extract resupdate type-resolution logic (task 1.1) (2026-08-17)
|
||||||
|
- `aabddf60` fix(scheduler,controller): worker resource values always display as 0.00 (bug-1568) (2026-08-17)
|
||||||
|
- `986a3304` fix(controller): persist_flush now fires durable-advance callback (bug-1564) (2026-08-17)
|
||||||
|
- `846713ab` fix(ha): close standby election-freeze (bug-1562) + flusher retry storm (bug-1563) (2026-08-17)
|
||||||
|
|
||||||
|
### worker
|
||||||
|
- `1c906e95` fix(worker): job ids print human-readable format via fmt_jobid() (bug-1588) (2026-08-22)
|
||||||
|
- `d3d9522f` feat(dirty-content-fingerprint): worker shows fingerprint in --version (2026-08-21)
|
||||||
|
- `bc5c669b` feat(build-time-versioning): worker --version and startup log (2026-08-20)
|
||||||
|
- `70fe9e46` fix(worker): dbus_workers_update() starves workers after first DONE/EXITED (bug-1589) (2026-08-19)
|
||||||
|
- `6e5c3a89` fix(worker): bug-1569 periodic resval reports tagged with wrong host (2026-08-17)
|
||||||
|
|
||||||
|
### scheduler
|
||||||
|
- `01c83066` test(scheduler-dispatch-commit): bug-1624 multi-round regression coverage (task 5.1) (2026-08-23)
|
||||||
|
- `281a026d` feat(scheduler-dispatch-commit): scheduler processes dispatch-fail notify, returns job to PEND (task 2.2) (2026-08-23)
|
||||||
|
- `8607348c` feat(scheduler-dispatch-commit): scheduler transitions job to RUN at dispatch commit (task 2.1) (2026-08-23)
|
||||||
|
- `3c4091d8` test(scheduler-sim-decision-parity): live-verify against a real 24-round recording (task 4.3, final) (2026-08-22)
|
||||||
|
- `88bed529` test(scheduler-sim-decision-parity): regression suite for existing guarantees (task 4.2) (2026-08-22)
|
||||||
|
- `b3190732` test(scheduler-sim-decision-parity): prove capacity-aware parity under contention (task 4.1) (2026-08-22)
|
||||||
|
- `12644094` feat(scheduler-sim-decision-parity): wire reconciliation into the round boundary (task 3) (2026-08-22)
|
||||||
|
- `0c237f60` feat(scheduler-sim-decision-parity): report reconciled decisions through the existing channel (task 2.4) (2026-08-22)
|
||||||
|
- `eccba841` feat(scheduler-sim-decision-parity): reconstruct job allocation with safe ownership (task 2.3) (2026-08-22)
|
||||||
|
- `81358b52` feat(scheduler-sim-decision-parity): add reconciliation trigger and lookup (task 2.2) (2026-08-22)
|
||||||
|
- `134380b8` fix(scheduler-sim-decision-parity): give sim mode driver.c's round-boundary split (task 2.1) (2026-08-22)
|
||||||
|
- `f11f0df3` test(scheduler-sim-decision-parity): add resource-contention replay fixture (task 1.2) (2026-08-22)
|
||||||
|
- `3106c82e` feat(scheduler-sim-decision-parity): relocate resource accounting to core-level (task 1.1) (2026-08-22)
|
||||||
|
- `25e35bac` docs(scheduler): document driver-equivalence invariant, scope decision-parity fix spec (2026-08-22)
|
||||||
|
- `fd62e26b` feat(dirty-content-fingerprint): controller/scheduler show fingerprint in --version (2026-08-21)
|
||||||
|
- `88649a04` feat(build-time-versioning): controller/scheduler --version and startup log (2026-08-20)
|
||||||
|
- `8abab049` feat(scheduler): show event time/session in sim echo, total sessions on replay load (2026-08-20)
|
||||||
|
- `94733616` fix(scheduler): gate resource value/used display on XCE_RESF_CONSUMABLE (2026-08-20)
|
||||||
|
- `185feca2` fix(scheduler): "info resource all" printed nothing when no global/shared resources exist (2026-08-20)
|
||||||
|
- `8cb304d2` fix(scheduler): sim_print_decision() bypassed tlog, producing inconsistent PEND/DISPATCH/PREEMPT output (2026-08-20)
|
||||||
|
- `1bc65d2c` docs(resource-value-verification): close design-review doc/comment gaps (2026-08-19)
|
||||||
|
- `a13068dc` fix(scheduler): extend core-isolation nm-audit to core_events.c/core_session.c/core_clock.c (Req 5 AC 1/2) (2026-08-19)
|
||||||
|
- `b450a680` fix(scheduler): activate virtual-clock logging for sim/REPL sessions (Req 5 AC 3) (2026-08-19)
|
||||||
|
- `c9acd502` fix(scheduler): clock/resval/jobid-format fixes plus REPL live-operator gaps (task 12.5) (2026-08-19)
|
||||||
|
- `e68444f1` feat(resource-value-verification): add scheduler resval trace log (task 4.1) (2026-08-18)
|
||||||
|
- `7041d988` fix(scheduler): bug-1569 three stacked crashes blocking ALL job dispatch (2026-08-17)
|
||||||
|
- `6e5c3a89` fix(worker): bug-1569 periodic resval reports tagged with wrong host (2026-08-17)
|
||||||
|
- `aabddf60` fix(scheduler,controller): worker resource values always display as 0.00 (bug-1568) (2026-08-17)
|
||||||
|
- `0c8e417b` fix(scheduler): sim mode picker never learns about the initial snapshot's own jobs/queues (bug-1567) (2026-08-17)
|
||||||
|
|
||||||
|
### frontend
|
||||||
|
- `d65d18ad` feat(dirty-content-fingerprint): xcecli shows fingerprint in --version (2026-08-21)
|
||||||
|
- `a4c1b5ad` feat(dirty-content-fingerprint): apiserver shows fingerprint in -version (2026-08-21)
|
||||||
|
- `aa3391a9` feat(dirty-content-fingerprint): Makefile computes dev-loop fingerprint (2026-08-21)
|
||||||
|
- `502dd023` feat(dirty-content-fingerprint): apiserver/xcecli meson pass fingerprint to go_build (2026-08-21)
|
||||||
|
- `fb09afe6` feat(build-time-versioning): xcecli --version via Cobra rootCmd.Version (2026-08-20)
|
||||||
|
- `17e837b1` feat(build-time-versioning): apiserver -version flag and startup log (2026-08-20)
|
||||||
|
- `45c9b026` feat(build-time-versioning): Makefile ldflags for make build-server/build-cli (2026-08-20)
|
||||||
|
- `071f3895` feat(build-time-versioning): thread commit_id/build_time into go_build env (2026-08-20)
|
||||||
|
- `1bc65d2c` docs(resource-value-verification): close design-review doc/comment gaps (2026-08-19)
|
||||||
|
- `8b4fbffa` feat(resource-value-verification): add apiserver resval trace log (task 4.2) (2026-08-18)
|
||||||
|
|
||||||
|
### xceadmin
|
||||||
|
- `c79ccf0f` feat(dirty-content-fingerprint): xceadmin shows fingerprint in --version (2026-08-21)
|
||||||
|
- `2bf62a64` feat(dirty-content-fingerprint): xceadmin build-info module gains fingerprint (2026-08-21)
|
||||||
|
- `7f5c87ce` feat(build-time-versioning): xceadmin committed fallback stub and CLI version wiring (2026-08-20)
|
||||||
|
- `e1cd9d25` feat(build-time-versioning): xceadmin generates build-info module (2026-08-20)
|
||||||
|
- `0f1df499` feat(deploy,xceadmin): standalone xceadmin packaging + deploy-ha usability (bug-1566) (2026-08-17)
|
||||||
|
|
||||||
|
### lib
|
||||||
|
- `7ebcba6d` feat(dirty-content-fingerprint): add --version-only composition with dirty_fingerprint (2026-08-21)
|
||||||
|
- `48a86cda` feat(build-time-versioning): version macro/helper functions with unknown fallback (2026-08-20)
|
||||||
|
- `ddee884c` fix(libxce): bug-1600 -- makeGroup() freed rtti makeString() pointer via raw free() (2026-08-20)
|
||||||
|
|
||||||
|
### include
|
||||||
|
- `19bbac6a` feat(scheduler-dispatch-commit): add dispatch-fail retry counter to job_t (task 1.3) (2026-08-23)
|
||||||
|
- `8eea7490` feat(scheduler-dispatch-commit): add durable WAL_JOB_DISPATCH fact type (task 1.2) (2026-08-23)
|
||||||
|
- `7ebcba6d` feat(dirty-content-fingerprint): add --version-only composition with dirty_fingerprint (2026-08-21)
|
||||||
|
- `48a86cda` feat(build-time-versioning): version macro/helper functions with unknown fallback (2026-08-20)
|
||||||
|
- `846713ab` fix(ha): close standby election-freeze (bug-1562) + flusher retry storm (bug-1563) (2026-08-17)
|
||||||
|
|
||||||
|
### specs
|
||||||
|
- `f784e2bd` chore(scheduler-dispatch-commit): mark spec implementation-complete (2026-08-23)
|
||||||
|
- `a9cfee5f` test(scheduler-dispatch-commit): fix last stale pinned assertion, full regression clean (task 5.4, final) (2026-08-23)
|
||||||
|
- `bea95699` test(scheduler-dispatch-commit): terminal dispatch failure e2e coverage (task 5.3) (2026-08-23)
|
||||||
|
- `16d90319` test(scheduler-dispatch-commit): retryable and bounded-retry dispatch failure e2e coverage (task 5.2) (2026-08-23)
|
||||||
|
- `01c83066` test(scheduler-dispatch-commit): bug-1624 multi-round regression coverage (task 5.1) (2026-08-23)
|
||||||
|
- `7b3dfed5` feat(scheduler-dispatch-commit): handle_jobdispatch no longer duplicates dispatch-outcome reporting (task 4.1) (2026-08-23)
|
||||||
|
- `05c45ef3` feat(scheduler-dispatch-commit): controller resolves retryable dispatch failures with bounded escalation (task 3.3) (2026-08-23)
|
||||||
|
- `8a4266ad` feat(scheduler-dispatch-commit): controller resolves terminal dispatch failures to EXIT (task 3.2) (2026-08-23)
|
||||||
|
- `d9aedc4b` feat(scheduler-dispatch-commit): controller commits alloc+RUN transition together on successful dispatch (task 3.1) (2026-08-23)
|
||||||
|
- `281a026d` feat(scheduler-dispatch-commit): scheduler processes dispatch-fail notify, returns job to PEND (task 2.2) (2026-08-23)
|
||||||
|
- `8607348c` feat(scheduler-dispatch-commit): scheduler transitions job to RUN at dispatch commit (task 2.1) (2026-08-23)
|
||||||
|
- `19bbac6a` feat(scheduler-dispatch-commit): add dispatch-fail retry counter to job_t (task 1.3) (2026-08-23)
|
||||||
|
- `8eea7490` feat(scheduler-dispatch-commit): add durable WAL_JOB_DISPATCH fact type (task 1.2) (2026-08-23)
|
||||||
|
- `5724d1e9` feat(scheduler-dispatch-commit): add controller dispatch-failure classification (task 1.1) (2026-08-23)
|
||||||
|
- `ca01509f` chore(scheduler-sim-decision-parity): mark feature complete (2026-08-22)
|
||||||
|
- `3c4091d8` test(scheduler-sim-decision-parity): live-verify against a real 24-round recording (task 4.3, final) (2026-08-22)
|
||||||
|
- `88bed529` test(scheduler-sim-decision-parity): regression suite for existing guarantees (task 4.2) (2026-08-22)
|
||||||
|
- `b3190732` test(scheduler-sim-decision-parity): prove capacity-aware parity under contention (task 4.1) (2026-08-22)
|
||||||
|
- `12644094` feat(scheduler-sim-decision-parity): wire reconciliation into the round boundary (task 3) (2026-08-22)
|
||||||
|
- `0c237f60` feat(scheduler-sim-decision-parity): report reconciled decisions through the existing channel (task 2.4) (2026-08-22)
|
||||||
|
- `eccba841` feat(scheduler-sim-decision-parity): reconstruct job allocation with safe ownership (task 2.3) (2026-08-22)
|
||||||
|
- `81358b52` feat(scheduler-sim-decision-parity): add reconciliation trigger and lookup (task 2.2) (2026-08-22)
|
||||||
|
- `134380b8` fix(scheduler-sim-decision-parity): give sim mode driver.c's round-boundary split (task 2.1) (2026-08-22)
|
||||||
|
- `f11f0df3` test(scheduler-sim-decision-parity): add resource-contention replay fixture (task 1.2) (2026-08-22)
|
||||||
|
- `3106c82e` feat(scheduler-sim-decision-parity): relocate resource accounting to core-level (task 1.1) (2026-08-22)
|
||||||
|
- `0b2c856d` docs(scheduler-sim-decision-parity): finalize design and generate tasks (2026-08-22)
|
||||||
|
- `25e35bac` docs(scheduler): document driver-equivalence invariant, scope decision-parity fix spec (2026-08-22)
|
||||||
|
- `94fd1504` chore: sync harness/session state and OpenWolf bookkeeping (2026-08-22)
|
||||||
|
- `979cd24f` docs(scheduler-sim-repl-verbosity): add approved spec (requirements/design/tasks) (2026-08-22)
|
||||||
|
- `aaca3768` docs(dirty-content-fingerprint): mark task 2 container checkbox complete (2026-08-21)
|
||||||
|
- `f4d2c123` test(dirty-content-fingerprint): add Nix production path fingerprint check (2026-08-21)
|
||||||
|
- `a4785418` test(dirty-content-fingerprint): add startup-log-unchanged consistency check (2026-08-21)
|
||||||
|
- `a7d64c82` test(dirty-content-fingerprint): add dev-loop distinguishability regression (2026-08-21)
|
||||||
|
- `c79ccf0f` feat(dirty-content-fingerprint): xceadmin shows fingerprint in --version (2026-08-21)
|
||||||
|
- `2bf62a64` feat(dirty-content-fingerprint): xceadmin build-info module gains fingerprint (2026-08-21)
|
||||||
|
- `d65d18ad` feat(dirty-content-fingerprint): xcecli shows fingerprint in --version (2026-08-21)
|
||||||
|
- `a4c1b5ad` feat(dirty-content-fingerprint): apiserver shows fingerprint in -version (2026-08-21)
|
||||||
|
- `aa3391a9` feat(dirty-content-fingerprint): Makefile computes dev-loop fingerprint (2026-08-21)
|
||||||
|
- `502dd023` feat(dirty-content-fingerprint): apiserver/xcecli meson pass fingerprint to go_build (2026-08-21)
|
||||||
|
- `b9499493` feat(dirty-content-fingerprint): go_build injects fingerprint via ldflags (2026-08-21)
|
||||||
|
- `d3d9522f` feat(dirty-content-fingerprint): worker shows fingerprint in --version (2026-08-21)
|
||||||
|
- `fd62e26b` feat(dirty-content-fingerprint): controller/scheduler show fingerprint in --version (2026-08-21)
|
||||||
|
- `7ebcba6d` feat(dirty-content-fingerprint): add --version-only composition with dirty_fingerprint (2026-08-21)
|
||||||
|
- `fc5cba15` feat(dirty-content-fingerprint): Nix flake computes fingerprint via self.narHash (2026-08-21)
|
||||||
|
- `c9d40ba1` feat(dirty-content-fingerprint): xce-build.sh computes dev-loop fingerprint (2026-08-21)
|
||||||
|
- `a860141b` feat(dirty-content-fingerprint): add dirty_fingerprint meson option and C define (2026-08-21)
|
||||||
|
- `d3c583b3` docs(dirty-content-fingerprint): add spec (requirements, design, tasks) (2026-08-21)
|
||||||
|
- `084a5a95` test(build-time-versioning): add xceadmin end-to-end build/install/version cycle check (2026-08-20)
|
||||||
|
- `79941a9f` test(build-time-versioning): add version/startup-log consistency check (2026-08-20)
|
||||||
|
- `0eefc95f` test(build-time-versioning): add motivating-incident regression script (2026-08-20)
|
||||||
|
- `7f5c87ce` feat(build-time-versioning): xceadmin committed fallback stub and CLI version wiring (2026-08-20)
|
||||||
|
- `e1cd9d25` feat(build-time-versioning): xceadmin generates build-info module (2026-08-20)
|
||||||
|
- `fb09afe6` feat(build-time-versioning): xcecli --version via Cobra rootCmd.Version (2026-08-20)
|
||||||
|
- `17e837b1` feat(build-time-versioning): apiserver -version flag and startup log (2026-08-20)
|
||||||
|
- `45c9b026` feat(build-time-versioning): Makefile ldflags for make build-server/build-cli (2026-08-20)
|
||||||
|
- `071f3895` feat(build-time-versioning): thread commit_id/build_time into go_build env (2026-08-20)
|
||||||
|
- `18ae0cc3` feat(build-time-versioning): go_build injects commit/build-time via ldflags (2026-08-20)
|
||||||
|
- `bc5c669b` feat(build-time-versioning): worker --version and startup log (2026-08-20)
|
||||||
|
- `88649a04` feat(build-time-versioning): controller/scheduler --version and startup log (2026-08-20)
|
||||||
|
- `48a86cda` feat(build-time-versioning): version macro/helper functions with unknown fallback (2026-08-20)
|
||||||
|
- `d08fda76` feat(build-time-versioning): nix packages compute build time at build phase (2026-08-20)
|
||||||
|
- `bca63974` feat(build-time-versioning): xce-build.sh computes real commit id/build time (2026-08-20)
|
||||||
|
- `1ff9e66c` docs(build-time-versioning): add spec (requirements, design, research) (2026-08-20)
|
||||||
|
- `4b7cc786` feat(build-time-versioning): add build_time meson option and C-wide define (2026-08-20)
|
||||||
|
- `e61fa2b3` docs(roadmap): add resource-distribution-mapping (existed since 2026-07-23, never tracked here) (2026-08-20)
|
||||||
|
- `a16d32e3` docs(roadmap): close out four low-risk backlog cleanup items (2026-08-19)
|
||||||
|
- `0519d24d` docs(scheduler-core): reconcile stale doc references after today's fixes (2026-08-19)
|
||||||
|
- `aa42752d` docs(scheduler-core): mark task 12.5 complete with fresh verification evidence (2026-08-19)
|
||||||
|
- `1bc65d2c` docs(resource-value-verification): close design-review doc/comment gaps (2026-08-19)
|
||||||
|
- `e52c68f4` fix(controller): admission-gate OPCode_schedAdmin's blocking wait (Req 5 AC 10) (2026-08-19)
|
||||||
|
- `c9acd502` fix(scheduler): clock/resval/jobid-format fixes plus REPL live-operator gaps (task 12.5) (2026-08-19)
|
||||||
|
- `70fe9e46` fix(worker): dbus_workers_update() starves workers after first DONE/EXITED (bug-1589) (2026-08-19)
|
||||||
|
- `ef404131` docs(resource-value-verification): commit spec files, mark implementation complete (2026-08-18)
|
||||||
|
- `9be9e954` test(resource-value-verification): verdict-logic regression selftest (task 8.2) (2026-08-18)
|
||||||
|
- `19104d31` test(resource-value-verification): full live e2e validation run (task 8.1) (2026-08-18)
|
||||||
|
- `13646647` feat(resource-value-verification): verdict aggregation, reporting, cleanup (task 7.1) (2026-08-18)
|
||||||
|
- `8222c031` feat(resource-value-verification): apiserver-side verification check (task 6.4) (2026-08-18)
|
||||||
|
- `b673b7c9` fix(resource-value-verification): close bug-1579 for real via xce_test's ! shell-escape (2026-08-18)
|
||||||
|
- `72b6fdea` feat(resource-value-verification): scheduler-side verification check (task 6.3) (2026-08-18)
|
||||||
|
- `cb5bbfcc` feat(resource-value-verification): controller-side verification check (task 6.2) (2026-08-18)
|
||||||
|
- `75a0839d` feat(resource-value-verification): worker join and resource injection (task 6.1) (2026-08-18)
|
||||||
|
- `da41cce2` feat(resource-value-verification): wire deployment into skill via start-all.sh (task 5.1) (2026-08-18)
|
||||||
|
- `8b4fbffa` feat(resource-value-verification): add apiserver resval trace log (task 4.2) (2026-08-18)
|
||||||
|
- `e68444f1` feat(resource-value-verification): add scheduler resval trace log (task 4.1) (2026-08-18)
|
||||||
|
- `8f542a38` feat(resource-value-verification): extend xce-deploy-ha with apiserver step + start-all.sh (task 3.1) (2026-08-18)
|
||||||
|
- `f5390c14` feat(resource-value-verification): per-name accept/reject verification in resupdate (task 2.2) (2026-08-18)
|
||||||
|
- `058335d2` feat(resource-value-verification): wire resolver into resupdate, add -t flag (task 2.1) (2026-08-17)
|
||||||
|
- `642e2362` feat(resource-value-verification): scaffold xce-test-resources skill (task 1.2) (2026-08-17)
|
||||||
|
- `98b8c34b` feat(resource-value-verification): extract resupdate type-resolution logic (task 1.1) (2026-08-17)
|
||||||
|
- `9f539583` docs: sync skills and steering docs with bug-1562..1566 findings (2026-08-17)
|
||||||
|
|
||||||
|
### build
|
||||||
|
- `f4d2c123` test(dirty-content-fingerprint): add Nix production path fingerprint check (2026-08-21)
|
||||||
|
- `a4785418` test(dirty-content-fingerprint): add startup-log-unchanged consistency check (2026-08-21)
|
||||||
|
- `a7d64c82` test(dirty-content-fingerprint): add dev-loop distinguishability regression (2026-08-21)
|
||||||
|
- `b9499493` feat(dirty-content-fingerprint): go_build injects fingerprint via ldflags (2026-08-21)
|
||||||
|
- `fc5cba15` feat(dirty-content-fingerprint): Nix flake computes fingerprint via self.narHash (2026-08-21)
|
||||||
|
- `c9d40ba1` feat(dirty-content-fingerprint): xce-build.sh computes dev-loop fingerprint (2026-08-21)
|
||||||
|
- `a860141b` feat(dirty-content-fingerprint): add dirty_fingerprint meson option and C define (2026-08-21)
|
||||||
|
- `084a5a95` test(build-time-versioning): add xceadmin end-to-end build/install/version cycle check (2026-08-20)
|
||||||
|
- `79941a9f` test(build-time-versioning): add version/startup-log consistency check (2026-08-20)
|
||||||
|
- `0eefc95f` test(build-time-versioning): add motivating-incident regression script (2026-08-20)
|
||||||
|
- `18ae0cc3` feat(build-time-versioning): go_build injects commit/build-time via ldflags (2026-08-20)
|
||||||
|
- `d08fda76` feat(build-time-versioning): nix packages compute build time at build phase (2026-08-20)
|
||||||
|
- `bca63974` feat(build-time-versioning): xce-build.sh computes real commit id/build time (2026-08-20)
|
||||||
|
- `4b7cc786` feat(build-time-versioning): add build_time meson option and C-wide define (2026-08-20)
|
||||||
|
- `83e61eec` feat(xce-deploy-ha): support real xcecd worker deployment alongside virtual (2026-08-19)
|
||||||
|
- `8f542a38` feat(resource-value-verification): extend xce-deploy-ha with apiserver step + start-all.sh (task 3.1) (2026-08-18)
|
||||||
|
- `0f1df499` feat(deploy,xceadmin): standalone xceadmin packaging + deploy-ha usability (bug-1566) (2026-08-17)
|
||||||
|
- `1c943e98` feat(build): add --incremental flag to xce-build.sh (bug-1565) (2026-08-17)
|
||||||
|
|
||||||
|
### misc
|
||||||
|
- `94fd1504` chore: sync harness/session state and OpenWolf bookkeeping (2026-08-22)
|
||||||
|
- `83e61eec` feat(xce-deploy-ha): support real xcecd worker deployment alongside virtual (2026-08-19)
|
||||||
|
- `9be9e954` test(resource-value-verification): verdict-logic regression selftest (task 8.2) (2026-08-18)
|
||||||
|
- `13646647` feat(resource-value-verification): verdict aggregation, reporting, cleanup (task 7.1) (2026-08-18)
|
||||||
|
- `8222c031` feat(resource-value-verification): apiserver-side verification check (task 6.4) (2026-08-18)
|
||||||
|
- `b673b7c9` fix(resource-value-verification): close bug-1579 for real via xce_test's ! shell-escape (2026-08-18)
|
||||||
|
- `72b6fdea` feat(resource-value-verification): scheduler-side verification check (task 6.3) (2026-08-18)
|
||||||
|
- `cb5bbfcc` feat(resource-value-verification): controller-side verification check (task 6.2) (2026-08-18)
|
||||||
|
- `75a0839d` feat(resource-value-verification): worker join and resource injection (task 6.1) (2026-08-18)
|
||||||
|
- `da41cce2` feat(resource-value-verification): wire deployment into skill via start-all.sh (task 5.1) (2026-08-18)
|
||||||
|
- `8f542a38` feat(resource-value-verification): extend xce-deploy-ha with apiserver step + start-all.sh (task 3.1) (2026-08-18)
|
||||||
|
- `642e2362` feat(resource-value-verification): scaffold xce-test-resources skill (task 1.2) (2026-08-17)
|
||||||
|
- `9f539583` docs: sync skills and steering docs with bug-1562..1566 findings (2026-08-17)
|
||||||
111
reports/aug23-aug29-2026.md
Normal file
111
reports/aug23-aug29-2026.md
Normal file
|
|
@ -0,0 +1,111 @@
|
||||||
|
# XCE Project Status — Aug 23 – Aug 29, 2026
|
||||||
|
|
||||||
|
> Note: the calendar-default window (Aug 17–22) is already covered by
|
||||||
|
> `aug16-aug22-2026.md`; this report covers the actual gap — the last
|
||||||
|
> complete day before this report was generated (Aug 30 is in progress).
|
||||||
|
|
||||||
|
## TL;DR
|
||||||
|
|
||||||
|
An unusually dense week: 97 real commits shipped two specs end-to-end
|
||||||
|
(`scheduler-dispatch-commit`, `xce-test-rc-file`), drove
|
||||||
|
`scheduler-sim-eager-reconciliation` and `scheduler-snapshot-job-allocation`
|
||||||
|
to substantial completion, and closed out a systematic five-bug sweep of
|
||||||
|
WAL-replay field-completeness gaps alongside several safety-critical HA
|
||||||
|
correctness fixes (vote-denial on persist failure, stale-term
|
||||||
|
re-election blocking, fail-loud disconnect handling). Last week's #1
|
||||||
|
open thread — 39 duplicate buglog IDs — was fully repaired on Monday.
|
||||||
|
The week closed with a deep multi-session live-debugging investigation
|
||||||
|
that fixed three confirmed production bugs (a scheduler resource leak,
|
||||||
|
a controller heap-use-after-free that crashed 2 of 3 controllers under
|
||||||
|
load, and a set of permanently-dead job-notification hooks that could
|
||||||
|
leave a job's state stuck as RUN forever in the scheduler and apiserver)
|
||||||
|
and made major forensic progress on a fourth, still-open scheduler crash
|
||||||
|
(bug-1714) — proved it's a genuine cross-thread use-after-free, not the
|
||||||
|
overflow first suspected, but the exact stale-pointer source remains
|
||||||
|
unisolated.
|
||||||
|
|
||||||
|
## Commits by day
|
||||||
|
|
||||||
|
### Sun 08-23 — `scheduler-dispatch-commit` spec, start to finish in one day
|
||||||
|
|
||||||
|
- **`5724d1e9`→`a9cfee5f`** (13 commits) — dispatch-failure classification, a durable `WAL_JOB_DISPATCH` fact type, scheduler transitions to RUN at commit time, controller-side retryable/terminal dispatch-failure resolution with bounded escalation, and full multi-round + terminal + retryable e2e coverage. Spec marked implementation-complete (`f784e2bd`).
|
||||||
|
|
||||||
|
### Mon 08-24 — Gitea issue triage and buglog integrity repair
|
||||||
|
|
||||||
|
- **`1c9cfc1f`, `e83d207a`** — reproduced and logged 4 Gitea issues (#40–42, #44) plus a `gitea-sync` bug found while doing it; corrected an earlier false-negative on #43 (bug-1678 — turned out reproducible).
|
||||||
|
- **`db142d41`→`88c7c05d`** — fixed the 4 reproduced issues: dispatch-time resource allocation not persisted into `WAL_JOB_DISPATCH` (bug-1678), live job-status pushes to the apiserver never firing (bug-1679), `jobs list -l/-o` validating against the wrong struct (bug-1674), unmatched workers not falling back to a default `HostConfig` (bug-1673) — plus `from_host` threaded end-to-end through the wire format, CGO layer, and job-submit interning object (bug-1676).
|
||||||
|
- **`755976fc`** — **repaired last week's #1 open thread**: renumbered 40 buglog entries that had collided with `bug-875`–`bug-949` after the Aug 13–17 counter restart.
|
||||||
|
- **`5c82ad0b`** — **repaired last week's #2 open thread**: synced `spec.json` phase metadata for `build-time-versioning`/`dirty-content-fingerprint` (both were 21/21 done but stuck on `tasks-generated`) and flipped 23 stale section-header checkboxes across 3 more specs whose numbered subtasks were already complete.
|
||||||
|
|
||||||
|
### Tue 08-25 — peak day (46 commits): WAL-replay field-completeness sweep + HA correctness
|
||||||
|
|
||||||
|
- **`955efecd`→`d0e5f85a`** — five-bug systematic sweep: `WAL_JOB_START` losing pidinfo/start_time (bug-1687), `WAL_JOB_FINISH` losing exit_info/finish_time — and exit_info was never set in production at all (bug-1689), `WAL_JOB_RUSAGE` losing run_usage (bug-1690), `WAL_JOB_CKPT` losing ckptinfo (bug-1691), `WAL_JOB_PREASON` losing pend_reasons (bug-1692) — all on restart replay.
|
||||||
|
- **`e5e6cd18`** — **`raft_cb_persist` must deny the vote and roll back state on a persist failure** (bug-1684): `ha_persist_term()`'s every open/write/fsync/rename failure was silently swallowed; the raft library always granted the vote regardless. Safety-critical.
|
||||||
|
- **`3eba7c16`** — fail loud on scheduler/worker disconnect instead of relying on the reply-wait timeout (bug-1693).
|
||||||
|
- **`b4eb9ec8`** — offload `checkpoint_task_fn` off the delaymgr thread (bug-1694, async-I/O blocking-violation).
|
||||||
|
- **`ac46bfa6`** — restored-but-stale raft term permanently blocked re-election after restart (bug-1695), found via the mandatory live HA chaotic soak for this session's controller audit campaign.
|
||||||
|
- **`b040dfc0`, `bc3f76b4`, `6400f96d`** — removed a double-free in the limit plugin's config update (bug-1685), invalidated stale `builtin_fcfs` picker cursors across calls (bug-1686), mirrored `plugin_begin_session`/`plugin_end_session` into the sim-mode driver (bug-1681).
|
||||||
|
- **`9a230f08`** — apiserver `FilterRows` routing non-bitmask operators through bitmask matching (bug-1680, found while verifying bug-1679).
|
||||||
|
|
||||||
|
### Wed 08-26 — `xce-test-rc-file` spec
|
||||||
|
|
||||||
|
- **`9de4db95`→`8fc86ef4`** (7 commits) — rc-file path resolution with XDG precedence, `--norc` flag, pluggable rc-line-loading runner wired into `xce_test` startup, and full e2e coverage for discovery/loading/`--norc`/failure-handling, confirmed not to disturb existing automation.
|
||||||
|
|
||||||
|
### Thu 08-27 — `scheduler-sim-eager-reconciliation` spec + a fix batch + `scheduler-snapshot-job-allocation` kickoff
|
||||||
|
|
||||||
|
- **`e2bb98c8`→`446437bf`** (6 commits) — state-independent dispatch capture, resInUse deferral proof, capture-safety unit coverage, resource-contention/sim-verify/full regression proofs. 6/9 tasks done.
|
||||||
|
- **`f886c809`** — standby dispatch-gate rejections now redirect to a known primary (bug-1700, bug-1701) — closes a gap `admin-config-tool`'s own design doc had flagged as a documented blocker.
|
||||||
|
- **`ac1e312e`, `0ebf3a81`, `47532f0a`, `cfd7db8c`** — `job.start_time` never set due to a dispatch/`WAL_JOB_START` RUN-state collision (bug-1697); repeated `WORKERS_PUSH` with a NULL resval clobbering good values (bug-1688); first-boot promotion evaluating eligibility before its own WAL recovery ran (bug-1696); xceadmin retrying transient scheduler-admin failures after a redirect (bug-1713, bug-1715).
|
||||||
|
- **`097e33ae`→`22353849`** — `scheduler-snapshot-job-allocation` begins: `dup_job_alloc` deep-copy primitive, schedobj construction for restored jobs.
|
||||||
|
|
||||||
|
### Fri 08-28 — `scheduler-snapshot-job-allocation` completion + a heap-overflow fix
|
||||||
|
|
||||||
|
- **`6fa2d970`→`5948e079`** — allocs populated on snapshot save, restored on load, full cross-instance round-trip proof, backward-compat for allocation-less snapshots, allocator-crash regression test, and the final invariant-audit follow-up closed. Spec docs mark it feature-complete.
|
||||||
|
- **`00207503`** — **heap buffer overflow in `make_job_account()` for multi-host jobs** (bug-1722): the function `calloc`'d room for exactly 1 host account pointer but its own loop wrote up to `nhosts-1` — any job spanning more than one host overflowed the allocation.
|
||||||
|
|
||||||
|
### Sat 08-29 — live-debugging investigation: 4 bugs found, 3 fixed, 1 deeply diagnosed
|
||||||
|
|
||||||
|
- **`b1323f8d`** — scheduler leaked `job->accounts`/`job->allocs` on a `JOB_UPDATE` terminal transition (bug-1730).
|
||||||
|
- **`42e1eeaa`** — **heap-use-after-free in `vtask_enq()`** (bug-1733): a task was published to the dispatch queue before its own `owner` field finished initializing; a concurrent worker could dequeue, run, and free it before the producer's own write executed. ASAN-confirmed, crashed 2 of 3 controllers under a live burst-array-job HA workload.
|
||||||
|
- **`ebe691c3`, `67009815`** — `apply_committed_entry()` only ever fired the generic `HOOK_DETAIL_JOB_UPDATE` for job-state changes; `HOOK_DETAIL_JOB_START`/`JOB_EXIT`/`JOB_FAIL` were permanently dead code, leaving the Go apiserver's already-wired `OnJobFinished` handler starved of data — a finished (or failed) job's state could stay stuck as RUN in the scheduler's own view forever (bug-1731). Live-verified end to end on both the scheduler and apiserver sides.
|
||||||
|
- **`83cb660b`** — `xce-build.sh`'s `worker` target never built `xcecd_virtual`, so deploy tooling silently skipped generating the worker-start script in sandboxes with no real systemd manager (bug-1732).
|
||||||
|
- **`347e996e`, `14f95b06`** — worker now distinguishes signal-killed job exits from genuine crashes in its logs; scheduler snapshot load re-derives worker/shared resource "used" totals instead of leaving them at zero until the first session.
|
||||||
|
- **`6193fa7e`** — `test_e2e_live_ha_push.py`'s standby-redirect test rewritten to assert the new success behavior (was asserting the old, now-fixed rejection behavior).
|
||||||
|
- **Not landed as a commit, but the week's deepest investigation**: `bug-1714` (a scheduler `SIGSEGV` in `sched_get_project`, first reported live days earlier) was reproduced twice more this session — once under ASAN (no shadow-memory violation fired), once via a scripted gdb watchpoint that caught the actual corrupting write: freed `job_submit_data_t` memory being recycled by malloc for an unrelated Cap'n Proto decode buffer on a different thread. The free itself was verified legitimate by static reading — the bug is an unidentified second/stale reader elsewhere. Two related side-bugs found during the chase (an orphaned duplicate job silently handed to the picker, and the dedup-race that made it reachable) were fixed and confirmed independent of bug-1714's own root cause, which remains open.
|
||||||
|
|
||||||
|
## Bugs opened/fixed this window
|
||||||
|
|
||||||
|
- **69 genuinely new bugs** (bug-1625 – bug-1733), **66 fixed** across the window's 75 touched entries, **9 still open** at window end. Zero ID collisions — the buglog counter stayed clean all week.
|
||||||
|
- **Both of last week's open threads are resolved**: the 39 duplicate IDs were repaired Monday (`755976fc`); `build-time-versioning`/`dirty-content-fingerprint`'s phase-metadata lag was fixed the same day (`5c82ad0b`).
|
||||||
|
- Highest-severity fixes: **bug-1722** (heap buffer overflow in `make_job_account()` for any multi-host job — memory corruption, not just a crash), **bug-1733** (controller UAF crashing 2/3 nodes under live HA load), **bug-1684** (raft would grant a vote even when its own persist call failed — HA safety-critical), **bug-1695** (a restored-but-stale raft term could permanently block re-election after restart).
|
||||||
|
- Still open at window end (carry-over risk into next week):
|
||||||
|
- **bug-1714** — scheduler UAF crash, mechanism proven, stale-reader source not yet isolated (highest priority — this is the week's one unresolved thread with active investigation momentum).
|
||||||
|
- **bug-1429** — `controller_unit_test` 300s TIMEOUT, deadlock-suspected, open since Jul 10, no owner.
|
||||||
|
- **bug-1560** — `test_recordstart_symlink_escape_replies_error`, a known sandbox filesystem-permissions flake, open since Aug 15.
|
||||||
|
- **bug-1630, bug-1632, bug-1723, bug-1726, bug-1727, bug-1728** — six newer, lower-priority findings (a pre-existing ASAN flake in the sim driver, a test-fixture memory leak, a flaky HA e2e test distinct from bug-1485, a REPL resource-display bug, and a false-ERROR log severity on worker join) — none blocking, none investigated further this week.
|
||||||
|
- **Note on bug-1630 specifically**: its own `fix` field is still empty in the buglog, but the underlying problem it describes (the 39 duplicate IDs) is verifiably resolved — `755976fc`'s renumbering left **zero duplicate IDs** in the current file (re-verified directly). The entry itself just never got marked fixed; worth a one-line buglog update, not further investigation.
|
||||||
|
|
||||||
|
## Spec status changes
|
||||||
|
|
||||||
|
- **`scheduler-dispatch-commit`** — went 0→implementation-complete in one day (Sun); `spec.json` still reads 13/18 tasks despite the phase — metadata lag, not work lag (same pattern as last week's build-time-versioning).
|
||||||
|
- **`xce-test-rc-file`** — 7/10 tasks, phase `tasks-generated`.
|
||||||
|
- **`scheduler-sim-eager-reconciliation`** — 6/9 tasks, phase `tasks-generated`.
|
||||||
|
- **`scheduler-snapshot-job-allocation`** — driven to feature-complete per its own closing commit message, but `spec.json` phase still reads `tasks-generated` at 11/15 tasks — another phase/reality mismatch to sync.
|
||||||
|
- **`scheduler-config-version-tag`** — new spec created (requirements/design/research/tasks all approved), 0/8 tasks — not started.
|
||||||
|
- **`admin-config-tool`** — no new work, but its own design doc's last documented blocker (the standby-redirect gap) was closed this week by `f886c809`.
|
||||||
|
|
||||||
|
## Stats
|
||||||
|
|
||||||
|
- **164 commits** in the window — **97 real** (feature/fix/docs/test), **67 `.wolf` bookkeeping/checkpoint commits**.
|
||||||
|
- **341 non-`.wolf` files changed**, **+26,205 / −1,002** lines (excluding `.wolf/`; including it the raw figure is +220,722 / −1,049,394, dominated by buglog renumbering and token-ledger churn).
|
||||||
|
- Busiest areas: `src/controller` (46 files), `.kiro` (42), `src/scheduler` (35), `.wolf` (25), `src/frontend` (16).
|
||||||
|
- **2 specs shipped end-to-end** (`scheduler-dispatch-commit`, `xce-test-rc-file`), **2 more substantially advanced** (`scheduler-sim-eager-reconciliation`, `scheduler-snapshot-job-allocation`), **1 new spec created**.
|
||||||
|
- **18 sessions** logged in `.wolf/memory.md`, single busiest day Aug 23 (383 logged actions).
|
||||||
|
- Commits landed on **all 7 days**; peak day **Tue 08-25** (46 commits — the WAL-replay + HA sweep).
|
||||||
|
|
||||||
|
## Open threads / to-do heading into next session
|
||||||
|
|
||||||
|
1. **Isolate bug-1714's stale-reader.** The UAF mechanism is proven (freed `job_submit_data_t` recycled across threads); the actual second reader that still holds a pointer past the job's legitimate free is not yet found. Two leads already ruled unlikely or weak by this week's own static reading — needs either deeper tracing of the pending-update queue lifecycle or another live session with pointer-tracking instrumentation.
|
||||||
|
2. **Sync `scheduler-dispatch-commit` and `scheduler-snapshot-job-allocation`'s `spec.json` phase metadata** — both report a phase behind their actual completion state, the same lag class fixed for two other specs this Monday.
|
||||||
|
3. **Five lower-priority open bugs** (1632, 1723, 1726, 1727, 1728) have no assigned follow-up yet — worth a triage pass to confirm none has quietly become higher-priority. (bug-1630 is housekeeping only — see the note above; just needs its buglog entry marked fixed.)
|
||||||
|
4. **bug-1429** (controller test-suite deadlock-suspected timeout) has been open since Jul 10 with no owner — the oldest unresolved item in the log.
|
||||||
Loading…
Add table
Add a link
Reference in a new issue