myxce/reports/sep13-sep19-2026.md

XCE Project Status — Sep 13 – Sep 19, 2026

Note: the calendar-default window (Sep 07–Sep 12) is already covered by sep06-sep12-2026.md; this report covers the actual gap — the last completed day before this report was generated.

TL;DR

A scheduler-architecture week, one spec per day or two: Monday relocated xce_test out of controller into its own standalone harness component; Tuesday was a live-replay-driven resource-accounting firefight (7 bugs fixed same-day — stale JOBS_PUSH resync races, a systemd rusage sentinel bug, a reconciliation-freeze gap); Wednesday shipped controller-status's Req 12 (election term/time) and Req 13 (per-controller connection status) deltas to feature-level GO; Thursday landed scheduler-sim-diagnostics' human-readable limit-inspection feature (Req 1–5) end-to-end, plus a run of deploy-ha worker-cleanup fixes; Friday–Saturday built and shipped scheduler-sim-live-allocation — a genuinely large unification merging the normal-mode driver's and sim-mode driver's job-election/allocation logic into one shared implementation (sched_election.c), closing bug-1825/1826 (sim-mode limit-plugin bootstrap and matching) along the way. 25 new bugs were opened, 22 fixed, 2 deliberately deferred, and 1 fix attempt was reverted after it broke the recorded-decision-reconciliation live-verify test. ⚠️ A mass bug-log retag on Saturday touched 1,164 historical entries with an identical timestamp — at least one now carries a "fixed" tag that contradicts its own fix-field prose (see Bugs section).

Commits by day

Sun 09-13 (Sunday) — bookkeeping only

• 04807985..b8ef82c1 (2 commits) — added last week's report (sep06-sep12-2026.md) and CHANGES.md, then a routine .wolf sync. No code changes.

Mon 09-14 (Monday) — xce-harness-component: full spec in one day

• cf55d691..868c6427 (19 commits) — relocated xce_test and its four standalone test binaries out of controller into a new standalone harness component (own /xce-build harness target, decoupled from controller's build), promoted ctrlstatus_fmt.h to the shared xce/ include path, and wired harness into xce-test-ha/xce-test-multihost/xce-test-resources's explicit build/check steps. Verified build isolation, xce_test's post-relocation behavioral parity, and every downstream workflow end-to-end (tasks 1–4, all in one day). Also fixed bug-1807 (getopt_long's optind=0 reset, cross-command state corruption) and bug-1808 (a longjmp'd Unity assertion leaking a spawned xce_test child process).

Tue 09-15 (Tuesday) — live-replay resource-accounting firefight

• b7250c2e..7f3237ae (8 commits) — a live-replay debugging session found and fixed, same-day: bug-1809 (routine EEXIST resync log flooding operational output), bug-1810 (cbuf_put/cbuf_get orphaning their mutex on cancellation, hanging vtaskmgr_fini), bug-1812/1813 (a stale JOBS_PUSH resync claim could destroy a real allocation; job-array expansion moved off the vtask thread pool as the underlying architectural fix), bug-1814 (worker rusage collection was treating systemd's UINT64_MAX sentinel as real memory data), bug-1815 (mem-usage reconciliation now freezes for a RECONCILING/UNAVAIL worker too, closing a gap next to bug-1801), and bug-1816 (xce-deploy-ha: stop a stale deploy's processes surviving a hand-typed cleanup mistake). Also added a live-computed default for xce-build's --jobs flag.

Wed 09-16 (Wednesday) — controller-status: two requirement deltas to GO

• 63de9473..d19e9f27 (20 commits) — Req 12 (primary election term/time): new schema fields + ha_state_t, stamped on both promotion roads, carried on the heartbeat, sourced in ctrlstatus_fill, rendered in the ctrlstatus CLI — reached feature-level GO. Req 13 (per-controller connection status): same shape (schema field → ctrlstatus_fill → CLI render) — reached feature-level GO, but its own live E2E validation (task 8.6) found bug-1819: ctrlstatus_fill was preferring a stale disconnected registry slot over a live one (slot-reuse, first-match-wins bug), fixed same-day. Also fixed bug-1818 (a dead-code gate had cmd_ctrlstatus never actually reaching its own display-layer election check) and a zarray_fini/xlist_fini bug that left stale caps/count behind instead of resetting them.

Thu 09-17 (Thursday) — scheduler-sim-diagnostics limit inspection, plus deploy-ha/xceadmin fixes

• baa07d78..77924f78 (18 commits, 2 .wolf-only) — built info limit's full feature set end-to-end: name/all resolution, configured-setting display via fmt_limit(), effective host-scope notes (Req 1.6), live usage for both cluster-wide and host-scoped limit instances, and REPL dispatch wiring — with a read-only/deterministic proof as the closing task. Found and logged bug-1824 (a latent, pre-existing zarray bounds imprecision in repl_info_resource_one(), left unfixed as out-of-scope) along the way. Same day: bug-1820/1823 (stop-all.sh/cleanup.sh couldn't find a real, sudo-launched worker regardless of process-tree depth — two rounds, the second a regression on the first's own fix) and bug-1821/1822 (xceadmin now accepts a comma-separated --controller failover list; found a NoPrimaryError retry gap live while verifying it).

Fri 09-18 (Friday) — scheduler-sim-live-allocation: foundation

• fc23ea8d..2f1ffd1e (8 commits) — started the driver-unification effort: threaded core state through allocation context and host search, made plugin-hook dispatch, capacity checks, allocation-dispatch/decision-commit, and limit matching all "core-safe" (usable from either driver). Same day, fixed bug-1825: the limit plugin was never registered on any sim-mode driver path at all — a limit_new_plugin() factory plus a new sim_bootstrap_limit_plugin() (mirroring sim_picker.c's own bootstrap shape) closed the gap.

Sat 09-19 (Saturday) — scheduler-sim-live-allocation: unification to done

• 4195331b..8c363de7 (13 commits, 1 .wolf-only) — extracted the shared picker/allocator into a new sched_election.c, wired simulation mode onto it, narrowed reconciliation to a timing-gap safety net, closed a host-scoped limit fan-out seam gap, then proved contention parity across drivers and replay-driven limit visibility (closing bug-1826, the sim-mode limit-matching follow-up to Friday's bootstrap fix) and reconciled the equivalence test suites. Also fixed bug-1682/1492 (plugin_end_session() guarding on the wrong field) and bug-1827 (a core_clock.c coverage gap in the core-isolation test found during an /xce-audit-invariants scheduler run), and corrected a stale bug-1611 status claim in the driver-equivalence steering doc.

Bugs opened/fixed this window

• 25 genuinely new bugs (bug-1805 – bug-1829): 22 fixed, 2 deliberately deferred (bug-1806 — accepted as a documented Meson platform limitation, user decision 2026-09-14; bug-1824 — latent pre-existing bug, out of its discovering task's scope), 1 reverted (bug-1811 — an initial guard-based fix broke test_e2e_original_motivation_long_replay, the live-verify for the entire Recorded-Decision Reconciliation mechanism, and was rolled back; reclassified as a design decision, not a defect).
• Carry-over status from last week's report: xce-test-cli-ux's spec.json metadata lag — still unsynced, now 2 windows running. bug-1796/1797/1793/1777 — still open, deliberately deferred, unowned. bug-1714 — still zero movement, now a 3rd consecutive window (last real progress Aug 26). bug-1632/1726/1728 — still zero movement, also a 3rd consecutive window. gitea MCP connectivity — still failing to connect (confirmed again this session).
• Highest-severity fixes: bug-1817 (a SIGSEGV core dump in a live, HA-connected xce_scheduler — a "previously-unroot-caused, recurring bug class" per its own tags, fixed at the zarray/xlist library-primitive level), bug-1814/1815 (a live-user-traced memory-accounting chain: a systemd rusage sentinel bug plus the reconciliation-freeze gap it exposed), bug-1819 (a display-layer stale-slot bug found during controller-status's own live E2E validation).
• ⚠️ Integrity concern: a mass edit on Sat, last_seen=2026-09-19T09:00:00Z, added the tag closed-by-scheduler-sim-live-allocation (plus, on many entries, a bare fixed tag) to 1,164 historical bug entries — including clearly unrelated ones from as far back as May (bug-002). At least one (bug-1811) now carries a fixed tag that directly contradicts its own fix-field text ("REVERTED"). Any future tooling that trusts .tags or .fix-field-non-empty as a status oracle will misclassify entries like this — the fix field's actual prose (read fresh) remains the only reliable signal. Worth either a stale-tag cleanup pass or documenting this caveat permanently in the buglog's own schema notes.

Spec status changes

• xce-harness-component — implementation-complete (Mon), 16/20 tasks checked (the 4 unchecked are top-level group headers whose own subtasks are all done — a checkbox-convention gap, not real incompleteness). Metadata is correctly synced this time.
• controller-status — Req 12 and Req 13 deltas both reached feature-level GO this window (Wed); 29/30 tasks done, the 1 remaining explicitly marked optional (8.5, live-cluster observation). spec.json still reads phase tasks-generated / ready_for_implementation: true rather than implementation-complete — a metadata-sync candidate, same class of gap flagged for xce-test-cli-ux.
• scheduler-sim-diagnostics — Requirements 1–5 (limit inspection) shipped end-to-end (Thu), 17/19 tasks checked (tasks 8 and 10's own subtasks are all done; only their parent headers are unflipped, same convention gap as above). Requirement 6 (human-readable units) had not started as of this window's close.
• scheduler-sim-live-allocation — a new spec, taken from Task 1.1 to Task 5.4 (all subtasks complete per commit history) in two days (Fri–Sat). spec.json still reads tasks-generated / ready_for_implementation: false — the THIRD spec this window showing the same phase/checkbox metadata lag (alongside xce-test-cli-ux and controller-status), suggesting a quick batch phase-sync pass would clear all three at once.

Stats

• 89 commits — 83 touching at least one non-.wolf file, 6 pure-.wolf bookkeeping (4 routine chore(openwolf) syncs, 2 buglog-only edits).
• 454 non-.wolf file changes, +21,856 / −2,047 (non-wolf headline; raw incl. .wolf: +30,969 / −5,452).
• Busiest areas by files touched: src/scheduler (78), src/harness (50), .kiro (25), src/lib (12), src/controller (12), .claude (8), src/xceadmin (7), .wolf (6), src/include (4).
• By spec scope: controller-status (19 commits), xce-harness-component (18), scheduler-sim-live-allocation (14), scheduler-sim-diagnostics (13), scheduler (7, unscoped fixes).
• 1 spec shipped and correctly synced (xce-harness-component); 2 requirement deltas reached feature-level GO (controller-status Req 12/13); 2 specs functionally complete pending metadata sync (scheduler-sim-diagnostics Req 1–5, scheduler-sim-live-allocation).
• 10 sessions logged in .wolf/memory.md touching this window (Sep 14 ×3, Sep 15 ×3, Sep 16 ×2, Sep 17 ×1, Sep 18 ×1); busiest logged day Sep 18 (383 actions — the scheduler-sim-live-allocation foundation day).
• Commits landed on all 7 days; peak day Wed 09-16 (20 commits, controller-status Req 12+13), close behind by Sat 09-19 (13 commits, closing scheduler-sim-live-allocation).

Open threads / to-do heading into next session

1. Metadata-lag pattern now spans three specs: xce-test-cli-ux (2 windows unaddressed), and newly controller-status (29/30 done, both deltas GO) and scheduler-sim-live-allocation (all subtasks done) — all three still read a stale phase/ready_for_implementation in spec.json. A single batch phase-sync pass would clear all three at once.
2. Buglog integrity: the Sat mass retag (1,164 entries, closed-by-scheduler-sim-live-allocation, identical last_seen) leaves .tags and .fix-non-empty unreliable as status signals — confirmed via bug-1811's contradicting fixed tag. Needs a cleanup pass or a documented caveat before any tooling trusts those fields directly again.
3. gitea sync backlog now 32 bugs deep (bug-1798 through bug-1829) — the MCP server is still failing to connect (CONNECT_TIMEOUT, reconfirmed this session). Retry gitea-sync once reachable.
4. bug-1714/bug-1632/bug-1726/bug-1728 — zero movement for a 3rd consecutive window each. Worth an explicit triage-or-accept decision rather than letting them keep aging silently.
5. bug-1796/bug-1797/bug-1793/bug-1777 — still deliberately deferred from xce-test-cli-ux's validation tasks, still unowned.
6. scheduler-sim-eager-reconciliation's live-replay smoke pass (flagged last week) — still not directly done; scheduler-sim-live-allocation's tasks 5.1/5.2/5.4 delivered adjacent live-replay proof, but for a different spec — the original open item remains untouched.
7. scheduler-sim-live-allocation shipped a full driver-unification architecture in two days — before treating it as fully done, it warrants a live multi-round replay or HA-adjacent smoke pass beyond the unit suite, the same caution this project's own /xce-audit-invariants methodology applies to architectural/consistency changes of this size.