myxce/.wolf/STATUS.md

STATUS

Single source of truth for resuming work. Read this FIRST when starting a session. Update at the end of every work phase so the next /clear resumes in 1 read.

---

✅ Done (2026-10-03..08, all pushed on scheduler-framework)

• HA fixes (Gitea #338-#340): bug-1915 (a PreVote round is decided once; a leader promotes only with its own election's votes), bug-1919 (promotion fence released once a RECOVER_TRUNCATED node has caught up), bug-1920 (replicated WAL writes refused before this boot's persist_recover; catch-up streams from min(join position, standby report)).
• bug-1914/1917/1918 (Gitea #341-#343): config-push barrier 2000 ms under the commit-wait admission reservation; xce_ctx_reconnect quiesces the old endpoint (follow-up f26a2d45: from its own task it skips the wait, and stale-generation tasks are dropped); refused inbound packets are no longer leaked.
• bug-1736 exposure (Gitea #344, 1cedd7f7): capnp Job.runUsage carries the controller's run usage to the apiserver and into snapshots (checkpoints used to drop it); REST columns max_mem, avg_mem, used_mem, cpu_time, peak_cpu, peak_cpu_time, peak_cpu_duration, mem_efficiency, avg_cpu_efficiency, peak_cpu_efficiency; xcecli -o fields MAX_MEM, AVG_MEM, MEM_EFFICIENCY, CPU_PEAK, CPU_PEAK_DURATION, CPU_EFFICIENCY, CPU_PEAK_EFFICIENCY and an LSF-style usage block in jobs list -l.
• Harness fake worker (Gitea #345, #346): bug-1921 (handler notifications deferred to the delay-manager thread; re-join after a reconnect), bug-1922 (answers the worker-rejoin-resync job-state query).
• Soak (5cf92335): wal-layout-check.py gate before PASS (sequential/chaotic/truncated-fence) and the --truncated-fence mode (truncated boot -> refused promotion -> fence released -> leads).
• Verification across these: 127/127 suites; soaks sequential 4x20 + 50 + 20, chaotic 50, loaded 30 (40 spinners on 32 cores), truncated-fence -- all PASS; 80/80 live HA config pushes under load; live run-usage check (fake worker -> controller -> apiserver -> xcecli).

---

🚀 Next phase

Goal: None active. Pick from the open items below.

Open items (not yet actioned)

• Review + merge PR #347 (nexa/xce#347, rshen:scheduler-framework -> nexa/xce develop, the repo's first PR): 2,140 commits, fast-forward onto develop; most of the +2 M lines are vendored Go modules. master (fdfc42f1) is 2 commits ahead of develop and was not merged into the branch.
• Checks needing a host with systemd/dbus (this container has none): a running array element prints the right XCE_JOBINDEX; bug-1908's kept signal reaches a real (non-virtual) worker over IPC at RUNNING; real xcecd (cgroup-measured) run usage in xcecli jobs list -l end to end.
• Specs awaiting the next phase: design review -- xce-node-manager; requirements written -- job-status-detail-display, scheduler-preemption, scheduler-sim-decision-trace; initialized -- commit-batch-window, resource-distribution-mapping; brief -- component-rejoin-resync.
• Roadmap backlog (briefs only): application-profile → job-sandbox-execution → job-sandbox-multinode; job-time-window → job-dependency → job-dependency-insight; rbac-persistence; scheduler-alter-pend-reasons; controller-job-cleanup.
• Small, optional: the fake worker reports finished jobs as unrecognized in its resync answer (no finished-job retention); --truncated-fence is opt-in (could join a default soak, ~3 min).

Files to create / edit

• none pending

Open decisions

• REST job output for jobs with a custom out_file/err_file: the worker never writes those paths (always /tmp/job_<id>_task<N>.out), so since bug-1508 GET /jobs/:id/output for such a job returns a permission error from the worker. Either the apiserver's resolveOutputPath() (controllers/output.go) always targets the worker's own output file, or the worker starts honouring out_file. Waiting on the user.
• Bug records not mirrored to Gitea: ~400 older records plus bug-1885–bug-1908 (bug-1736 and bug-1914/1915/1917-1922 are mirrored). A sync without --since/--ids would create an issue for each. Default: leave them off Gitea.

---

📁 Active architecture

• Stack: Go API/CLI tier (src/frontend/apiserver) fronting C daemons (src/controller, src/worker, src/scheduler); Meson-based build via src/tools/xce-build.sh, hermetic nix dev shell.
• Bug tracking: .wolf/bugs/*.md (records) → scripts/buglog.py (CLI + checks) → .wolf/buglog.json (generated index) → scripts/gitea-sync.py (Gitea issues in nexa/xce).
• Local-only, never committed: .claude/settings.json (hook paths rewritten for this machine), data/ (450 MB replays/logs).

---

🔧 Useful commands

python3 scripts/buglog.py list --open | new ... --binary <bin> | check <id> | status <id> fixed --commit <sha>
ln -sf ../../scripts/git-hooks/pre-commit .git/hooks/pre-commit   # once per clone
XCE_HA_ROUNDS=6 nix develop ./src --command bash .claude/skills/xce-test-ha/run-ha-soak.sh --sequential <dir>
nix develop ./src --command bash .claude/skills/xce-test-ha/run-ha-soak.sh --truncated-fence <dir>
python3 .claude/skills/xce-test-ha/wal-layout-check.py <deploy dir>   # WAL segment layout (bug-1920)
python3 scripts/gitea-sync.py --dry-run --bugs --since <date> [--ids bug-N,bug-M]   # dry-run first