lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs(wolf): STATUS.md -- bug-1736 exposure, bug-1914..1922, soak gates; merge pending

Browse source 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-08 03:08:20 +00:00
parent 145e41d8e0
commit 99f479082a
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
1 changed files with 13 additions and 14 deletions
Download patch file Download diff file Expand all files Collapse all files

27
.wolf/STATUS.md
View file

@ -5,18 +5,14 @@
---
## ✅ Done (2026-10-03/04)
## ✅ Done (2026-10-03..08, all pushed on `scheduler-framework`)
- **HA fixes, committed + pushed** (`88a8bf8a`..`e27bc2ed`, Gitea #338-#340 closed):
- bug-1915: a PreVote round is decided once; a leader promotes only with its own election's votes; `raft_on_leader_lost` keeps `voted_for`.
- bug-1919: the promotion fence releases once a node that booted RECOVER_TRUNCATED has caught up (runtime `recovery_truncation_healed` latch; the re-check re-arms).
- bug-1920: replicated WAL writes are refused before this boot's `persist_recover` (they truncated and misfiled `seg-...0001`, stalling catch-up); catch-up streams from min(join position, standby report).
- Soaks on that code: sequential 4x20 + 50, chaotic 50, loaded 30 (40 spinners on 32 cores) -- all PASS, WAL segment layout clean on every node.
- **bug-1914, bug-1917, bug-1918 fixed, committed + pushed:**
- bug-1918: `INVOKE_MSG_HANDLER` no longer drops a refused packet (leak).
- bug-1917: `xce_ctx_reconnect` quiesces the old endpoint (unmonitor, clear owner, drain refs, 2 s bound) before re-init.
- bug-1914: config-push barrier 500 -> 2000 ms under the commit-wait admission reservation; honest timeout reply.
- Verification: 127/127 suites; 80/80 live HA config pushes under 40 spinners; loaded seq-30 soak PASS, no ASAN.
- **HA fixes** (Gitea #338-#340): bug-1915 (a PreVote round is decided once; a leader promotes only with its own election's votes), bug-1919 (promotion fence released once a RECOVER_TRUNCATED node has caught up), bug-1920 (replicated WAL writes refused before this boot's `persist_recover`; catch-up streams from min(join position, standby report)).
- **bug-1914/1917/1918** (Gitea #341-#343): config-push barrier 2000 ms under the commit-wait admission reservation; `xce_ctx_reconnect` quiesces the old endpoint (follow-up `f26a2d45`: from its own task it skips the wait, and stale-generation tasks are dropped); refused inbound packets are no longer leaked.
- **bug-1736 exposure** (Gitea #344, `1cedd7f7`): capnp `Job.runUsage` carries the controller's run usage to the apiserver and into snapshots (checkpoints used to drop it); REST columns `max_mem`, `avg_mem`, `used_mem`, `cpu_time`, `peak_cpu`, `peak_cpu_time`, `peak_cpu_duration`, `mem_efficiency`, `avg_cpu_efficiency`, `peak_cpu_efficiency`; xcecli `-o` fields `MAX_MEM`, `AVG_MEM`, `MEM_EFFICIENCY`, `CPU_PEAK`, `CPU_PEAK_DURATION`, `CPU_EFFICIENCY`, `CPU_PEAK_EFFICIENCY` and an LSF-style usage block in `jobs list -l`.
- **Harness fake worker** (Gitea #345, #346): bug-1921 (handler notifications deferred to the delay-manager thread; re-join after a reconnect), bug-1922 (answers the worker-rejoin-resync job-state query).
- **Soak** (`5cf92335`): `wal-layout-check.py` gate before PASS (sequential/chaotic/truncated-fence) and the `--truncated-fence` mode (truncated boot -> refused promotion -> fence released -> leads).
- Verification across these: 127/127 suites; soaks sequential 4x20 + 50 + 20, chaotic 50, loaded 30 (40 spinners on 32 cores), truncated-fence -- all PASS; 80/80 live HA config pushes under load; live run-usage check (fake worker -> controller -> apiserver -> xcecli).
---
@ -25,17 +21,18 @@
**Goal:** _None active._ Pick from the open items below.
### Open items (not yet actioned)
- **Checks needing a host with systemd/dbus** (this container has none): a running array element prints the right `XCE_JOBINDEX`; bug-1908's kept signal reaches a real (non-virtual) worker over IPC at RUNNING.
- **Exposing bug-1736's peak/efficiency fields in the apiserver/xcecli** (decision pending): needs a wire change (`run_usage` on `Job`, or a new notify event) plus cgo handler, store update and xcecli display -- spec-sized (`/kiro-spec-quick`), or drop it; nothing consumes the fields yet.
- **Merge `scheduler-framework` upstream** (`nexa/xce`, which has never had a PR): the branch descends from upstream `develop` (2,139 commits ahead, 0 behind -- fast-forward) and is 2,162 ahead / 2 behind `master` (`fdfc42f1`, Nov 2025). Base branch to be decided.
- **Checks needing a host with systemd/dbus** (this container has none): a running array element prints the right `XCE_JOBINDEX`; bug-1908's kept signal reaches a real (non-virtual) worker over IPC at RUNNING; real `xcecd` (cgroup-measured) run usage in `xcecli jobs list -l` end to end.
- **Specs awaiting the next phase:** design review -- `xce-node-manager`; requirements written -- `job-status-detail-display`, `scheduler-preemption`, `scheduler-sim-decision-trace`; initialized -- `commit-batch-window`, `resource-distribution-mapping`; brief -- `component-rejoin-resync`.
- **Roadmap backlog (briefs only):** application-profile → job-sandbox-execution → job-sandbox-multinode; job-time-window → job-dependency → job-dependency-insight; rbac-persistence; scheduler-alter-pend-reasons; controller-job-cleanup.
- Small, optional: the fake worker reports finished jobs as `unrecognized` in its resync answer (no finished-job retention); `--truncated-fence` is opt-in (could join a default soak, ~3 min).
### Files to create / edit
- _none pending_
### Open decisions
- **REST job output for jobs with a custom `out_file`/`err_file`**: the worker never writes those paths (always `/tmp/job_<id>_task<N>.out`), so since bug-1508 `GET /jobs/:id/output` for such a job returns a permission error from the worker. Either the apiserver's `resolveOutputPath()` (controllers/output.go) always targets the worker's own output file, or the worker starts honouring `out_file`. Waiting on the user.
- **Bug records not mirrored to Gitea:** ~400 older records plus bug-1885–bug-1908 (bug-1914/1915/1917-1920 are mirrored). A sync without `--since`/`--ids` would create an issue for each. Default: leave them off Gitea.
- **Bug records not mirrored to Gitea:** ~400 older records plus bug-1885–bug-1908 (bug-1736 and bug-1914/1915/1917-1922 are mirrored). A sync without `--since`/`--ids` would create an issue for each. Default: leave them off Gitea.
---
@ -53,5 +50,7 @@
python3 scripts/buglog.py list --open | new ... --binary <bin> | check <id> | status <id> fixed --commit <sha>
ln -sf ../../scripts/git-hooks/pre-commit .git/hooks/pre-commit # once per clone
XCE_HA_ROUNDS=6 nix develop ./src --command bash .claude/skills/xce-test-ha/run-ha-soak.sh --sequential <dir>
nix develop ./src --command bash .claude/skills/xce-test-ha/run-ha-soak.sh --truncated-fence <dir>
python3 .claude/skills/xce-test-ha/wal-layout-check.py <deploy dir> # WAL segment layout (bug-1920)
python3 scripts/gitea-sync.py --dry-run --bugs --since <date> [--ids bug-N,bug-M] # dry-run first
```