chore(wolf): regenerate handoff, sync anatomy/memory bookkeeping
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
25d8a461fa
commit
eee76a75ab
GPG key ID:
80533536B0EB651C
6 changed files with 952 additions and 825 deletions
|
|
@ -5,54 +5,48 @@
|
|||
|
||||
---
|
||||
|
||||
## ✅ Done
|
||||
## ✅ Done (2026-09-27, all pushed)
|
||||
|
||||
- **Worker security fixes** (2026-09-26, pushed, all TDD RED→GREEN, worker suite 12/12, sequential HA soaks PASS on xcecd_virtual):
|
||||
- **bug-1508** (`7723a4f4`): worker `fetchfile` (root) opened any path. Now only `<dir>/job_<jobid>_task<N>.out` for the request's jobid (exact match, `xcecd_is_job_output_path()` in `worker/include/job_output_path.h`), O_NOFOLLOW|O_NONBLOCK, regular single-link file. Also closed the REST route (submit with `out_file=/etc/shadow`, then GET output).
|
||||
- **bug-1868** (`7864a3da`): job output file created as root in /tmp followed planted links. `xcecd_create_job_output()` unlinks + O_EXCL|O_NOFOLLOW.
|
||||
- **bug-1869** (`5df33e69`): `xcecd_worker.log` (was chmod 0666 by path), `/tmp/xcecd_worker.log` append, and the restart flag. `worker/include/xcecd_tmpfile.h`: `xcecd_open_owned_file()` / `xcecd_owned_file_exists()`, fchmod instead of chmod(path).
|
||||
- Live gap: the sandbox only runs `xcecd_virtual`, so the real root worker's output/log creation is covered by unit tests only.
|
||||
- **bug-1754** (2026-09-26, `6876e4f6`, pushed): a failed job's exit code is now persisted. A worker reports every nonzero exit as a FAIL (not FINISH), and WAL_JOB_FAIL had no code slot, so `job.exit_info` was NULL for every failed job. WAL_JOB_FAIL now carries an int32 code trailer (`persist_submit_job_fail_fact`); both apply seams adopt it via `apply_job_fail_exit_code()` (non-POST_EXEC stages only). `apply_wal_entry` also no longer assigns wire state 0 for STAGE_FLAG FAIL entries. New `test_wal_job_fail_fact.c`; controller suite 1228/0. Soaks: sequential 6 rounds PASS (SIMVERIFY 7 files, 0 mismatches) + factstream 2 rounds PASS.
|
||||
- **HA stability fixes** (2026-09-25, pushed, soak-verified): bug-1864 (per-endpoint FIFO in vtask dispatcher), bug-1865 (primary control endpoint refcounted, no double-close), bug-1850 (NOT_READY retries in place, no failover rotate), bug-1854 (re-pushed resource defs are idempotent).
|
||||
- **bug-1714 / bug-1840** marked cannot-reproduce (0 crashes across 2,400-element burst on old + current builds).
|
||||
- **Gitea sync** (2026-09-26): issues #264-#289 created (#287-#289 = bug-1508/1868/1869) (recent bugs + bug-1798..1804, 1714, 1754, 1850, 1854, 1508, 1868, 1869), #74 closed. Only a subset of the buglog is mirrored by design; `scripts/gitea-sync.py` gained `--ids` because bulk triage bumped the date fields `--since` relies on.
|
||||
- Logged: **bug-1866** (worker `launch_type` leak), **bug-1867** (controller snapshot drops `exit_info` — capnp `Job` has no exit field).
|
||||
- **Bug records are Markdown** (`d26ff887`): one `.wolf/bugs/<id>.md` per bug, written to `.claude/skills/buglog/bug-record.md` (bug-1866 = worked example). `buglog.json` is a generated index; 710 hook auto-detections archived in `buglog-archive.json`; 500 migrated records carry `legacy: true`. `buglog.py status` and the pre-commit hook enforce the standard. `gitea-sync.py` renders records and links fix commits; 23 issues refreshed (`25d8a461`).
|
||||
- **bug-1866** fixed (`e84a8d43`): `remove_worker` frees 5 more strings. Worker suite 12/12.
|
||||
- **HA / async-I/O group**: bug-1450/1451/1728/1433/1474/1469/1741 + bug-1870 follow-up (`cb1415d7`); all 6 HA soak modes PASS. bug-1747 FAIL half and bug-1867 were already fixed (`1d6c04c2`, `e4bb492a`).
|
||||
- Gitea #290–#297 created; weekly report + `CHANGES.md` for Sep 20–26.
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Next phase
|
||||
|
||||
**Goal:** _None active._ Suggested order: bug-1747 FAIL half (add finish_time to the WAL_JOB_FAIL trailer) → bug-1867 (snapshot exit_info, capnp schema) → `xce_test` harness batch + bug-1866 → a tasks-ready spec.
|
||||
**Goal:** _None active._ Suggested order: `xce_test` harness bug batch → bug-1871 → spec metadata sync → a tasks-ready spec.
|
||||
|
||||
### Open items (not yet actioned)
|
||||
- _none pending_
|
||||
- **`xce_test` harness batch** (all in `src/harness`, small): bug-1574, 1702, 1703, 1705, 1707–1710, 1712, 1777, 1793, 1796, 1797. Use `buglog.py status <id> fixed --commit <sha>` — the record must meet the standard first.
|
||||
- **bug-1871** (investigating): HA snapshot transfer to a rejoining standby stalls 3+ s mid-stream (2 of 4 force-snapshot runs). Suspect: edge-triggered epoll not re-signalled for data buffered while the endpoint task was blocked. Evidence: `/home/claude/tmp/xce-ha-final-force-snapshot`.
|
||||
- **Other open bugs** (25 total, `buglog.py list --open`): bug-1553/1580 (intermittent controller_unit_test), bug-1632/1763 (ASAN in sim tests), bug-1604, bug-1736 partial.
|
||||
- **Spec metadata lag**: `controller-status`, `scheduler-sim-live-allocation`, `xce-test-cli-ux`, `scheduler-sim-diagnostics` are functionally done but `spec.json` still reads `tasks-generated`.
|
||||
- **Pending specs awaiting approval**: `controller-peers-env-fallback` (design review), `libxce-alloc-preason-equality` (tasks, 0/8, not ready).
|
||||
|
||||
### Files to create / edit
|
||||
- _none pending_
|
||||
|
||||
### Open decisions
|
||||
- **REST job output for jobs with a custom `out_file`/`err_file`**: the worker never writes those paths (always `/tmp/job_<id>_task<N>.out`), so since bug-1508 `GET /jobs/:id/output` for such a job returns a permission error from the worker. Either the apiserver's `resolveOutputPath()` (controllers/output.go) always targets the worker's own output file, or the worker starts honouring `out_file`. Waiting on the user.
|
||||
- **~400 older bug records were never mirrored to Gitea.** A sync without `--since`/`--ids` would create an issue for each. Default: leave them off Gitea.
|
||||
|
||||
---
|
||||
|
||||
## 📁 Active architecture
|
||||
|
||||
- **Stack:** Go API/CLI tier (`src/frontend/apiserver`) fronting C daemons (`src/controller`, `src/worker`, `src/scheduler`); Meson-based build via `src/tools/xce-build.sh`, hermetic nix dev shell.
|
||||
- **Current focus area:** `src/controller` persistence (WAL fact payloads/apply seams in `persist.c`, `job.c`) and worker root-privilege file handling (`src/worker/include/job_output_path.h`, `xcecd_tmpfile.h`).
|
||||
- **Bug tracking:** `.wolf/bugs/*.md` (records) → `scripts/buglog.py` (CLI + checks) → `.wolf/buglog.json` (generated index) → `scripts/gitea-sync.py` (Gitea issues in `nexa/xce`).
|
||||
- **Local-only, never committed:** `.claude/settings.json` (hook paths rewritten for this machine), `data/` (450 MB replays/logs).
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Useful commands
|
||||
|
||||
```bash
|
||||
# Build controller unit tests in place (nix shell) and run one group
|
||||
cd src && nix develop . --command meson compile -C xce.build controller_unit_test xce_controller
|
||||
ASAN_OPTIONS=detect_leaks=0 ./xce.build/controller/controller_unit_test -t test_wal_job_fail_fact
|
||||
|
||||
# HA soaks (sequential with real worker+scheduler; worker fact stream)
|
||||
python3 scripts/buglog.py list --open | new ... --binary <bin> | check <id> | status <id> fixed --commit <sha>
|
||||
ln -sf ../../scripts/git-hooks/pre-commit .git/hooks/pre-commit # once per clone
|
||||
XCE_HA_ROUNDS=6 nix develop ./src --command bash .claude/skills/xce-test-ha/run-ha-soak.sh --sequential <dir>
|
||||
nix develop ./src --command bash .claude/skills/xce-test-ha/run-ha-soak.sh --factstream <dir>
|
||||
|
||||
# Gitea bug sync (REST, no MCP needed) — always dry-run first
|
||||
python3 scripts/gitea-sync.py --dry-run --bugs --since <date> [--ids bug-N,bug-M]
|
||||
python3 scripts/gitea-sync.py --dry-run --bugs --since <date> [--ids bug-N,bug-M] # dry-run first
|
||||
```
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
{
|
||||
"last_scanned": "2026-09-27T13:00:00.171Z",
|
||||
"git_head": "c21ca0b72fed27413cf289c1162e89639a3e155c",
|
||||
"last_scanned": "2026-09-27T19:00:00.177Z",
|
||||
"git_head": "27b7a84980f63db97c7eb8f24bd42f75e736d2ca",
|
||||
"file_count": 505
|
||||
}
|
||||
File diff suppressed because it is too large
Load diff
|
|
@ -1,7 +1,7 @@
|
|||
# anatomy.md
|
||||
|
||||
> Auto-maintained by OpenWolf. Last scanned: 2026-09-27T13:00:00.163Z
|
||||
> Files: 505 tracked | Anatomy hits: 0 | Misses: 0
|
||||
> Auto-maintained by OpenWolf. Last scanned: 2026-09-28T03:39:29.656Z
|
||||
> Files: 509 tracked | Anatomy hits: 0 | Misses: 0
|
||||
|
||||
## ./
|
||||
|
||||
|
|
@ -9,10 +9,19 @@
|
|||
- `.gitignore` — Git ignore rules (~413 tok)
|
||||
- `.mcp.json` (~7 tok)
|
||||
- `.opencode.json` (~7 tok)
|
||||
- `CHANGES.md` — XCE Change Log (~21260 tok)
|
||||
- `CHANGES.md` — XCE Change Log (~23719 tok)
|
||||
- `CLAUDE.md` — OpenWolf (~1944 tok)
|
||||
- `README.md` — Project documentation (~1157 tok)
|
||||
|
||||
## .claude/rules/
|
||||
|
||||
- `buglog-status.md` — Bug records and their status (~554 tok)
|
||||
|
||||
## .claude/skills/buglog/
|
||||
|
||||
- `bug-record.md` — Bug Record Standard (~3501 tok)
|
||||
- `SKILL.md` — Buglog Skill (~1239 tok)
|
||||
|
||||
## .codegraph/
|
||||
|
||||
- `.gitignore` — Git ignore rules (~61 tok)
|
||||
|
|
@ -685,8 +694,12 @@
|
|||
|
||||
## scripts/
|
||||
|
||||
- `buglog.py` — Read and edit .wolf/buglog.json without re-serializing it. (~3338 tok)
|
||||
- `gitea-sync.py` — GiteaAPI: die, warn, bug_status, bug_state + 13 more (~10815 tok)
|
||||
- `buglog.py` — Read, write and check the XCE bug records: one Markdown file per bug. (~8336 tok)
|
||||
- `gitea-sync.py` — GiteaAPI: die, warn, bug_status, bug_state + 13 more (~10888 tok)
|
||||
|
||||
## scripts/tests/
|
||||
|
||||
- `test_buglog.py` — Tests for scripts/buglog.py (one Markdown record per bug). (~3841 tok)
|
||||
|
||||
## src/
|
||||
|
||||
|
|
|
|||
|
|
@ -1,19 +1,7 @@
|
|||
{
|
||||
"last_heartbeat": "2026-09-27T12:59:52.498Z",
|
||||
"last_heartbeat": "2026-09-28T03:29:52.522Z",
|
||||
"engine_status": "running",
|
||||
"execution_log": [
|
||||
{
|
||||
"task_id": "memory-consolidation",
|
||||
"status": "success",
|
||||
"timestamp": "2026-06-02T09:00:00.411Z",
|
||||
"duration_ms": 3
|
||||
},
|
||||
{
|
||||
"task_id": "memory-consolidation",
|
||||
"status": "success",
|
||||
"timestamp": "2026-06-08T09:00:00.899Z",
|
||||
"duration_ms": 4
|
||||
},
|
||||
{
|
||||
"task_id": "project-suggestions",
|
||||
"status": "failed",
|
||||
|
|
@ -620,6 +608,18 @@
|
|||
"status": "success",
|
||||
"timestamp": "2026-09-27T13:00:00.171Z",
|
||||
"duration_ms": 170
|
||||
},
|
||||
{
|
||||
"task_id": "anatomy-rescan",
|
||||
"status": "success",
|
||||
"timestamp": "2026-09-27T19:00:00.178Z",
|
||||
"duration_ms": 175
|
||||
},
|
||||
{
|
||||
"task_id": "anatomy-rescan",
|
||||
"status": "success",
|
||||
"timestamp": "2026-09-28T01:00:00.013Z",
|
||||
"duration_ms": 11
|
||||
}
|
||||
],
|
||||
"dead_letter_queue": [],
|
||||
|
|
|
|||
|
|
@ -1632,3 +1632,17 @@
|
|||
| 03:51 | Created reports/sep20-sep26-2026.md | — | ~3846 |
|
||||
| 03:51 | Edited reports/sep20-sep26-2026.md | inline fix | ~7 |
|
||||
| 13:06 | Weekly report + CHANGES.md section for Sep 20–26 (82 commits, 41 new bugs, 37 fixed); bug-1747/1867 confirmed already fixed, controller ASAN suite 1245/0 | reports/sep20-sep26-2026.md, CHANGES.md | done, uncommitted | ~6000 |
|
||||
| 06:11 | Edited src/worker/testsuite/test_finished_job_retention.c | modified __SANITIZE_ADDRESS__() | ~512 |
|
||||
| 06:11 | Edited src/worker/testsuite/test_finished_job_retention.c | 2→3 lines | ~32 |
|
||||
| 06:12 | Edited src/worker/testsuite/test_finished_job_retention.c | 2→3 lines | ~35 |
|
||||
| 06:12 | Edited src/worker/src/xcecd_worker_utils.c | expanded (+6 lines) | ~83 |
|
||||
| 20:22 | Created .claude/skills/buglog/bug-record.md | — | ~3735 |
|
||||
| 20:28 | Created scripts/tests/test_buglog.py | — | ~3021 |
|
||||
| 20:30 | Created scripts/buglog.py | — | ~8374 |
|
||||
| 20:30 | Edited scripts/buglog.py | modified exists() | ~90 |
|
||||
| 20:33 | Edited scripts/gitea-sync.py | modified bug_status() | ~76 |
|
||||
| 20:33 | Edited scripts/gitea-sync.py | modified bug_labels() | ~187 |
|
||||
| 20:35 | Edited scripts/tests/test_buglog.py | added error handling | ~598 |
|
||||
| 20:36 | Created .claude/skills/buglog/SKILL.md | — | ~1322 |
|
||||
| 20:37 | Created .claude/rules/buglog-status.md | — | ~591 |
|
||||
| 20:39 | Edited scripts/tests/test_buglog.py | modified test_migrated_record_matching_the_old_index_passes() | ~244 |
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue