chore: gitea-sync --ids, sync recent bugs (#264-#286), refresh STATUS.md

gitea-sync.py: --ids syncs named bugs regardless of dates (OR'd with
--since). Bulk triage bumped last_seen/status_updated across most of the
buglog, so --since 2026-09-01 matched ~1150 bugs while recently fixed older
bugs (bug-1798..1804, 1714, 1754, 1850, 1854) could never be reached.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

.claude/skills/gitea-sync/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: gitea-sync
 description: Sync cc-sdd specs and OpenWolf buglog to Gitea milestones/issues in nexa/xce
-argument-hint: [--dry-run] [--spec <name>] [--bugs] [--since <date>]
+argument-hint: [--dry-run] [--spec <name>] [--bugs] [--since <date>] [--ids <ids>]
 triggers:
   - /gitea-sync
   - sync to gitea
@@ -50,6 +50,11 @@ Keep the script and this procedure in step when changing either.
   date; for already-synced bugs (`gitea_issue` set), also mirror those whose
   `status_updated >=` this date, so a status change on an old bug still
   reaches Gitea without flooding it with old never-synced bugs
+- `--ids <bug-N,bug-M,...>`: also sync these bugs regardless of dates (OR'd
+  with `--since`). Needed because bulk triage has bumped `last_seen` /
+  `status_updated` on most of the buglog, and for a never-synced bug
+  `--since` only checks `last_seen` — a recently fixed older bug is otherwise
+  unreachable. Always `--dry-run` first: `--since` alone can match ~1100 bugs.
 
 ### 1. Ensure labels exist
 

.wolf/STATUS.md

@@ -7,20 +7,20 @@
 
 ## ✅ Done
 
-- **scheduler-sim-eager-reconciliation spec, revision 4** (2026-09-12, commit `25fb5c11`): captured jobs' `job->state` now set to `JOB_STAT_RUN` synchronously at dispatch capture (`sim_reconcile_unaccounted_jobs`), mirroring normal-mode's `sched_commit_decision` semantics (bug-1624/bug-1804). All 4 revision tasks in `tasks.md` checked off.
-- **bug-1799/1801**: fixed sim replay resInUse over-commit — `sched_account_jobs`'s per-round re-derive now skips RECONCILING/UNAVAIL workers (matches `core_reset_workers`'s existing bug-1751 skip). Previously a worker's `used` slots could exceed its own `value` capacity after a resync replay (observed 26 used vs. 22 capacity).
-- **bug-1800**: bytes-valued CONSUMABLE resources (mem/swp/tmp) in `info worker` now render with human-scaled units, matching maxmem/maxswap/maxtmp's existing formatting.
-- **bug-1802/1803**: `break ANY` breakpoint kind added then reverted after live feedback it over-fired inside busy sessions; replaced with a prescan-based "next non-dummy session" hint surfaced in `info summary`.
-- Full scheduler test suite verified GREEN: 77/77 tests pass (asan profile, `xce.build`), confirmed via fresh build+test run before committing.
+- **bug-1754** (2026-09-26, `6876e4f6`, pushed): a failed job's exit code is now persisted. A worker reports every nonzero exit as a FAIL (not FINISH), and WAL_JOB_FAIL had no code slot, so `job.exit_info` was NULL for every failed job. WAL_JOB_FAIL now carries an int32 code trailer (`persist_submit_job_fail_fact`); both apply seams adopt it via `apply_job_fail_exit_code()` (non-POST_EXEC stages only). `apply_wal_entry` also no longer assigns wire state 0 for STAGE_FLAG FAIL entries. New `test_wal_job_fail_fact.c`; controller suite 1228/0. Soaks: sequential 6 rounds PASS (SIMVERIFY 7 files, 0 mismatches) + factstream 2 rounds PASS.
+- **HA stability fixes** (2026-09-25, pushed, soak-verified): bug-1864 (per-endpoint FIFO in vtask dispatcher), bug-1865 (primary control endpoint refcounted, no double-close), bug-1850 (NOT_READY retries in place, no failover rotate), bug-1854 (re-pushed resource defs are idempotent).
+- **bug-1714 / bug-1840** marked cannot-reproduce (0 crashes across 2,400-element burst on old + current builds).
+- **Gitea sync** (2026-09-26): issues #264-#286 created (recent bugs + bug-1798..1804, 1714, 1754, 1850, 1854), #74 closed. Only a subset of the buglog is mirrored by design; `scripts/gitea-sync.py` gained `--ids` because bulk triage bumped the date fields `--since` relies on.
+- Logged: **bug-1866** (worker `launch_type` leak), **bug-1867** (controller snapshot drops `exit_info` — capnp `Job` has no exit field).
 
 ---
 
 ## 🚀 Next phase
 
-**Goal:** _None currently active — last quest (scheduler-sim-eager-reconciliation revision + bug-1799..1804) is committed and closed out._
+**Goal:** _None active._ Suggested order: bug-1508 (fetchfile path check, security) → bug-1747 FAIL half (add finish_time to the new WAL_JOB_FAIL trailer) → bug-1867 → `xce_test` harness batch + bug-1866 → a tasks-ready spec.
 
 ### Open items (not yet actioned)
-- `gitea` MCP server failed to connect this session (CONNECT_TIMEOUT) — buglog entries bug-1798 through bug-1804 have not been synced to Gitea issues via the `gitea-sync` skill; retry once the server is reachable.
+- _none pending_
 
 ### Files to create / edit
 - _none pending_
@@ -33,16 +33,21 @@
 ## 📁 Active architecture
 
 - **Stack:** Go API/CLI tier (`src/frontend/apiserver`) fronting C daemons (`src/controller`, `src/worker`, `src/scheduler`); Meson-based build via `src/tools/xce-build.sh`, hermetic nix dev shell.
-- **Current focus area:** `src/scheduler` — sim/replay mode (`sim.c`, `sim_loop.c`, `repl_info.c`) and resource accounting (`resource.c`).
+- **Current focus area:** `src/controller` persistence — WAL fact payloads and apply seams (`persist.c` `apply_committed_entry` / `apply_wal_entry`, `job.c` `record_job_*_fact`).
 
 ---
 
 ## 🔧 Useful commands
 
 ```bash
-# Build + test scheduler only (ASAN profile)
-bash src/tools/xce-build.sh scheduler asan xce.build --jobs <N>
+# Build controller unit tests in place (nix shell) and run one group
+cd src && nix develop . --command meson compile -C xce.build controller_unit_test xce_controller
+ASAN_OPTIONS=detect_leaks=0 ./xce.build/controller/controller_unit_test -t test_wal_job_fail_fact
 
-# Replay a live-captured scheduler session
-./xce.build/scheduler/xce_scheduler -i -s <replay-file>
+# HA soaks (sequential with real worker+scheduler; worker fact stream)
+XCE_HA_ROUNDS=6 nix develop ./src --command bash .claude/skills/xce-test-ha/run-ha-soak.sh --sequential <dir>
+nix develop ./src --command bash .claude/skills/xce-test-ha/run-ha-soak.sh --factstream <dir>
+
+# Gitea bug sync (REST, no MCP needed) — always dry-run first
+python3 scripts/gitea-sync.py --dry-run --bugs --since <date> [--ids bug-N,bug-M]
 ```

.wolf/buglog.json

@@ -15785,6 +15785,8 @@
     },
     {
       "id": "bug-1429",
+      "gitea_synced_status": "fixed",
+      "gitea_issue": 264,
       "status_note": "b5d59253",
       "status": "fixed",
       "status_updated": "2026-09-25",
@@ -23648,6 +23650,8 @@
     },
     {
       "id": "bug-1714",
+      "gitea_synced_status": "cannot-reproduce",
+      "gitea_issue": 265,
       "repro_attempts": "0/2 runs, 2026-09-25: documented recipe (3-node HA + worker + scheduler, 8 waves x 6 concurrent 'jobsub -a 50' = 2400 array elements each) on b63bb581 build.dbg and on d87b78dc-era ASAN build -- no crash, scheduler alive throughout, 0 redelivered array pushes (0 'sched_add_job failed' duplicates; the original repro saw up to 18x redelivery, the recipe's precondition)",
       "status": "cannot-reproduce",
       "status_updated": "2026-09-25",
@@ -24850,6 +24854,8 @@
     },
     {
       "id": "bug-1754",
+      "gitea_synced_status": "fixed",
+      "gitea_issue": 266,
       "status": "fixed",
       "status_updated": "2026-09-26",
       "status_note": "6876e4f6: WAL_JOB_FAIL carries an int32 code trailer; both apply seams adopt it into exit_info (non-POST_EXEC stages). Also fixed apply_wal_entry assigning wire state 0 for STAGE_FLAG FAIL entries.",
@@ -26071,6 +26077,8 @@
     },
     {
       "id": "bug-1798",
+      "gitea_synced_status": "fixed",
+      "gitea_issue": 267,
       "status": "fixed",
       "status_updated": "2026-09-24",
       "timestamp": "2026-09-10T06:57:00Z",
@@ -26099,6 +26107,8 @@
     },
     {
       "id": "bug-1799",
+      "gitea_synced_status": "fixed",
+      "gitea_issue": 268,
       "status": "fixed",
       "status_updated": "2026-09-24",
       "status_note": "25fb5c11",
@@ -26140,6 +26150,8 @@
     },
     {
       "id": "bug-1800",
+      "gitea_synced_status": "fixed",
+      "gitea_issue": 269,
       "status": "fixed",
       "status_updated": "2026-09-24",
       "timestamp": "2026-09-11T07:55:00Z",
@@ -26174,6 +26186,8 @@
     },
     {
       "id": "bug-1801",
+      "gitea_synced_status": "fixed",
+      "gitea_issue": 270,
       "status": "fixed",
       "status_updated": "2026-09-24",
       "timestamp": "2026-09-11T08:35:00Z",
@@ -26210,6 +26224,8 @@
     },
     {
       "id": "bug-1802",
+      "gitea_synced_status": "fixed",
+      "gitea_issue": 271,
       "status": "fixed",
       "status_updated": "2026-09-24",
       "timestamp": "2026-09-11T08:35:00Z",
@@ -26238,6 +26254,8 @@
     },
     {
       "id": "bug-1803",
+      "gitea_synced_status": "fixed",
+      "gitea_issue": 272,
       "status": "fixed",
       "status_updated": "2026-09-24",
       "timestamp": "2026-09-11T09:35:00Z",
@@ -26269,6 +26287,8 @@
     },
     {
       "id": "bug-1804",
+      "gitea_synced_status": "fixed",
+      "gitea_issue": 273,
       "status": "fixed",
       "status_updated": "2026-09-24",
       "timestamp": "2026-09-11T09:55:00Z",
@@ -27404,7 +27424,7 @@
     {
       "id": "bug-1840",
       "repro_attempts": "0/1 ASAN run, 2026-09-25: 3-node HA + worker + scheduler, 8 waves x 6 concurrent 'jobsub -a 50' (2400 elements) on the current ASAN build -- no AddressSanitizer report from the scheduler; plus ~10 ASAN HA soaks today (sequential/chaotic, all with scheduler replay-verify) with no scheduler heap error",
-      "gitea_synced_status": "investigating",
+      "gitea_synced_status": "cannot-reproduce",
       "gitea_issue": 74,
       "status": "cannot-reproduce",
       "status_updated": "2026-09-25",
@@ -27691,6 +27711,8 @@
     },
     {
       "id": "bug-1850",
+      "gitea_synced_status": "fixed",
+      "gitea_issue": 274,
       "status": "fixed",
       "status_updated": "2026-09-25",
       "status_note": "6b1862f8; soaks 2026-09-25: 0 NOT_READY-triggered rotations, 1 live same-controller retry (worker, recovered in 1.2s), all PASS",
@@ -27765,6 +27787,8 @@
     },
     {
       "id": "bug-1854",
+      "gitea_synced_status": "fixed",
+      "gitea_issue": 275,
       "status": "fixed",
       "status_updated": "2026-09-25",
       "status_note": "663ea63c",
@@ -27829,6 +27853,8 @@
     },
     {
       "id": "bug-1857",
+      "gitea_synced_status": "fixed",
+      "gitea_issue": 276,
       "status": "fixed",
       "status_updated": "2026-09-24",
       "status_note": "3cc2f6e3",
@@ -27849,6 +27875,8 @@
     },
     {
       "id": "bug-1858",
+      "gitea_synced_status": "fixed",
+      "gitea_issue": 277,
       "status": "fixed",
       "status_updated": "2026-09-24",
       "status_note": "9e9f2798",
@@ -27870,6 +27898,8 @@
     },
     {
       "id": "bug-1859",
+      "gitea_synced_status": "fixed",
+      "gitea_issue": 278,
       "status": "fixed",
       "status_updated": "2026-09-25",
       "status_note": "7c98c056",
@@ -27892,6 +27922,8 @@
     },
     {
       "id": "bug-1860",
+      "gitea_synced_status": "open",
+      "gitea_issue": 279,
       "status": "open",
       "status_updated": "2026-09-24",
       "status_note": "bug-1812 guard blocks authoritative PEND after disconnected requeue; needs per-job freshness in JOBS_PUSH",
@@ -27914,6 +27946,8 @@
     },
     {
       "id": "bug-1861",
+      "gitea_synced_status": "fixed",
+      "gitea_issue": 280,
       "status": "fixed",
       "status_updated": "2026-09-25",
       "status_note": "b5d59253",
@@ -27934,6 +27968,8 @@
     },
     {
       "id": "bug-1862",
+      "gitea_synced_status": "fixed",
+      "gitea_issue": 281,
       "status": "fixed",
       "status_updated": "2026-09-25",
       "status_note": "c29518b2",
@@ -27956,6 +27992,8 @@
     },
     {
       "id": "bug-1863",
+      "gitea_synced_status": "fixed",
+      "gitea_issue": 282,
       "status": "fixed",
       "status_updated": "2026-09-25",
       "status_note": "aea34738",
@@ -27976,6 +28014,8 @@
     },
     {
       "id": "bug-1864",
+      "gitea_synced_status": "fixed",
+      "gitea_issue": 283,
       "status": "fixed",
       "status_updated": "2026-09-25",
       "status_note": "0d3bb965",
@@ -27999,6 +28039,8 @@
     },
     {
       "id": "bug-1865",
+      "gitea_synced_status": "fixed",
+      "gitea_issue": 284,
       "status": "fixed",
       "status_updated": "2026-09-25",
       "status_note": "910b43cd; soaks 2026-09-25 (2 chaotic + 1 sequential) fdatasync_failed=0, all PASS",
@@ -28022,6 +28064,8 @@
     },
     {
       "id": "bug-1866",
+      "gitea_synced_status": "open",
+      "gitea_issue": 285,
       "status": "open",
       "status_updated": "2026-09-25",
       "status_note": "remove_worker never frees worker->launch_type",
@@ -28041,6 +28085,8 @@
     },
     {
       "id": "bug-1867",
+      "gitea_synced_status": "open",
+      "gitea_issue": 286,
       "status": "open",
       "status_updated": "2026-09-26",
       "status_note": "found while fixing bug-1754; needs a Job.exitCode capnp field + encode/decode, snapshot round-trip test",

.wolf/cerebrum.md

@@ -151,6 +151,7 @@
 - **(2026-08-26, bug-1713) In this sandbox's local-loopback 3-node deploy, the FIRST-BOOT election-settling window varies wildly run to run (sometimes near-instant, sometimes ~10-12s via a term-1 election-timeout-then-PreVote-failover fallback path) — a fixed "wait for start-all.sh's health-check, then fire N requests" tight loop is NOT a reliable way to catch a transient `E_XCE_NOT_READY`/`E_XCE_NOMASTER` window: by the time start-all.sh's own 1s-granularity health-check first observes `CTRL_PRIMARY` in a log and returns, `ha_has_write_quorum()`'s `standby_endpoints` (populated synchronously inside `raft_cb_become_leader` from the raft library's own connected-voter set) is typically ALREADY satisfied, so the gate is already open. A REAL FAILOVER (kill -9 the current primary, then hammer the survivors with NO sleep, no health-check wait) reproduced the exact live symptom reliably instead — got a genuine bare `-10013` at ~t+8.6s post-kill on an unfixed client, and confirmed zero such failures surfaced across the equivalent window on the fixed client. The bug's own root-cause writeup already named this as an equally-valid trigger ("...or any time a real failover is in flight") — prefer the failover trigger over racing first-boot timing when a live repro is needed and cold-boot timing proves flaky.
 - **(2026-08-26) A hand-rolled multi-connection mock TCP/TLS server in a test file needs a genuinely FINITE accept loop (`for handler in handlers:`), not `while self._handlers:` guarding a list that's never mutated — the latter is an always-true condition that leaves the server thread blocked in `accept()` indefinitely once the test's real client stops connecting, which does not fail the test but silently adds `thread.join(timeout=N)`'s FULL timeout to every such test's teardown (measured: 3 tests each added a flat 5s of pure teardown wait with zero effect on pass/fail).** Match the mock's handler-list length exactly to the number of connections the test actually expects (assert on `connection_count` for the "stops on its own budget" property instead of over-provisioning handlers "just in case") and additionally set a short `socket.settimeout()` on the listener as a safety net so a hypothetical future regression that causes MORE connections than expected still fails within one timeout window instead of silently reintroducing the same multi-second stall.
 - **[2026-09-24: open/closed now lives in each entry's `status` field (`.claude/rules/buglog-status.md`), not in `tags` or `fix` text.]** **(2026-09-19, `/xce-audit-invariants scheduler` after `scheduler-sim-live-allocation`) A buglog entry's `tags` array can say `"fixed"` while its own `fix` text says `"Not yet fixed"` — a self-contradiction that survives silently unless something actually re-reads the entry, because most tooling in this project checks tags, not the narrative field.** Found on bug-1682 (and its unnoticed duplicate, bug-1492 — same defect, `plugin_end_session()` guarding on `begin_session_fn` instead of `end_session_fn`, independently filed twice 7 weeks apart because bug-1681's investigation never cross-referenced the earlier entry) via an audit subagent that read BOTH the code and the buglog text, not just the tags — it caught that `plugin.c:441` was still visibly broken despite the tags claiming otherwise. **Lesson 1**: when closing out a large feature and bulk-stamping buglog entries as fixed/superseded, verify each stamped entry's `fix` narrative actually matches the tag before writing it — a tag-only update is a claim with no evidence behind it. **Lesson 2**: before filing a new bug for a defect discovered "while fixing X," grep buglog.json for the exact symptom text first (e.g. `grep -i "begin_session_fn.*end_session_fn"`) — this exact one-line guard/call mismatch was independently re-discovered and re-filed as a "new" bug once already in this project's history.
+- **(2026-09-26, bug-1754) A worker reports a job that exits NONZERO as a FAIL fact, not a FINISH: `xcecd_job_finished()` (worker/src/controller_connector.c) calls `xcecd_notify_job_fail(jobid, stage, exit_code)` whenever `exit_code != 0`, and only sends `xcecli_notify_jobfinish` for exit 0.** So `on_job_finish`/WAL_JOB_FINISH only ever sees successful jobs in production, and anything about a failed job's exit code (exit_info, JOB_FAIL hook payloads, apiserver notify) lives on the `on_job_fail` -> `record_job_fail_fact` -> WAL_JOB_FAIL path, where `job_failinfo_t.code` is the job's real exit code for MAIN-stage failures (but the post-exec script's code for POST_EXEC/Q_POST_EXEC, and a -errno for controller-internal `mark_job_fail` callers). Check both paths before assuming a job-exit fix is complete.
 
 ## Do-Not-Repeat
 - **(2026-09-24) Never hand-edit or re-serialize `.wolf/buglog.json` to log a bug or change its status; use the `buglog` skill (`scripts/buglog.py add | status | note | recur | list | show | triage`).** Each entry's `status` (open / partial / investigating / fixed / cannot-reproduce / wontfix / duplicate, defined in `.claude/rules/buglog-status.md`) is the only source of truth for open/closed. Never infer it from `fix` text or `tags`; both drifted (bug-1772, bug-1682), and gitea-sync closed unfixed bugs by trusting a non-empty `fix` (bug-1855). `add` picks max id + 1. OpenWolf's hook uses entry count + 1, which collides, so its auto-detection is off (bug-1856). After a fix commit: `scripts/buglog.py status <id> fixed --note <sha>`.

scripts/gitea-sync.py

@@ -788,7 +788,7 @@ def _resolve_label_ids(api, label_names):
 
 # ── Bug Sync ─────────────────────────────────────────────────────────────────
 
-def sync_bugs(api, dry_run=False, quiet=False, since=None):
+def sync_bugs(api, dry_run=False, quiet=False, since=None, ids=None):
     """Sync buglog entries to Gitea issues.
 
     Open/closed and the status label come from each entry's `status` (see
@@ -801,6 +801,10 @@ def sync_bugs(api, dry_run=False, quiet=False, since=None):
         since: ISO date (YYYY-MM-DD). Create issues only for bugs with
                last_seen >= since; update already-synced ones whose
                last_seen or status_updated >= since.
+        ids:   set of bug ids to sync regardless of dates. Combined with
+               since, a bug is synced if it matches either -- bulk triage
+               has bumped last_seen/status_updated across most of the log,
+               so dates alone cannot pick out recently fixed older bugs.
     """
     if not BUGLOG_PATH.exists():
         warn(f"buglog not found: {BUGLOG_PATH}")
@@ -827,10 +831,14 @@ def sync_bugs(api, dry_run=False, quiet=False, since=None):
 
         # --since: new issues only for recently seen bugs; status changes
         # (status_updated) are still mirrored for already-synced ones.
-        if since:
-            recent = (bug.get("last_seen") or "") >= since
-            if bug.get("gitea_issue"):
-                recent = recent or (bug.get("status_updated") or "") >= since
+        if since or ids:
+            recent = False
+            if since:
+                recent = (bug.get("last_seen") or "") >= since
+                if bug.get("gitea_issue"):
+                    recent = recent or (bug.get("status_updated") or "") >= since
+            if ids and bid in ids:
+                recent = True
             if not recent:
                 skipped += 1
                 continue
@@ -972,6 +980,8 @@ def main():
                         help="Sync bugs only")
     parser.add_argument("--since", metavar="DATE",
                         help="Create issues only for bugs last seen >= DATE; update synced ones changed since (YYYY-MM-DD)")
+    parser.add_argument("--ids", metavar="ID[,ID...]",
+                        help="Also sync these bug ids regardless of --since (comma-separated, e.g. bug-1798,bug-1799)")
     parser.add_argument("--quiet", action="store_true",
                         help="Suppress progress output, print only errors")
     args = parser.parse_args()
@@ -1007,7 +1017,10 @@ def main():
 
     # Sync bugs
     if not args.spec:
-        sync_bugs(api, dry_run=args.dry_run, quiet=args.quiet, since=args.since)
+        ids = ({i.strip() for i in args.ids.split(",") if i.strip()}
+               if args.ids else None)
+        sync_bugs(api, dry_run=args.dry_run, quiet=args.quiet,
+                  since=args.since, ids=ids)
 
     if not args.quiet:
         if args.dry_run: