myxce/scripts/check-no-fork.py
lei.chen 2fcacac2cb E2 Phase 1: external-license-resources — xceesd daemon, extres role, EXTERNAL admission
Implements Phase 1 of .kiro/specs/external-license-resources/ (design.md):
externally-reported license pools pushed by a dedicated daemon through the
existing resval pipeline, with EXTERNAL-aware admission arithmetic.

Wire & controller:
- controller.capnp HostRole.extres (@5, appended); xce_join_req accepts it
- extres.c: HostRole_extres join table (endpoint+name, no workermgr slot),
  leave cleanup; join.c/cluster.c dispatch
- opupd.c do_update_resvals: extres branch runs the resource hooks and
  update_resval_list(c, NULL, ...) — host=="" entries route through
  update_val_res' SHARED branch into c->sharedinst; per-entry -EPERM for
  non-shared targets is logged, never fails the list
- worker.c update_val_res: NULL worker only writes SHARED resources
- cluster.c cluster_build_resources: rebuilt on resourceApplyDef (bug-1854)
  — EEXIST-tolerant so definitions added by an activation register on a
  RUNNING controller (bug-1973), and RES_APPLY_CHANGED updates definition
  fields (valtype/ordtype/flags/interval) in place so a corrected config
  heals a stale registration (bug-1974); seed_sharedinst_from_config skips
  EXTERNAL pools (daemon-reported free counts only; seeding would fabricate
  capacity while the daemon is down)

Scheduler:
- core_events.c RESVALS_DELTA: NULL/"" host marks a SHARED value and lands
  in core->sharedInst via dup_val (bug-1585 ownership rule), mirroring
  snapshot_load's host=="" precedent — previously WARN-skipped as unknown
  worker
- resource.c unsatisfy_: EXTERNAL skips the inuse term
  (F >= uncomit + wanted; F is already net of our checkouts)

xceesd (src/esd, new component):
- script collectors polled per source, debounce on unchanged values,
  fail-open stale policy; pushes host=="" shared resvals over xcecli
- reconnect gap fix (bug-1975): with a stable value the debounce pushed
  nothing and reconnects only fire inside a push — after a controller
  restart the daemon sat on a dead connection forever; the main loop now
  forces a full re-push when xcecli_is_connection_broken() (bug-1760
  predicate) latches

Config/mapping:
- resources.yaml: extlic sample (location: [external])
- mapping.py: RESOURCE_FLAG_EXTERNAL (0x10) + location:["external"] mapping;
  xceadmin tests

Tests (TDD, red->green per fix): test_extres_resvals.c (join+resval
pipeline), test_config_resource_reconcile.c +2 (register past EEXIST /
hot-apply changed definition), test_core_events.c +131 lines (shared DELTA
landing), mapping tests. Suites: controller 1358/0, scheduler 100/0,
esd 7/0, lib 1/0; check-no-fork covers src/esd.

Live t1 E2E: config gen=15 push -> xceesd push extlic=4 lands in
sharedinst -> rusage[extlic=3] job RUN; second job PENDs at free=1;
release re-admits it; controller restart -> forced re-push restores the
value and admission resumes. Full precommit job verification passed.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-27 03:45:28 +00:00

133 lines
4.6 KiB
Python

#!/usr/bin/env python3
"""Enforce the no-fork rule (.kiro/steering/tech.md, "Process creation").
XCE daemons are multi-threaded (libxce delaymgr/netio/vtaskmgr threads, xcecd's
thread pool); fork() in a multi-threaded process copies only the calling thread,
so any lock another thread held (malloc, logger, stdio) stays locked in the
child. Component code therefore never calls fork-family functions; children are
started with posix_spawn(), or by systemd over D-Bus.
Scanned: C sources of the components the rule covers (COVERED below), minus
test code. Excluded by decision (2026-10-10): tests, the harness (xce_test),
the autoscaler (not finalized; provider calls use posix_spawn since bug-1969)
and the Go frontend.
A call is allowed only in a process proven single-threaded, marked on the call
line or the line above with: /* xce-fork-ok: <why it is single-threaded> */
Usage: check-no-fork.py [repo-root] [file ...] (exit 1 on a violation)
"""
import os
import re
import sys
COVERED = ["src/controller", "src/scheduler", "src/lib", "src/worker",
"src/esd"]
EXCLUDED_DIRS = {"testsuite", "test", "tests"}
# Fork-family calls. uv_spawn() is libuv's fork()+exec on Linux.
BANNED = re.compile(
r"(?<![\w.>])(fork|vfork|forkpty|daemon|system|popen|uv_spawn)\s*\(")
MARK = re.compile(r"xce-fork-ok:\s*\S")
def strip_code(text):
"""Blank comments and string/char literals, keeping line structure, so
names inside them never match. Returns the stripped text."""
out, i, n = [], 0, len(text)
while i < n:
c = text[i]
if text.startswith("/*", i):
j = text.find("*/", i + 2)
j = n if j < 0 else j + 2
out.append(re.sub(r"[^\n]", " ", text[i:j]))
i = j
elif text.startswith("//", i):
j = text.find("\n", i)
j = n if j < 0 else j
out.append(" " * (j - i))
i = j
elif c in "\"'":
j = i + 1
while j < n and text[j] != c and text[j] != "\n":
j += 2 if text[j] == "\\" else 1
j = min(j + 1, n)
out.append(c + " " * (j - i - 2) + c if j - i >= 2 else c)
i = j
else:
out.append(c)
i += 1
return "".join(out)
def disabled_lines(lines):
"""Line indexes inside `#if 0` blocks (nesting aware)."""
dead, depth = set(), 0
for k, line in enumerate(lines):
s = line.strip()
if depth:
if re.match(r"#\s*if", s):
depth += 1
elif re.match(r"#\s*endif", s):
depth -= 1
elif depth == 1 and re.match(r"#\s*(else|elif)", s):
depth = 0
dead.add(k)
elif re.match(r"#\s*if\s+0\b", s):
depth = 1
dead.add(k)
return dead
def check_file(path, rel):
raw = open(path, encoding="utf-8", errors="replace").read()
raw_lines = raw.split("\n")
code_lines = strip_code(raw).split("\n")
dead = disabled_lines(raw_lines)
bad = []
for k, line in enumerate(code_lines):
if k in dead:
continue
for m in BANNED.finditer(line):
prev = raw_lines[k - 1] if k else ""
if MARK.search(raw_lines[k]) or MARK.search(prev):
continue
bad.append(f"{rel}:{k + 1}: {m.group(1)}() "
f"{raw_lines[k].strip()}")
return bad
def covered_files(root):
for top in COVERED:
for d, dirs, files in os.walk(os.path.join(root, top)):
dirs[:] = [x for x in dirs if x not in EXCLUDED_DIRS
and not x.startswith(".")]
for f in files:
if f.endswith((".c", ".h")) and not f.startswith("test"):
yield os.path.join(d, f)
def main(argv):
root = os.path.abspath(argv[1]) if len(argv) > 1 else os.getcwd()
if len(argv) > 2:
files = [os.path.join(root, f) for f in argv[2:]]
covered = set(map(os.path.abspath, covered_files(root)))
files = [f for f in files if os.path.abspath(f) in covered]
else:
files = list(covered_files(root))
bad = []
for f in sorted(files):
bad += check_file(f, os.path.relpath(f, root))
if bad:
print("no-fork rule violated (.kiro/steering/tech.md, "
"\"Process creation\"):")
print("\n".join(" " + b for b in bad))
print("Use posix_spawn() (or a systemd unit over D-Bus). If the process "
"is proven single-threaded, mark the call with "
"/* xce-fork-ok: <why> */.")
return 1
print(f"no-fork rule: {len(files)} file(s) OK")
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))