From 6dd70f10916073a0eeff3aadbfcfa0b6127a3bb8 Mon Sep 17 00:00:00 2001 From: Rongsong Shen Date: Sat, 10 Oct 2026 13:35:52 +0000 Subject: [PATCH] feat(build): no fork in multi-threaded XCE processes, enforced by check-no-fork Design constraint (tech.md "Process Creation"): the controller, scheduler, xcecd and anything that starts libxce threads never call fork, vfork, forkpty, daemon, system, popen or uv_spawn; children start with posix_spawn() or a systemd unit. A process proven single-threaded may fork, marked /* xce-fork-ok: */. Tests, harness, autoscaler (not finalized; bug-1969) and the Go frontend are out of scope. - scripts/check-no-fork.py scans src/{controller,scheduler,lib,worker} (comment/string/#if 0 aware); runs as meson test "no-fork rule" in the lib/controller/scheduler/worker suites and in the pre-commit hook. - worker: drop xcecd's unused fork-based worker start paths (process_manager_start_process, process_manager_start_worker; xcecd starts xcecd_worker over D-Bus) and a dead #if 0 daemonize fork. - xcecd_worker's fork/uv_spawn/system calls are marked: its call graph from main(), callbacks included, reaches no pthread_create or libuv thread-pool call. Co-Authored-By: Claude Opus 5.5 (1M context) --- .kiro/steering/tech.md | 9 + CLAUDE.md | 1 + scripts/check-no-fork.py | 131 +++++++++++++ scripts/git-hooks/pre-commit | 16 +- src/meson.build | 10 + src/tools/xce-build.sh | 2 +- src/worker/CLAUDE.md | 3 +- src/worker/include/process_manager.h | 9 - src/worker/include/process_worker_manager.h | 7 - src/worker/src/main.c | 15 -- src/worker/src/process_manager.c | 199 -------------------- src/worker/src/process_worker_manager.c | 182 ------------------ src/worker/src/uv_proc.c | 4 + src/worker/src/xcecd.c | 37 +--- src/worker/src/xcecd_worker.c | 1 + 15 files changed, 182 insertions(+), 444 deletions(-) create mode 100644 scripts/check-no-fork.py diff --git a/.kiro/steering/tech.md b/.kiro/steering/tech.md index 793389b8..64ba8f69 100644 --- a/.kiro/steering/tech.md +++ b/.kiro/steering/tech.md @@ -85,6 +85,15 @@ Early return on every error. Log with zap fields before returning. - Go: `gofmt` / `goimports` (post-edit hooks configured) - C: `clang-format` using project config at `xce/.clang-format` +### Process Creation (no fork in multi-threaded processes) + +- **Rule**: a multi-threaded XCE process never calls `fork()`, `vfork()`, `forkpty()`, `daemon()`, `system()`, `popen()` or libuv's `uv_spawn()` (a `fork()`+`exec` on Linux). `fork()` copies only the calling thread, so a lock another thread held (malloc arena, logger mutex, stdio, ASAN) stays locked in the child, which then deadlocks before it reaches `exec`. +- **Threaded processes**: the controller, the scheduler, the `xcecd` daemon (`xcecd`, `xcecd_virtual`: thread pool, network, resource-sync and connector threads), and any process that starts libxce's delaymgr/netio/vtaskmgr threads (`xce_ctx_start()`). +- **How to start a child instead**: `posix_spawn()`/`posix_spawnp()` (glibc runs it with `CLONE_VM|CLONE_VFORK`, no copied user code), or a systemd unit over D-Bus as `xcecd` does for `xcecd_worker`. Setup that `posix_spawn()` cannot do (setuid, rlimits, cgroup join) belongs in a single-threaded process such as `xcecd_worker`, never in the threaded daemon. +- **Allowed**: `exec*()` without a fork (process replacement, e.g. the controller's HA self-restart); and a fork-family call in a process proven single-threaded, marked on the call line or the line above with `/* xce-fork-ok: */`. `xcecd_worker` (the per-job process) is single-threaded: its call graph from `main()`, callbacks included, reaches no `pthread_create()` and no libuv thread-pool call (`uv_queue_work`, `uv_fs_*`, `uv_getaddrinfo`); re-prove it before adding a thread there. +- **Scope**: C code under `src/controller`, `src/scheduler`, `src/lib`, `src/worker`. Excluded (decision 2026-10-10): test code (`testsuite/`, `test*` files; forking to isolate a test case is fine), the harness (`src/harness`, `xce_test`), the autoscaler (not finalized; its `plug_mgr.c` fork is tracked as a bug and must be fixed before it ships), and the Go frontend (`os/exec` is safe). +- **Enforcement**: `scripts/check-no-fork.py` runs as the meson test `no-fork rule` (suites lib, controller, scheduler, worker, so `/xce-build` runs it) and in the pre-commit hook on staged C files. + ### Auth and RBAC - Auth backend set in `config/auth.yaml` (`provider: config_file | pam | ldap | oauth2`). Only `config_file` and `pam` (Linux + CGO) are functional; `ldap` and `oauth2` are documented as future/stub providers whose `Authenticate()` always returns a "not yet implemented" error. diff --git a/CLAUDE.md b/CLAUDE.md index 97c8849e..9f4e1529 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -34,6 +34,7 @@ For details on any topic below, read the corresponding steering file in `.kiro/s - **TDD mandatory**: red → green → refactor; test in `testsuite/` before C changes, `_test.go` before Go changes - **Async I/O**: delaymgr is trigger-only and must never block (no `connect()`, no file I/O) — it drives heartbeats/election timeouts, so blocking there causes spurious failover. Long/unbounded blocking goes on the slow lane; bounded blocking in vtask handlers is permitted only under an admission reservation (`COMMIT_WAIT_RESERVED`). `xce_request`/`xce_reply` are always non-blocking. See `ha-async-io.md` - **Connections are long-lived**: closing signals RDHUP failover. Inbound connections are vote-only; outbound carry all state push +- **No fork in multi-threaded processes**: controller, scheduler, `xcecd` and anything that starts libxce threads never call `fork`/`vfork`/`daemon`/`system`/`popen`/`uv_spawn`; start children with `posix_spawn()` or a systemd unit. A process proven single-threaded (`xcecd_worker`) may fork, marked `/* xce-fork-ok: */`. Tests, harness, autoscaler and Go are out of scope; enforced by `scripts/check-no-fork.py` (meson test + pre-commit). See `tech.md` "Process Creation" - **Middleware chain** (non-negotiable): WhitelistURL → WhitelistIP → RateLimit → JWT → CORS → RBAC - **Error pattern**: `ReturnError(code, msg, nil); return` — early return on every error - **Build**: `/xce-build [component]` — debug + ASAN + tests. Components: `all` (default), `lib`, `controller`, `harness`, `scheduler`, `worker`, `apiserver`, `xcecli`, `xceadmin`. See `build-deployment.md` for full commands diff --git a/scripts/check-no-fork.py b/scripts/check-no-fork.py new file mode 100644 index 00000000..09bfcf7e --- /dev/null +++ b/scripts/check-no-fork.py @@ -0,0 +1,131 @@ +#!/usr/bin/env python3 +"""Enforce the no-fork rule (.kiro/steering/tech.md, "Process creation"). + +XCE daemons are multi-threaded (libxce delaymgr/netio/vtaskmgr threads, xcecd's +thread pool); fork() in a multi-threaded process copies only the calling thread, +so any lock another thread held (malloc, logger, stdio) stays locked in the +child. Component code therefore never calls fork-family functions; children are +started with posix_spawn(), or by systemd over D-Bus. + +Scanned: C sources of the components the rule covers (COVERED below), minus +test code. Excluded by decision (2026-10-10): tests, the harness (xce_test), +the autoscaler (not finalized; tracked as a bug) and the Go frontend. + +A call is allowed only in a process proven single-threaded, marked on the call +line or the line above with: /* xce-fork-ok: */ + +Usage: check-no-fork.py [repo-root] [file ...] (exit 1 on a violation) +""" +import os +import re +import sys + +COVERED = ["src/controller", "src/scheduler", "src/lib", "src/worker"] +EXCLUDED_DIRS = {"testsuite", "test", "tests"} +# Fork-family calls. uv_spawn() is libuv's fork()+exec on Linux. +BANNED = re.compile( + r"(?])(fork|vfork|forkpty|daemon|system|popen|uv_spawn)\s*\(") +MARK = re.compile(r"xce-fork-ok:\s*\S") + + +def strip_code(text): + """Blank comments and string/char literals, keeping line structure, so + names inside them never match. Returns the stripped text.""" + out, i, n = [], 0, len(text) + while i < n: + c = text[i] + if text.startswith("/*", i): + j = text.find("*/", i + 2) + j = n if j < 0 else j + 2 + out.append(re.sub(r"[^\n]", " ", text[i:j])) + i = j + elif text.startswith("//", i): + j = text.find("\n", i) + j = n if j < 0 else j + out.append(" " * (j - i)) + i = j + elif c in "\"'": + j = i + 1 + while j < n and text[j] != c and text[j] != "\n": + j += 2 if text[j] == "\\" else 1 + j = min(j + 1, n) + out.append(c + " " * (j - i - 2) + c if j - i >= 2 else c) + i = j + else: + out.append(c) + i += 1 + return "".join(out) + + +def disabled_lines(lines): + """Line indexes inside `#if 0` blocks (nesting aware).""" + dead, depth = set(), 0 + for k, line in enumerate(lines): + s = line.strip() + if depth: + if re.match(r"#\s*if", s): + depth += 1 + elif re.match(r"#\s*endif", s): + depth -= 1 + elif depth == 1 and re.match(r"#\s*(else|elif)", s): + depth = 0 + dead.add(k) + elif re.match(r"#\s*if\s+0\b", s): + depth = 1 + dead.add(k) + return dead + + +def check_file(path, rel): + raw = open(path, encoding="utf-8", errors="replace").read() + raw_lines = raw.split("\n") + code_lines = strip_code(raw).split("\n") + dead = disabled_lines(raw_lines) + bad = [] + for k, line in enumerate(code_lines): + if k in dead: + continue + for m in BANNED.finditer(line): + prev = raw_lines[k - 1] if k else "" + if MARK.search(raw_lines[k]) or MARK.search(prev): + continue + bad.append(f"{rel}:{k + 1}: {m.group(1)}() " + f"{raw_lines[k].strip()}") + return bad + + +def covered_files(root): + for top in COVERED: + for d, dirs, files in os.walk(os.path.join(root, top)): + dirs[:] = [x for x in dirs if x not in EXCLUDED_DIRS + and not x.startswith(".")] + for f in files: + if f.endswith((".c", ".h")) and not f.startswith("test"): + yield os.path.join(d, f) + + +def main(argv): + root = os.path.abspath(argv[1]) if len(argv) > 1 else os.getcwd() + if len(argv) > 2: + files = [os.path.join(root, f) for f in argv[2:]] + covered = set(map(os.path.abspath, covered_files(root))) + files = [f for f in files if os.path.abspath(f) in covered] + else: + files = list(covered_files(root)) + bad = [] + for f in sorted(files): + bad += check_file(f, os.path.relpath(f, root)) + if bad: + print("no-fork rule violated (.kiro/steering/tech.md, " + "\"Process creation\"):") + print("\n".join(" " + b for b in bad)) + print("Use posix_spawn() (or a systemd unit over D-Bus). If the process " + "is proven single-threaded, mark the call with " + "/* xce-fork-ok: */.") + return 1 + print(f"no-fork rule: {len(files)} file(s) OK") + return 0 + + +if __name__ == "__main__": + sys.exit(main(sys.argv)) diff --git a/scripts/git-hooks/pre-commit b/scripts/git-hooks/pre-commit index e7b8aecb..a8b1493d 100755 --- a/scripts/git-hooks/pre-commit +++ b/scripts/git-hooks/pre-commit @@ -1,7 +1,19 @@ #!/bin/sh # Refuse commits whose bug records (.wolf/bugs/*.md) do not meet -# .claude/skills/buglog/bug-record.md, or whose .wolf/buglog.json index is stale. +# .claude/skills/buglog/bug-record.md, or whose .wolf/buglog.json index is stale; +# and commits whose staged C sources break the no-fork rule +# (.kiro/steering/tech.md, "Process creation"; scripts/check-no-fork.py). # Install: ln -sf ../../scripts/git-hooks/pre-commit .git/hooks/pre-commit +top="$(git rev-parse --show-toplevel)" if git diff --cached --name-only -- .wolf/bugs/ .wolf/buglog.json | grep -q .; then - exec python3 "$(git rev-parse --show-toplevel)/scripts/buglog.py" check --staged + python3 "$top/scripts/buglog.py" check --staged || exit 1 fi +c_files="$(git diff --cached --name-only --diff-filter=ACMR -- '*.c' '*.h')" +if [ -n "$c_files" ]; then + # shellcheck disable=SC2086 # one path per word; repo paths have no spaces + python3 "$top/scripts/check-no-fork.py" "$top" $c_files >/dev/null || { + python3 "$top/scripts/check-no-fork.py" "$top" $c_files + exit 1 + } +fi +exit 0 diff --git a/src/meson.build b/src/meson.build index f271f9cd..62075ed5 100644 --- a/src/meson.build +++ b/src/meson.build @@ -72,6 +72,16 @@ if component != 'xcecli' subdir('frontend/apiserver') subdir('xceadmin') + # No-fork rule (.kiro/steering/tech.md, "Process creation"): fails if a + # covered component's C code calls fork()/system()/popen()/uv_spawn() + # outside a call marked xce-fork-ok. In each covered suite, so + # `xce-build.sh ` runs it too. + test('no-fork rule', + find_program('python3'), + args: [meson.project_source_root() / '..' / 'scripts' / 'check-no-fork.py', + meson.project_source_root() / '..'], + suite: ['lib', 'controller', 'scheduler', 'worker']) + configure_file( input: 'tools/appimage/entry.sh', output: 'xce', diff --git a/src/tools/xce-build.sh b/src/tools/xce-build.sh index 15841679..e2058ebd 100755 --- a/src/tools/xce-build.sh +++ b/src/tools/xce-build.sh @@ -333,7 +333,7 @@ case "$COMPONENT" in meson compile -C "$BUILD_DIR" "${MESON_COMPILE_JOB_OPTS[@]}" xcelib_test if [ "$RUN_TESTS" = 1 ]; then echo "=== Running lib tests ===" - meson test -C "$BUILD_DIR" "${MESON_TEST_JOB_OPTS[@]}" "libxce test" --print-errorlogs + meson test -C "$BUILD_DIR" "${MESON_TEST_JOB_OPTS[@]}" "libxce test" "no-fork rule" --print-errorlogs fi ;; controller) diff --git a/src/worker/CLAUDE.md b/src/worker/CLAUDE.md index d80be6de..1695bf72 100644 --- a/src/worker/CLAUDE.md +++ b/src/worker/CLAUDE.md @@ -11,7 +11,8 @@ C daemon running on each compute node. Manages job execution, hardware monitorin ## Architecture - `src/main.c` — Main daemon entry point (`int main`); links against `xcecd_srcs` from `src/xcecd.c` and friends - `src/hardware_monitor.c` — CPU/memory resource monitoring (no GPU) -- `src/process_worker_manager.c` + `src/uv_proc.c` — Job process lifecycle (fork `xcecd_worker`, spawn/exec the job command); `src/process_manager.c` is unused dead code for job spawn (its `process_manager_start_process` is never called) but is still wired up for generic create/update/destroy/reap in `xcecd.c` +- `src/uv_proc.c` — job process lifecycle inside `xcecd_worker` (pre/post-exec, spawn/exec the job command). `xcecd` starts `xcecd_worker` as a systemd unit over D-Bus (`xcecd_dbus_manager.c`); `process_manager.c`/`process_worker_manager.c` only track and reap worker pids (their fork-based start paths were removed) +- **No fork in `xcecd`**: the daemon is multi-threaded, so it never forks (no `fork`/`system`/`popen`/`uv_spawn`). `xcecd_worker` is single-threaded and may fork; each such call carries `/* xce-fork-ok: ... */`. Keep `xcecd_worker` thread-free, or move its child starts to `posix_spawn()` first. See `.kiro/steering/tech.md` "Process Creation" Links against `libxce` from `src/lib/`. diff --git a/src/worker/include/process_manager.h b/src/worker/include/process_manager.h index e228a046..2fe61ffe 100644 --- a/src/worker/include/process_manager.h +++ b/src/worker/include/process_manager.h @@ -57,15 +57,6 @@ void process_info_init(process_info_t* proc); */ process_manager_t* process_manager_create(void); -/** - * @brief start a new process - * @param manager process manager without resource limits - * @param command command with arguments - * @return 0 if success, -1 if failed - */ -int process_manager_start_process(process_manager_t* manager, - const char* command); - /** * @brief get the number of processes * @param manager process manager diff --git a/src/worker/include/process_worker_manager.h b/src/worker/include/process_worker_manager.h index 0a28b934..e6ed4acb 100644 --- a/src/worker/include/process_worker_manager.h +++ b/src/worker/include/process_worker_manager.h @@ -13,13 +13,6 @@ #include "xcecd_worker_utils.h" #include "xcelog.h" -/** - * @brief start worker without resource limits (disabled by default) - * @param command command with arguments - * @return 0 if success, -1 if failed - */ -int process_manager_start_worker(process_manager_t* manager); - /** * @brief update the process status in the list * @param manager process manager diff --git a/src/worker/src/main.c b/src/worker/src/main.c index 1edf009f..104e2332 100644 --- a/src/worker/src/main.c +++ b/src/worker/src/main.c @@ -588,21 +588,6 @@ int main(int argc, char* argv[]) init_static_array(&pids, sizeof(int)); #ifndef _WIN32 -#if 0 - pid_t pid = fork(); - if (pid < 0) { - fprintf(stderr, "First fork failed\n"); - return 1; - } - if (pid > 0) { - exit(0); - } - - if (setsid() < 0) { - fprintf(stderr, "setsid failed\n"); - return 1; - } -#endif char current_dir[1024]; getcwd(current_dir, sizeof(current_dir)); diff --git a/src/worker/src/process_manager.c b/src/worker/src/process_manager.c index 9d5f6e03..44e47bf0 100644 --- a/src/worker/src/process_manager.c +++ b/src/worker/src/process_manager.c @@ -89,205 +89,6 @@ process_manager_t* process_manager_create(void) return manager; } -int process_manager_start_process(process_manager_t* manager, - const char* command) -{ - - if (!manager || !command) { - XCECD_WORKER_LOG(XCE_LOG_ERROR, "Invalid process manager or command"); - return -1; - } - - XCECD_WORKER_LOG(XCE_LOG_DEBUG, "Starting process: %s", command); - -#ifdef _WIN32 - EnterCriticalSection(&manager->lock); -#else - pthread_mutex_lock(&manager->lock); -#endif - - // 创建新的进程信息结构 - process_info_t* new_processes - = realloc(manager->processes, - (manager->process_count + 1) * sizeof(process_info_t)); - if (!new_processes) { - XCECD_WORKER_LOG(XCE_LOG_ERROR, - "Failed to allocate memory for process info"); -#ifdef _WIN32 - LeaveCriticalSection(&manager->lock); -#else - pthread_mutex_unlock(&manager->lock); -#endif - return -1; - } - manager->processes = new_processes; - - process_info_t* proc = &manager->processes[manager->process_count]; - process_info_init(proc); - proc->command = strdup(command); - - char* jobid_str = getenv("XCE_JOBID"); - uint64_t jobid = 0; - if (jobid_str != NULL) { - jobid = atoll(jobid_str); - } - - // 创建日志文件 - char log_path[256]; - time_t now; - time(&now); - struct tm* timeinfo = localtime(&now); - if (jobid != 0) { - char idstring[128]; - - fmt_jobid(idstring, sizeof(idstring) - 1, jobid); - snprintf(log_path, sizeof(log_path), "/tmp/job_%lu_%s.out", - (unsigned long)now, idstring); - XCECD_WORKER_LOG(XCE_LOG_INFO, "Job output file path is: %s", log_path); - } else { - snprintf(log_path, sizeof(log_path), - "/tmp/job_%d_%02d%02d_%02d%02d%02d.out", (int)now, - timeinfo->tm_mon + 1, timeinfo->tm_mday, timeinfo->tm_hour, - timeinfo->tm_min, timeinfo->tm_sec); - } - proc->log_file = strdup(log_path); - - // 打开日志文件 - proc->output_fd = open(proc->log_file, O_WRONLY | O_CREAT | O_APPEND, 0644); - if (proc->output_fd == -1) { - XCECD_WORKER_LOG(XCE_LOG_ERROR, "Failed to open log file: %s", - strerror(errno)); - FREEUP(proc->log_file); - FREEUP(proc->command); -#ifdef _WIN32 - LeaveCriticalSection(&manager->lock); -#else - pthread_mutex_unlock(&manager->lock); -#endif - return -1; - } - -#ifdef _WIN32 - // Windows实现 - SECURITY_ATTRIBUTES sa - = { .nLength = sizeof(SECURITY_ATTRIBUTES), .bInheritHandle = TRUE }; - - // 创建管道 - HANDLE pipe_read, pipe_write; - if (!CreatePipe(&pipe_read, &pipe_write, &sa, 0)) { - XCECD_WORKER_LOG(XCE_LOG_ERROR, "Failed to create pipe: %lu", - GetLastError()); - close(proc->output_fd); - FREEUP(proc->log_file); - FREEUP(proc->command); - LeaveCriticalSection(&manager->lock); - return -1; - } - - // 确保读取端不被继承 - SetHandleInformation(pipe_read, HANDLE_FLAG_INHERIT, 0); - - STARTUPINFOA si = { 0 }; - si.cb = sizeof(STARTUPINFO); - si.dwFlags = STARTF_USESTDHANDLES; - si.hStdInput = GetStdHandle(STD_INPUT_HANDLE); - si.hStdOutput = pipe_write; - si.hStdError = pipe_write; - - PROCESS_INFORMATION pi; - - if (!CreateProcessA(NULL, proc->command, NULL, NULL, TRUE, CREATE_NO_WINDOW, - NULL, NULL, &si, &pi)) { - XCECD_WORKER_LOG(XCE_LOG_ERROR, "Failed to create process: %lu", - GetLastError()); - CloseHandle(pipe_read); - CloseHandle(pipe_write); - close(proc->output_fd); - FREEUP(proc->log_file); - FREEUP(proc->command); - LeaveCriticalSection(&manager->lock); - return -1; - } - - CloseHandle(pipe_write); // 关闭写端 - proc->pipe_read = pipe_read; - proc->handle = pi.hProcess; - proc->status = 1; // 运行中 - proc->monitor_active = true; - - // 创建监控线程 - proc->monitor_thread - = CreateThread(NULL, 0, output_monitor_thread, proc, 0, NULL); - if (!proc->monitor_thread) { - XCECD_WORKER_LOG(XCE_LOG_ERROR, "Failed to create monitor thread: %lu", - GetLastError()); - - CloseHandle(pipe_read); - CloseHandle(pi.hProcess); - close(proc->output_fd); - FREEUP(proc->log_file); - FREEUP(proc->command); - LeaveCriticalSection(&manager->lock); - return -1; - } - - CloseHandle(pi.hThread); -#else - // Linux实现 - pid_t pid = fork(); - if (pid == -1) { - XCECD_WORKER_LOG(XCE_LOG_ERROR, "Fork failed: %s", strerror(errno)); - - close(proc->output_fd); - FREEUP(proc->log_file); - FREEUP(proc->command); - pthread_mutex_unlock(&manager->lock); - return -1; - } - - if (pid == 0) { - // 子进程 - dup2(proc->output_fd, STDOUT_FILENO); - dup2(proc->output_fd, STDERR_FILENO); - close(proc->output_fd); - - char* args[] = { "/bin/sh", "-c", proc->command, NULL }; - execv("/bin/sh", args); - XCECD_WORKER_LOG(XCE_LOG_ERROR, "Failed to execute command: %s", - strerror(errno)); - - _exit(1); - } else { - // 父进程 - XCECD_WORKER_LOG( - XCE_LOG_DEBUG, - "Process started successfully, output redirected to %s", - proc->log_file); - - proc->handle = pid; - proc->status = 1; // 运行中 - - // 非阻塞等待子进程退出，避免僵尸进程 - XCECD_WORKER_LOG(XCE_LOG_DEBUG, "The pid of the child is: %d\n", pid); - } -#endif - - manager->process_count++; - manager->initialized = true; - -#ifdef _WIN32 - LeaveCriticalSection(&manager->lock); -#else - pthread_mutex_unlock(&manager->lock); -#endif - - XCECD_WORKER_LOG(XCE_LOG_DEBUG, "Process started successfully"); - - FREEUP(proc->log_file); - - return 0; -} - bool process_manager_update(process_manager_t* manager) { // XCECD_WORKER_LOG(XCE_LOG_DEBUG, "Updating process manager..."); diff --git a/src/worker/src/process_worker_manager.c b/src/worker/src/process_worker_manager.c index 04f50c1e..78400cf9 100644 --- a/src/worker/src/process_worker_manager.c +++ b/src/worker/src/process_worker_manager.c @@ -27,188 +27,6 @@ #include #endif -/** - * @brief start worker without resource limits (disabled by default) - * @param command command with arguments - * @return 0 if success, -1 if failed - */ -int process_manager_start_worker(process_manager_t* manager) -{ - XCECD_LOG(XCE_LOG_DEBUG, "Starting worker process"); - if (!manager) { - XCECD_LOG(XCE_LOG_ERROR, - "Invalid process manager and quit starting process."); - return -1; - } - -#ifdef _WIN32 - EnterCriticalSection(&manager->lock); -#else - pthread_mutex_lock(&manager->lock); -#endif - - process_info_t* new_processes - = realloc(manager->processes, - (manager->process_count + 1) * sizeof(process_info_t)); - if (!new_processes) { -#ifdef _WIN32 - LeaveCriticalSection(&manager->lock); -#else - pthread_mutex_unlock(&manager->lock); -#endif - return -1; - } - manager->processes = new_processes; - - process_info_t* proc = &manager->processes[manager->process_count]; - process_info_init(proc); - proc->command = strdup("xcecd_worker"); - proc->log_file = strdup("/tmp/xcecd_worker.log"); - - /* bug-1869: fixed name in /tmp, opened as root */ - proc->output_fd = xcecd_open_owned_file(proc->log_file, O_APPEND, 0644); - if (proc->output_fd < 0) - proc->output_fd = -1; - if (proc->output_fd == -1) { - XCECD_LOG(XCE_LOG_ERROR, "Failed to open log file: %s", - strerror(errno)); - FREEUP(proc->log_file); - FREEUP(proc->command); - -#ifdef _WIN32 - LeaveCriticalSection(&manager->lock); -#else - pthread_mutex_unlock(&manager->lock); -#endif - return -1; - } - -#ifdef _WIN32 - SECURITY_ATTRIBUTES sa - = { .nLength = sizeof(SECURITY_ATTRIBUTES), .bInheritHandle = TRUE }; - - HANDLE pipe_read, pipe_write; - if (!CreatePipe(&pipe_read, &pipe_write, &sa, 0)) { - XCECD_LOG(XCE_LOG_ERROR, "Failed to create pipe: %lu", GetLastError()); - close(proc->output_fd); - FREEUP(proc->log_file); - FREEUP(proc->command); - LeaveCriticalSection(&manager->lock); - return -1; - } - - SetHandleInformation(pipe_read, HANDLE_FLAG_INHERIT, 0); - - STARTUPINFOA si = { 0 }; - si.cb = sizeof(STARTUPINFO); - si.dwFlags = STARTF_USESTDHANDLES; - si.hStdInput = GetStdHandle(STD_INPUT_HANDLE); - si.hStdOutput = pipe_write; - si.hStdError = pipe_write; - - PROCESS_INFORMATION pi; - - if (!CreateProcessA(NULL, proc->command, NULL, NULL, TRUE, CREATE_NO_WINDOW, - NULL, NULL, &si, &pi)) { - XCECD_LOG(XCE_LOG_ERROR, "Failed to create process: %lu", - GetLastError()); - CloseHandle(pipe_read); - CloseHandle(pipe_write); - close(proc->output_fd); - FREEUP(proc->log_file); - FREEUP(proc->command); - LeaveCriticalSection(&manager->lock); - return -1; - } - - CloseHandle(pipe_write); - proc->pipe_read = pipe_read; - proc->handle = pi.hProcess; - proc->status = 1; - proc->monitor_active = true; - - proc->monitor_thread - = CreateThread(NULL, 0, output_monitor_thread, proc, 0, NULL); - if (!proc->monitor_thread) { - XCECD_LOG(XCE_LOG_ERROR, "Failed to create monitor thread: %lu", - GetLastError()); - CloseHandle(pipe_read); - CloseHandle(pi.hProcess); - close(proc->output_fd); - FREEUP(proc->log_file); - FREEUP(proc->command); - LeaveCriticalSection(&manager->lock); - return -1; - } - - CloseHandle(pi.hThread); -#else - pid_t pid = fork(); - if (pid == -1) { - XCECD_LOG(XCE_LOG_ERROR, "Fork failed: %s", strerror(errno)); - close(proc->output_fd); - FREEUP(proc->log_file); - FREEUP(proc->command); - pthread_mutex_unlock(&manager->lock); - return -1; - } - - if (pid == 0) { - dup2(proc->output_fd, STDOUT_FILENO); - dup2(proc->output_fd, STDERR_FILENO); - close(proc->output_fd); - - char* xce_server_dir = getenv("XCE_SERVER_DIR"); - char worker_path[1024]; - - if (xce_server_dir != NULL) { - - snprintf(worker_path, sizeof(worker_path), "%s/xcecd_worker", - xce_server_dir); - - if (access(worker_path, X_OK) != 0) { - XCECD_LOG(XCE_LOG_WARN, - "xcecd_worker not found or not executable in " - "XCE_SERVER_DIR: %s", - xce_server_dir); - } - } else { - XCECD_LOG( - XCE_LOG_WARN, - "XCE_SERVER_DIR not set, using default path for xcecd_worker"); - } - - execl(worker_path, proc->command, NULL); - - XCECD_LOG(XCE_LOG_ERROR, "Failed to execute command: %s", - strerror(errno)); - _exit(1); - } else { - /* parent */ - XCECD_LOG(XCE_LOG_DEBUG, - "Process started successfully, output redirected to %s", - proc->log_file); - proc->handle = pid; - proc->status = 1; - } - - close(proc->output_fd); - -#endif - - manager->process_count++; - manager->initialized = true; - -#ifdef _WIN32 - LeaveCriticalSection(&manager->lock); -#else - pthread_mutex_unlock(&manager->lock); -#endif - - XCECD_LOG(XCE_LOG_DEBUG, "Worker process started successfully"); - return 0; -} - /** * @brief update the process status in the list * @param manager process manager diff --git a/src/worker/src/uv_proc.c b/src/worker/src/uv_proc.c index 45b00d84..263aa274 100644 --- a/src/worker/src/uv_proc.c +++ b/src/worker/src/uv_proc.c @@ -96,6 +96,7 @@ static pid_t start_background_service(const char* command) pid_t pid; int status; + /* xce-fork-ok: xcecd_worker is single-threaded (tech.md) */ pid = fork(); if (pid < 0) { @@ -109,6 +110,7 @@ static pid_t start_background_service(const char* command) } // 2. double fork to ensure the process is not a session leader + /* xce-fork-ok: xcecd_worker is single-threaded (tech.md) */ pid_t child_pid = fork(); if (child_pid < 0) { exit(EXIT_FAILURE); @@ -153,6 +155,7 @@ static int execute_blocking_command(const char* command, char* const args[]) pid_t pid; int status; + /* xce-fork-ok: xcecd_worker is single-threaded (tech.md) */ pid = fork(); if (pid < 0) { @@ -285,6 +288,7 @@ int uv_proc_exec(uv_proc_manager_t* manager, const char* command, options.stdio = stdio; options.flags = UV_PROCESS_DETACHED; + /* xce-fork-ok: xcecd_worker is single-threaded (tech.md) */ int result = uv_spawn(manager->loop, process, &options); close(fd); diff --git a/src/worker/src/xcecd.c b/src/worker/src/xcecd.c index 8dbc7ebd..466656eb 100644 --- a/src/worker/src/xcecd.c +++ b/src/worker/src/xcecd.c @@ -33,8 +33,10 @@ static int virtual_pid = 1000; */ void xcecd_process_command(void* arg) { +#ifdef DBUS_MANAGER pid_t pid = 0; char unit_name[256]; +#endif char jobidStr[64]; fork_task_t* ctx = (fork_task_t*)arg; @@ -67,34 +69,13 @@ void xcecd_process_command(void* arg) } #ifndef DBUS_MANAGER - if (process_manager_start_worker(ctx->manager) != 0) { - XCECD_LOG(XCE_LOG_FATAL, "Failed to start worker process"); - - xcecd_job_start_error(ctx->xce_jobid); - return; - } - - /* adding worker to worker list，and update worker pid */ - pid = get_current_process_pid(ctx->manager); - if (pid != -1) { - - add_worker(&xcecd_ctx.worker_list, pid, WORKER_RUNNING, ctx->xce_jobid, - ctx->ntasks, ctx->startinfo); - - if (!xcecd_db_insert_worker(xcecd_ctx.db, pid, WORKER_RUNNING, - ctx->xce_jobid, JOB_RUNNING)) { - XCECD_LOG(XCE_LOG_ERROR, - "Failed to insert worker into database, XCECD exit..."); - dbus_stop_all_workers(&xcecd_ctx.worker_list, NULL); - xcecd_db_is_available(xcecd_ctx.db, xcecd_ctx.db_path); - } - } else { - XCECD_LOG(XCE_LOG_ERROR, "Failed to get worker pid"); - - xcecd_job_start_error(ctx->xce_jobid); - return; - } - /* send command to worker */ + /* Starting xcecd_worker by fork() from this multi-threaded daemon was + * removed (no-fork rule, .kiro/steering/tech.md): workers start only as + * systemd units over D-Bus, which every shipped xcecd build uses. */ + XCECD_LOG(XCE_LOG_FATAL, "xcecd built without DBUS_MANAGER cannot start " + "workers"); + xcecd_job_start_error(ctx->xce_jobid); + return; #else #ifndef VIRTUAL_CLUSTER diff --git a/src/worker/src/xcecd_worker.c b/src/worker/src/xcecd_worker.c index 9910e8fb..217fd3f8 100644 --- a/src/worker/src/xcecd_worker.c +++ b/src/worker/src/xcecd_worker.c @@ -1343,6 +1343,7 @@ static void do_criu_ckpt(uint64_t job_id, int kill) ckpt_dir, path, target_pid); } + /* xce-fork-ok: xcecd_worker is single-threaded (tech.md) */ int ret = system(criu_cmd); if (ret != 0) { XCECD_WORKER_LOG(XCE_LOG_ERROR, "Failed to run criu command: %s",