myxce/.wolf/bugs/bug-1969.md
Rongsong Shen 801d26db06
docs(buglog): record Gitea sync for bug-1969
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 14:17:02 +00:00

3.3 KiB

id status status_updated status_note component severity timestamp last_seen occurrences files tags related gitea_issue gitea_synced_status gitea_synced_hash
bug-1969 open 2026-10-10 newly logged autoscaler hang 2026-10-10T13:29:02Z 2026-10-10T13:29:02Z 1
src/autoscaler/scaler/plug_mgr.c
autoscaler
fork
threads
deadlock
400 open bcdb2d7efdf4

bug-1969: autoscaler forks from a multi-threaded process and runs unsafe code in the child (plug_mgr.c)

Description

Observed: (source reading, 2026-10-10; found while introducing the no-fork rule, .kiro/steering/tech.md "Process Creation") The autoscaler is multi-threaded (scaler.c:589/:596 start scaler_monitor and scaler_demand_handler with pthread_create). plug_mgr.c runs a provider script with fork() and, in the child, calls functions that take locks or allocate before execl():

pid = fork();                                   /* plug_mgr.c:441 */
...
} else if (pid == 0) {
    fd = open(out_file, ...);
    if (fd == -1) { ERROR("Create %s failed", out_file); exit(EXIT_FAILURE); }
    my_snprintf(cmd, sizeof(cmd), "%s/scripts/%s", ...);
    ...
    export_env(prov);                           /* setenv(): malloc */
    ret = execl("/bin/sh", "sh", cmd, "-f", in_file, (char*)NULL);
    if (ret == -1) { ERROR("execl() %s failed", cmd); exit(EXIT_FAILURE); }

If another thread holds the malloc arena or logger lock at the moment of fork(), the child deadlocks and the parent's waitpid() never returns; exit() also runs atexit handlers and flushes stdio copied from the parent.

Expected: the autoscaler starts provider scripts with posix_spawn() (output redirect via file actions, environment passed as envp), or forks only with async-signal-safe code before exec and _exit() on failure, so the no-fork rule can cover src/autoscaler. Impact: hang: an intermittent deadlocked provider-script child blocks the scaling thread in waitpid(). The autoscaler is excluded from the no-fork rule only because it is not finalized; this must be fixed before it ships. Found by: audit for the no-fork rule, 2026-10-10.

Environment

  • OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
  • libc: glibc 2.42-47 (by source reading; no binary run)
  • Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
  • Build: asan (xce.build)
  • Code: scheduler-framework @ c4fee107-dirty
  • Deployment: autoscaler with any provider plugin (found by source reading)

Reproduction

  1. Run the autoscaler with a provider while its monitor and demand threads are logging/allocating.
  2. Trigger provider actions repeatedly (scale up/down).
  3. Occasionally a provider child never reaches execl() and the autoscaler thread blocks in waitpid() (stack: child in __lll_lock_wait under malloc/logger).
  • Frequency: deterministic (code path: the unsafe child code runs on every provider call; the deadlock itself is timing-dependent, not yet observed)
  • Evidence: the code quoted in Description; no logs kept.

Root cause

TODO: required for fixed/partial

Fix

TODO: required for fixed/partial

Verification

  • Test: TODO: name + file; RED before the fix, GREEN after
  • Suites: TODO: suite results and build profile
  • Live: TODO: soak/deploy run, or n/a and why

References

  • Related: none