chore(wolf): checkpoint buglog.json (1036 bugs)

.wolf/buglog.json

@@ -18997,6 +18997,31 @@
       ],
       "occurrences": 1,
       "last_seen": "2026-08-25T00:00:00.000Z"
+    },
+    {
+      "id": "bug-1695",
+      "timestamp": "2026-08-25T15:05:00Z",
+      "error_message": "Found via the mandatory live HA chaotic soak (xce-audit-invariants controller campaign, Step 4) -- NOT caused by any of the 9 fixes that campaign made (bug-1684/1687/1689/1690/1691/1692/1693/1694), all of which are in src/controller/{ha,persist,job,cluster,sched,worker}.c; this bug is in myapp.c, untouched by any of them. 10-round --chaotic soak: every single round after the first shows 'NO new primary within window (possible election stall)'; ctrl-1's own log shows, on EVERY restart after the first boot, 'myapp_run: leader already known after startup delay (voted_for=ctrl-1, voted_term=1, raft_term=1, primary_fd=-1) -- skipping own election' -- primary_fd=-1 proves there is no live connection to any primary, yet the node skips holding its own election anyway. ctrl-2/ctrl-3 are simultaneously stuck in a permanent ~3s retry loop ('cluster_init_phase2: cluster_config not loaded yet', rc=-22, the bug-1477 self-rescheduling retry firing forever without ever succeeding). The cluster never elects a new, live, functioning primary after the very first election of the cluster's lifetime.",
+      "file": "src/controller/myapp.c",
+      "root_cause": "myapp_run()'s post-stagger 'is a leader already known' bootstrap check (myapp.c ~361-384) has two independent fallback conditions that can each unilaterally set leader_known=1: `c.ha.raft.term > 0` and `c.ha.primary_ctrl_fd >= 0`. The comment's stated intent (lines 345-353) is to catch a narrow race -- 'iomgr was NOT running during the nap ... but the winner's ctrlhello has already established the control channel' -- i.e. genuine NEW raft activity processed during THIS boot's startup-stagger nanosleep. But `ha_restore_term()` (bug-1684's neighbor, unmodified by that fix) restores ha->term from ha-term.bin BEFORE this check ever runs, as part of normal boot -- so `raft.term > 0` is trivially true on EVERY restart after the cluster's first-ever election in its entire lifetime, regardless of whether any NEW raft traffic was processed this boot at all. The check cannot distinguish 'term advanced because I just received a real, fresh vote/heartbeat during this nap' (genuine liveness evidence) from 'term is nonzero because I loaded an old value off disk at startup' (tells us nothing about whether ANY leader is currently reachable). Consequence: after the cluster's first election, EVERY node, on EVERY subsequent restart, unconditionally takes the goto skip_election fast path and never calls ha_start_election_async again -- so no node ever re-asserts leadership through a real raft election, and the cluster permanently loses the ability to elect a NEW primary after any restart, however far in the past the original election was. This is a related-but-distinct gap from bug-1477 (which fixed a stranded node never RETRYING cluster_init_phase2 after one failed attempt -- that retry loop is confirmed still firing correctly here, every ~3s, exactly as bug-1477 designed -- but the retry can never succeed because the underlying cause, no election is ever held to produce a real live primary to push config from, was never addressed).",
+      "fix": "Not yet fixed -- investigating a minimal, precise fix: capture ha.term's value once, immediately after ha_restore_term() completes at boot (before the startup-stagger nap), and compare `raft.term > term_at_boot` in the fallback check instead of `raft.term > 0` -- only a term that has genuinely ADVANCED during this boot cycle (real new raft traffic processed) should count as evidence a leader is known; a merely-nonzero restored value should not. This is the most surgical fix consistent with the check's own stated intent, without touching the primary_ctrl_fd fallback or the voted_for_name/vft==cur first check (both of which look correct as written).",
+      "tags": [
+        "controller",
+        "ha",
+        "raft",
+        "election",
+        "election-safety",
+        "bootstrap",
+        "myapp",
+        "live-deployment",
+        "found-via-soak"
+      ],
+      "related_bugs": [
+        "bug-1477",
+        "bug-1684"
+      ],
+      "occurrences": 1,
+      "last_seen": "2026-08-25T15:05:00Z"
     }
   ]
 }