chore(buglog): bug-1890 fixed (8b561547); log bug-1891 (snapshot sent to the wrong standby)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

.wolf/bugs/bug-1890.md

@@ -1,8 +1,8 @@
 ---
 id: "bug-1890"
-status: "open"
+status: "fixed"
 status_updated: "2026-09-29"
-status_note: "newly logged"
+status_note: ""
 component: "lib"
 severity: "noise"
 timestamp: "2026-09-29T14:01:08Z"
@@ -11,6 +11,7 @@ occurrences: 1
 files: ["src/lib/netio.c (socket_create, socket_connect, endpoint_accept)"]
 tags: ["netio", "tcp", "latency", "ha", "snapshot"]
 related: ["bug-1871"]
+fix_commits: ["8b561547"]
 gitea_issue: 336
 gitea_synced_status: "open"
 gitea_synced_hash: "f30c67d1bb31"
@@ -29,6 +30,8 @@ gitea_synced_hash: "f30c67d1bb31"
 **Impact:** latency only, ~40 ms on the tail of a burst (snapshot/WAL batches, request-reply pairs after bulk data); no loss. `grep -rn TCP_NODELAY src/lib` finds nothing, so every netio TCP socket runs with Nagle, and a small trailing TLS record waits for the receiver's delayed ACK.
 **Found by:** soak -- bug-1871's instrumented force-snapshot runs, 2026-09-29. Suspected (not yet proven with TCP_NODELAY on): Nagle + delayed ACK.
 
+UPDATE (2026-09-29): confirmed -- with TCP_NODELAY set the ~40 ms tail gaps disappear (see Verification), so the Nagle + delayed-ACK suspicion was right.
+
 ## Environment
 - OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
 - libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2)
@@ -44,15 +47,17 @@ gitea_synced_hash: "f30c67d1bb31"
 - Evidence: /home/claude/tmp/xce-1871-fs1, /home/claude/tmp/xce-1871-fs5, /home/claude/tmp/xce-1871-fs16 (ctrl-*.log)
 
 ## Root cause
-TODO: required for fixed/partial
+netio never set TCP_NODELAY (socket_create() and socket_accept() in src/lib/netio.c set only SO_REUSEADDR/FD_CLOEXEC), so every controller/worker/scheduler/apiserver TCP connection ran with Nagle's algorithm. netio writes a message as separate TLS records (an 8-byte size header, then the body split into <=16 KB records); when a small record follows data the peer has not ACKed yet, Nagle holds it until the ACK arrives, and the receiver delays that ACK (Linux delayed ACK, ~40 ms). The tail of a burst -- the last short chunk of a snapshot, a header, a reply -- therefore arrived ~40 ms after the sender had finished writing.
 
 ## Fix
-TODO: required for fixed/partial
+src/lib/netio.c: new socket_set_nodelay() sets TCP_NODELAY; socket_create() calls it for AF_INET stream sockets (outbound endpoints from endpoint_init, and listeners) and socket_accept() for every accepted connection. Small records are sent immediately instead of waiting for the peer's delayed ACK. Best effort: a failure leaves the socket working as before.
 
 ## Verification
-- Test: TODO: name + file; RED before the fix, GREEN after
-- Suites: TODO: suite results and build profile
-- Live: TODO: soak/deploy run, or n/a and why
+- Test: test_outbound_socket_has_nodelay and test_accepted_socket_has_nodelay in src/lib/testsuite/test_netio_nodelay.c check TCP_NODELAY on an endpoint_init socket and an endpoint_accept socket: RED before the fix (`bug-1890: outbound netio socket runs with Nagle on`, same for accepted), GREEN after
+- Suites: xcelib_test 699/699 and controller 1256/1256, asan, no sanitizer errors
+- Live: xce-test-ha --force-snapshot, 16 runs (2026-09-29, xce-1890-nd1..16): 40/40 snapshot transfers complete, max inter-chunk gap 4 ms, 0 gaps >= 20 ms -- versus 5/40 transfers with a 39-41 ms gap before the fix. 15/16 PASS; the one FAIL (nd5) is bug-1891 (snapshot sent to the wrong standby), unrelated to this change and seen before it
 
 ## References
-- Related: none
+- Fix commit: 8b561547
+- Related: bug-1871 (found while instrumenting it), bug-1891
+- Gitea: #336