chore(buglog): bug-1871 fixed (754d5bcf); log bug-1890 (Nagle tail latency)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
754d5bcf3e
commit
715c08f03b
3 changed files with 125 additions and 21 deletions
55
.wolf/bugs/bug-1890.md
Normal file
55
.wolf/bugs/bug-1890.md
Normal file
|
|
@ -0,0 +1,55 @@
|
|||
---
|
||||
id: "bug-1890"
|
||||
status: "open"
|
||||
status_updated: "2026-09-29"
|
||||
status_note: "newly logged"
|
||||
component: "lib"
|
||||
severity: "noise"
|
||||
timestamp: "2026-09-29T14:01:08Z"
|
||||
last_seen: "2026-09-29T14:01:08Z"
|
||||
occurrences: 1
|
||||
files: ["src/lib/netio.c (socket_create, socket_connect, endpoint_accept)"]
|
||||
tags: ["netio", "tcp", "latency", "ha", "snapshot"]
|
||||
related: ["bug-1871"]
|
||||
---
|
||||
|
||||
# bug-1890: netio sockets leave Nagle on: last TLS records of a message burst arrive ~40 ms late
|
||||
|
||||
## Description
|
||||
**Observed:** a 9-chunk HA snapshot transfer is fully written by the primary within 1 ms, but the standby reads the last chunks ~40 ms later. Primary trace (xce-1871-fs1, ctrl-1) shows the final send drained at 13:20:33.967 (`bug1871 upd ... 0x5->0x1 outq_empty=1 fsm_out=5`); the standby logs:
|
||||
```
|
||||
[2026-09-29 13:20:33,969] ... ha_handle_snapshot_chunk: chunk crc32 checkpoint_lsn=114 chunk_index=6 ...
|
||||
[2026-09-29 13:20:34,010] ... ha_handle_snapshot_chunk: chunk crc32 checkpoint_lsn=114 chunk_index=7 ...
|
||||
```
|
||||
|
||||
**Expected:** on loopback the whole burst arrives within a few ms (most runs: max inter-chunk gap 1-4 ms).
|
||||
**Impact:** latency only, ~40 ms on the tail of a burst (snapshot/WAL batches, request-reply pairs after bulk data); no loss. `grep -rn TCP_NODELAY src/lib` finds nothing, so every netio TCP socket runs with Nagle, and a small trailing TLS record waits for the receiver's delayed ACK.
|
||||
**Found by:** soak -- bug-1871's instrumented force-snapshot runs, 2026-09-29. Suspected (not yet proven with TCP_NODELAY on): Nagle + delayed ACK.
|
||||
|
||||
## Environment
|
||||
- OS: no /etc/os-release; Docker container; NixOS kernel; kernel Linux 6.18.33 x86_64
|
||||
- libc: glibc 2.42-47 (interpreter /nix/store/wb6rhpznjfczwlwx23zmdrrw74bayxw4-glibc-2.42-47/lib/ld-linux-x86-64.so.2)
|
||||
- Toolchain: gcc (GCC) 15.2.0, meson 1.9.1
|
||||
- Build: asan (src/xce.build.bug1871, with temporary netio TRACE instrumentation)
|
||||
- Code: scheduler-framework @ 3444cf16
|
||||
- Deployment: local 3-node HA controller cluster (xce-test-ha --force-snapshot), localhost:9991-9993, 32 CPUs
|
||||
|
||||
## Reproduction
|
||||
1. `nix develop ./src --command bash .claude/skills/xce-test-ha/run-ha-soak.sh --force-snapshot <dir>`
|
||||
2. On the rejoined standby's log, compute the gap between consecutive `ha_handle_snapshot_chunk: chunk crc32` lines of one transfer; a ~40 ms gap (while the primary finished sending 1 ms earlier) is the symptom.
|
||||
- Frequency: 5/40 transfers had a 39-41 ms gap, 2026-09-29 (xce-1871-fs1..fs16)
|
||||
- Evidence: /home/claude/tmp/xce-1871-fs1, /home/claude/tmp/xce-1871-fs5, /home/claude/tmp/xce-1871-fs16 (ctrl-*.log)
|
||||
|
||||
## Root cause
|
||||
TODO: required for fixed/partial
|
||||
|
||||
## Fix
|
||||
TODO: required for fixed/partial
|
||||
|
||||
## Verification
|
||||
- Test: TODO: name + file; RED before the fix, GREEN after
|
||||
- Suites: TODO: suite results and build profile
|
||||
- Live: TODO: soak/deploy run, or n/a and why
|
||||
|
||||
## References
|
||||
- Related: none
|
||||
Loading…
Add table
Add a link
Reference in a new issue