Mon 19 Sep 2016 12:49:31 PM UTC, original submission:
With a TCP connection where no application is reading from the receiving socket, as a result of silly window avoidance a TCP receiver may reduce the receive window to zero before its receive buffer is actually full. In that state, i.e. with a zero-sized window, the receiver may continue to accept new incoming data, thus further filling up the receive buffer without opening the window. This is in fact how lwIP itself behaves in this case.
In such a scenario, the sender side may issue zero window probes to test whether the window has been reopened despite packet loss. As per the above logic, the byte sent as part of such a probe may be accepted and acknowledged by the receiver side, while the window size stays zero.
The problem is that the current lwIP implementation of zero window probes does not seem to take this case into account: it sends a byte, without remembering that it ever sent that byte. As a result, if the receiving side accepts the probe byte, the sender and receiver end up desynchronized regarding the next sequence number. The result is an ACK storm.
This case can be reproduced even on a lwIP loopback device, simply by filling up the receive buffer. As such, I ran into this problem as part of regular testing of my own code (I am using the core API). The zero-window-probe-pre-patch.pcap attachment illustrates the scenario.
I believe that the right thing to do instead is to increase snd_nxt as needed upon sending a zero window probe, as per the attached patch. The resulting behavior is then as in the attached zero-window-probe-post-patch.pcap capture.
However, I am submitting this as a bug report rather than a patch because I have no idea whether this is the whole story, if it is even a correct analysis at all. Someone with more knowledge about the entire lwIP TCP state machine will probably have to take a more thorough look at this..
For convenience, below are the relevant parts of the attached captures.
===
pre-patch
14:02:56.016650 IP localhost.49154 > localhost.49153: S 6509:6509(0) win 8192 <mss 1460>
14:02:56.016650 IP localhost.49153 > localhost.49154: S 6510:6510(0) ack 6510 win 8192 <mss 1460>
14:02:56.016650 IP localhost.49154 > localhost.49153: . ack 1 win 8192
14:02:56.033325 IP localhost.49154 > localhost.49153: P 1:1025(1024) ack 1 win 8192
14:02:56.033325 IP localhost.49154 > localhost.49153: P 1025:2049(1024) ack 1 win 8192
14:02:56.033325 IP localhost.49153 > localhost.49154: . ack 2049 win 8192
14:02:56.033325 IP localhost.49154 > localhost.49153: P 2049:3073(1024) ack 1 win 8192
14:02:56.033325 IP localhost.49154 > localhost.49153: P 3073:4097(1024) ack 1 win 8192
14:02:56.033325 IP localhost.49153 > localhost.49154: . ack 4097 win 8192
14:02:56.033325 IP localhost.49154 > localhost.49153: P 4097:5121(1024) ack 1 win 8192
14:02:56.050000 IP localhost.49154 > localhost.49153: P 5121:6145(1024) ack 1 win 8192
14:02:56.050000 IP localhost.49153 > localhost.49154: . ack 6145 win 8192
14:02:56.050000 IP localhost.49154 > localhost.49153: P 6145:7169(1024) ack 1 win 8192
14:02:56.050000 IP localhost.49154 > localhost.49153: P 7169:8193(1024) ack 1 win 8192
14:02:56.050000 IP localhost.49153 > localhost.49154: . ack 8193 win 6144
14:02:56.050000 IP localhost.49154 > localhost.49153: P 8193:9217(1024) ack 1 win 8192
14:02:56.050000 IP localhost.49154 > localhost.49153: P 9217:10241(1024) ack 1 win 8192
14:02:56.050000 IP localhost.49153 > localhost.49154: . ack 10241 win 4096
14:02:56.050000 IP localhost.49154 > localhost.49153: P 10241:11265(1024) ack 1 win 8192
14:02:56.066650 IP localhost.49154 > localhost.49153: P 11265:12289(1024) ack 1 win 8192
14:02:56.066650 IP localhost.49153 > localhost.49154: . ack 12289 win 2048
14:02:56.066650 IP localhost.49154 > localhost.49153: P 12289:13313(1024) ack 1 win 8192
14:02:56.066650 IP localhost.49154 > localhost.49153: P 13313:14337(1024) ack 1 win 8192
14:02:56.066650 IP localhost.49153 > localhost.49154: . ack 14337 win 0
14:02:57.316650 IP localhost.49154 > localhost.49153: . 14337:14338(1) ack 1 win 8192
14:02:57.566650 IP localhost.49153 > localhost.49154: . ack 14338 win 0
14:02:57.566650 IP localhost.49154 > localhost.49153: . ack 1 win 8192
14:02:57.566650 IP localhost.49153 > localhost.49154: . ack 14338 win 0
14:02:57.566650 IP localhost.49154 > localhost.49153: . ack 1 win 8192
14:02:57.566650 IP localhost.49153 > localhost.49154: . ack 14338 win 0
14:02:57.566650 IP localhost.49154 > localhost.49153: . ack 1 win 8192
14:02:57.566650 IP localhost.49153 > localhost.49154: . ack 14338 win 0
14:02:57.566650 IP localhost.49154 > localhost.49153: . ack 1 win 8192
14:02:57.566650 IP localhost.49153 > localhost.49154: . ack 14338 win 0
14:02:57.566650 IP localhost.49154 > localhost.49153: . ack 1 win 8192
..etc
===
post-patch
14:04:53.133325 IP localhost.49154 > localhost.49153: S 6509:6509(0) win 8192 <mss 1460>
14:04:53.133325 IP localhost.49153 > localhost.49154: S 6510:6510(0) ack 6510 win 8192 <mss 1460>
14:04:53.133325 IP localhost.49154 > localhost.49153: . ack 1 win 8192
14:04:53.133325 IP localhost.49154 > localhost.49153: P 1:1025(1024) ack 1 win 8192
14:04:53.133325 IP localhost.49154 > localhost.49153: P 1025:2049(1024) ack 1 win 8192
14:04:53.150000 IP localhost.49153 > localhost.49154: . ack 2049 win 8192
14:04:53.150000 IP localhost.49154 > localhost.49153: P 2049:3073(1024) ack 1 win 8192
14:04:53.150000 IP localhost.49154 > localhost.49153: P 3073:4097(1024) ack 1 win 8192
14:04:53.150000 IP localhost.49153 > localhost.49154: . ack 4097 win 8192
14:04:53.150000 IP localhost.49154 > localhost.49153: P 4097:5121(1024) ack 1 win 8192
14:04:53.150000 IP localhost.49154 > localhost.49153: P 5121:6145(1024) ack 1 win 8192
14:04:53.150000 IP localhost.49153 > localhost.49154: . ack 6145 win 8192
14:04:53.166650 IP localhost.49154 > localhost.49153: P 6145:7169(1024) ack 1 win 8192
14:04:53.166650 IP localhost.49154 > localhost.49153: P 7169:8193(1024) ack 1 win 8192
14:04:53.166650 IP localhost.49153 > localhost.49154: . ack 8193 win 6144
14:04:53.166650 IP localhost.49154 > localhost.49153: P 8193:9217(1024) ack 1 win 8192
14:04:53.166650 IP localhost.49154 > localhost.49153: P 9217:10241(1024) ack 1 win 8192
14:04:53.166650 IP localhost.49153 > localhost.49154: . ack 10241 win 4096
14:04:53.166650 IP localhost.49154 > localhost.49153: P 10241:11265(1024) ack 1 win 8192
14:04:53.183325 IP localhost.49154 > localhost.49153: P 11265:12289(1024) ack 1 win 8192
14:04:53.183325 IP localhost.49153 > localhost.49154: . ack 12289 win 2048
14:04:53.183325 IP localhost.49154 > localhost.49153: P 12289:13313(1024) ack 1 win 8192
14:04:53.183325 IP localhost.49154 > localhost.49153: P 13313:14337(1024) ack 1 win 8192
14:04:53.183325 IP localhost.49153 > localhost.49154: . ack 14337 win 0
14:04:54.416650 IP localhost.49154 > localhost.49153: . 14337:14338(1) ack 1 win 8192
14:04:54.666650 IP localhost.49153 > localhost.49154: . ack 14338 win 0
14:04:57.416650 IP localhost.49154 > localhost.49153: . 14337:14338(1) ack 1 win 8192
14:04:57.416650 IP localhost.49153 > localhost.49154: . ack 14338 win 0
|