buglwIP - A Lightweight TCP/IP stack - Bugs: bug #29080, Remote side seems to uverrun our...


You are not allowed to post comments on this tracker with your current authentication level.

bug #29080: Remote side seems to uverrun our rcv_wnd with ooseq enabled

Submitter:  Petr Cernin <cerninp>
Submitted:  Fri 05 Mar 2010 03:42:41 PM UTC
Category:  TCP Severity:  3 - Normal
Item Group:  Crash Error Status:  Fixed
Privacy:  Public Assigned to:  goldsimon
Open/Closed:  Closed Planned Release:  None
lwIP version:  CVS Head

( Jump to the original submission )

Thu 25 Mar 2010 06:39:59 AM UTC, comment #6: 

Remote side overrunning our rcv_wnd was correctly handled in the inseq-case only, not in the ooseq-case. Reproduced, fixed and added test case.
Thanks for reporting.

Simon Goldschmidt <goldsimon>
Project Administrator
Tue 09 Mar 2010 10:31:03 AM UTC, comment #5: 

2) lwip is, you are correct.
3) I've changed TCP_MSS from 1476 to 1460. It didn't help. The result is the same. I attached wireshark trace (dupack_oos_on_mss_1470.pcap)
4) My PC is running Windows XP SP3, the trace contains HTTP request from Firefox 3.5.8.

(file #19888)

Petr Cernin <cerninp>
Mon 08 Mar 2010 04:51:55 PM UTC, comment #4: 

Thanks for the pcap files. From reading them, it really looks like the PC is overrunning lwIP's receive window, e.g. in file dupack_oos_on.pcap, packet #2351 (which seems to be the one that leads to the failed assertion):

- wireshark says "Number of bytes in flight: 35040"
- but lwIP (, is that correct?) sends a window of 32768 bytes only

this means that the PC sends more than it is allowed and we didn't handle that correctly. Could you tell us which OS and application the pc is running?

BTW: lwIP announces an MSS of 1476 with the SYN, which I doubt is correct. This should be 1460. Please check your TCP_MSS, the bug might be solved for you by changing that...

Simon Goldschmidt <goldsimon>
Project Administrator
Mon 08 Mar 2010 08:40:38 AM UTC, comment #3: 

I attached two pcap files:

(sending 2.5MB empty file to device with lwip)

dupack_oos_off.pcap - with oos processing off (ok, but dup acks)
dupack_oos_on.pcap  - with oos processing on (assert in tcp_in.c)

(file #19882, file #19883)

Petr Cernin <cerninp>
Sat 06 Mar 2010 12:00:37 PM UTC, comment #2: 

Updated summary to reflect the problem (2).

Simon Goldschmidt <goldsimon>
Project Administrator
Fri 05 Mar 2010 04:29:49 PM UTC, comment #1: 

First: Please append pcap files we can view in wireshark, not text-files that are harder to read and contain less details. Without having the complete log, I can only guess what's going wrong.

Now to 1):

The "bursts" of duplicate ACKs are OK: if one packet goes missing (one of the packets 114 to 123, I guess - details would be in the pcap file), lwIP sees the next packet having a sequence that it doesn't expect and so sends an ACK for the last received seqno to tell the other side about the lost packet. Wireshark marks this as "Dup ACK" as the data has already been ACKed before. As this is done for every received packet, more than one duplicate ACK is received.

The remote side goes into fast-retransmission after receiving two duplicate ACKs (packet #126) and eventually, lwIP has received all data and ACKs it all (packet #146: Ack= increases from 94901 to 96361).

As to 2): That looks indeed strange but I cannot say anything here without the pcap file. From the assert, it looks like the sender overran our receive window. Although that would be strange, we probably shouldn't assert...?

Simon Goldschmidt <goldsimon>
Project Administrator
Fri 05 Mar 2010 03:42:41 PM UTC, original submission:  


I've met two strange bugs inside the tcp while sending a large file (cca 4MB) from pc ( to the device with lwip (

(1) In periodic intervals tcp sends burst of duplicate acks. See attached document (packets 124, 693,1283, 1897, 2505, 3168, 3787).

(2) After upgrading to 1.3.2, lwip stack crashes on following assert during sending this file:


1278:     pcb->rcv_nxt += TCP_TCPLEN(cseg);
1279:     LWIP_ASSERT("tcp_receive: ooseq tcplen > rcv_wnd\n",
1280:                  pcb->rcv_wnd >= TCP_TCPLEN(cseg));
1281:     pcb->rcv_wnd -= TCP_TCPLEN(cseg);

It happens very often when I switch ethernet to 10Mbps and is very rare when ethernet is 100Mbps.

When I switch off out of order queue (by setting TCP_QUEUE_OOSEQ to 0) everything is ok.

Maybe there's some ralation between this two bugs.

I have tested it with  1.3.2 and also with head, the result is the same. I can provide complete wireshark trace. Text version of the trace and lwip configuration file is attached.

Please help. Now I am going to dig inside the code TCP_QUEUE_OOSEQ.

Petr Cernin <cerninp>


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #19888:  dupack_oos_on_mss_1460.zip added by cerninp (4KiB - application/octet-stream)
file #19883:  dupack_oos_off.zip added by cerninp (73KiB - application/octet-stream)
file #19882:  dupack_oos_on.zip added by cerninp (45KiB - application/octet-stream)
file #19853:  dupack.txt added by cerninp (463KiB - text/plain)
file #19854:  lwipopts.h added by cerninp (14KiB - text/plain)


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by goldsimon (Posted a comment)
  • -email is unavailable- added by cerninp (Submitted the item)
  • -email is unavailable- added by cerninp

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.




    Follow 12 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2010-03-25 goldsimon StatusNeed Info Fixed
        Assigned toNone goldsimon
        Open/ClosedOpen Closed
    2010-03-09 cerninp Attached File- Added dupack_oos_on_mss_1460.zip, #19888
    2010-03-08 cerninp Attached File- Added dupack_oos_on.zip, #19882
        Attached File- Added dupack_oos_off.zip, #19883
    2010-03-06 goldsimon SummaryRemote side seems to uverrun oru Remote side seems to uverrun our rcv_wnd with ooseq enabled
    2010-03-06 goldsimon SummaryBurst of duplicate ACKs Remote side seems to uverrun oru
    2010-03-05 goldsimon StatusNone Need Info
    2010-03-05 cerninp Attached File- Added dupack.txt, #19853
        Attached File- Added lwipopts.h, #19854
        Carbon-Copy- Added -email is unavailable-

    Back to the top

    Powered by Savane 3.9