buglwIP - A Lightweight TCP/IP stack - Bugs: bug #51138, RTT measurement after...


You are not allowed to post comments on this tracker with your current authentication level.

bug #51138: RTT measurement after retransmission

Submitter:  Ambroz Bizjak <abizjak>
Submitted:  Mon 29 May 2017 05:37:41 PM UTC
Category:  TCP Severity:  3 - Normal
Item Group:  Faulty Behaviour Status:  None
Privacy:  Public Assigned to:  None
Open/Closed:  Open Planned Release:  None
lwIP version:  git head

Thu 01 Jun 2017 07:36:22 PM UTC, comment #3: 

Hmm, from reading your posts here, you might be right. I don't have an immediate answer right now though. I simply don't know which direction makes it worse (measuring RTT too slow or possibly not measuring RTT at all due to too many retransmissions)...

Simon Goldschmidt <goldsimon>
Group administrator
Mon 29 May 2017 06:02:14 PM UTC, comment #2: 

(sorry for the multiple posts)
... maybe I am wrong about that last part. Starting an RTT measurement for a new segment sent while waiting for ACK of a retransmitted segment should be fine, becayse the new segment was sent after and is expected to be received after the retransmismitted one, hence in the absence of further loss the measured RTT should be fine. So I think my original proposal is okay.

Ambroz Bizjak <abizjak>
Mon 29 May 2017 05:51:17 PM UTC, comment #1: 

... actually I see that even my proposed solution (invalidate any RTT measurement on any retransmission) does not always prevent RTT over-estimation. Especially with fast retransmission. If a new segment is sent for the first time before the retransmitted segment is acknowledged, an RTT measurement would be started which could be over-estimated due to ACK being delayed due to the lost segment.

To prevent that we would need to inhibit starting any RTT measurement until all retransmitted segments are acked, right?

Ambroz Bizjak <abizjak>
Mon 29 May 2017 05:37:41 PM UTC, original submission:  

I was looking at the RTT measurement code in lwIP (for the implementation of my own TCP/IP stack), and I think there is a problem with retransmitted segments. I'm reporting this based on reading the code so I could have missed something. Also sorry for the long message :)

Let's look at RFC 6298:
"TCP MUST use Karn's algorithm [KP87] for taking RTT samples.  That
is, RTT samples MUST NOT be made using segments that were
retransmitted (and thus for which it is ambiguous whether the reply
was for the first instance of the packet or a later instance). ..."

And at lwIP:

1) When a data segment is sent and no RTT measurement is pending, then a measurement is started (rttest is 0 --> rttest is set to current time and sequence number is noted).

2) When an ACK acknowledges the sequence number of pending RTT measurement, the measurement is completed and the related variables are updated.

3) When the retransmission timeout expires, all segments are (re-)queued, any pending RTT measurement is stopped (rttest=0;) and THEN segments are transmitted (tcp_output).

And tcp_output-->tcp_output_segment will happily start a new RTT measurement with the first retransmitted segment. When that segment is finally ACKed, the measurement will be complete and effective. Which is clearly wrong since the ACK could have been in response to the original transmission in which case the RTT would have been underestimated.

Fast retransmission (tcp_rexmit) seems to have this issue.

And now for a more general concern. I hope you have some idea what would be the best behavior.

"RTT samples MUST NOT be made using segments that were retransmitted"

But what about if pending segments before the "RTT segment" are retransmitted? A retransmission of a previous segment suggests that the ACK that acknowledges the RTT  segment will be sent later. Because the original ACK response to the RTT segment would be a duplicate ACK, and the first ACK acknowledging the RTT segment would be in response to the retransmitted previous segment.

My TCP implementation currently does roughly this (which is different from lwIP):
1) A measurement is started only when a new segment (one that bumps snd_nxt) is sent and no measurement is already active.
2) If a segment for which measurement is active is retransmitted, the measurement it stopped (and no new measurement is started).

This will never underestimate the RTO even if (2) is not done, since the start of measurement is always the first transmission of a segment. But it can and will overestimate RTO in the face of retransmissions, and (2) doesn't really help due to what I described above (loss of any segment delays the ACK of later segments).

If (2) is changed so that any retransmission invalidates any pending measurement (as lwIP tries to but currently fails to do), then I think the problem is solved. What do you think about this solution? I have a concern that it is too restrictive, that it will refuse to do any kind of RTT update until complete recovery from any kind of loss, which could be detrimental.

May also help to look at the reference [KP87]. The Karn's algorithm is:

"When an acknowledgement arrives for a packet that has been sent more than once (i.e., retransmitted at least once), ignore any round-trip measurement based on this packet, thus avoiding the retransmission ambiguity problem. In addition, the backed-off RTO for this packet is kept for the next packet. Only when it (or a succeeding packet) is acknowledged without an intervening retransmission will the RTO be recalculated from SRTT."

The first part seems to be oblivious to the possible ACK delay due to lost and retransmitted segments. The remaining part is just some logic about keeping RTO and when to update RTO, not about which RTT measurements are valid.

Ambroz Bizjak <abizjak>


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by goldsimon (Posted a comment)
  • -email is unavailable- added by abizjak (Submitted the item)

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.


    No changes have been made to this item

    Back to the top

    Powered by Savane 3.13-ee12.
    Corresponding source code