(sorry for the multiple posts)
... maybe I am wrong about that last part. Starting an RTT measurement for a new segment sent while waiting for ACK of a retransmitted segment should be fine, becayse the new segment was sent after and is expected to be received after the retransmismitted one, hence in the absence of further loss the measured RTT should be fine. So I think my original proposal is okay.

Hi,
I was looking at the RTT measurement code in lwIP (for the implementation of my own TCP/IP stack), and I think there is a problem with retransmitted segments. I'm reporting this based on reading the code so I could have missed something. Also sorry for the long message :)

Let's look at RFC 6298:
"TCP MUST use Karn's algorithm [KP87] for taking RTT samples.  That
is, RTT samples MUST NOT be made using segments that were
retransmitted (and thus for which it is ambiguous whether the reply
was for the first instance of the packet or a later instance). ..."

And at lwIP:

1) When a data segment is sent and no RTT measurement is pending, then a measurement is started (rttest is 0 --> rttest is set to current time and sequence number is noted).
http://git.savannah.gnu.org/cgit/lwip.git/tree/src/core/tcp_out.c?id=356da76cc8f2f6cea98005a17c287679615ce588#n1276

2) When an ACK acknowledges the sequence number of pending RTT measurement, the measurement is completed and the related variables are updated.
http://git.savannah.gnu.org/cgit/lwip.git/tree/src/core/tcp_in.c?id=356da76cc8f2f6cea98005a17c287679615ce588#n1292

3) When the retransmission timeout expires, all segments are (re-)queued, any pending RTT measurement is stopped (rttest=0;) and THEN segments are transmitted (tcp_output).
http://git.savannah.gnu.org/cgit/lwip.git/tree/src/core/tcp_out.c?id=356da76cc8f2f6cea98005a17c287679615ce588#n1454

And tcp_output-->tcp_output_segment will happily start a new RTT measurement with the first retransmitted segment. When that segment is finally ACKed, the measurement will be complete and effective. Which is clearly wrong since the ACK could have been in response to the original transmission in which case the RTT would have been underestimated.

Fast retransmission (tcp_rexmit) seems to have this issue.

And now for a more general concern. I hope you have some idea what would be the best behavior.

"RTT samples MUST NOT be made using segments that were retransmitted"

But what about if pending segments before the "RTT segment" are retransmitted? A retransmission of a previous segment suggests that the ACK that acknowledges the RTT  segment will be sent later. Because the original ACK response to the RTT segment would be a duplicate ACK, and the first ACK acknowledging the RTT segment would be in response to the retransmitted previous segment.

My TCP implementation currently does roughly this (which is different from lwIP):
1) A measurement is started only when a new segment (one that bumps snd_nxt) is sent and no measurement is already active.
2) If a segment for which measurement is active is retransmitted, the measurement it stopped (and no new measurement is started).

This will never underestimate the RTO even if (2) is not done, since the start of measurement is always the first transmission of a segment. But it can and will overestimate RTO in the face of retransmissions, and (2) doesn't really help due to what I described above (loss of any segment delays the ACK of later segments).

If (2) is changed so that any retransmission invalidates any pending measurement (as lwIP tries to but currently fails to do), then I think the problem is solved. What do you think about this solution? I have a concern that it is too restrictive, that it will refuse to do any kind of RTT update until complete recovery from any kind of loss, which could be detrimental.

May also help to look at the reference [KP87]. The Karn's algorithm is:

"When an acknowledgement arrives for a packet that has been sent more than once (i.e., retransmitted at least once), ignore any round-trip measurement based on this packet, thus avoiding the retransmission ambiguity problem. In addition, the backed-off RTO for this packet is kept for the next packet. Only when it (or a succeeding packet) is acknowledged without an intervening retransmission will the RTO be recalculated from SRTT."

The first part seems to be oblivious to the possible ACK delay due to lost and retransmitted segments. The remaining part is just some logic about keeping RTO and when to update RTO, not about which RTT measurements are valid.