buglwIP - A Lightweight TCP/IP stack - Bugs: bug #20506, Initial congestion window is very...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #20506: Initial congestion window is very small

Submitter:  Vipin <vipin_dce>
Submitted:  Tue 17 Jul 2007 10:43:15 AM UTC
   
 
Category:  TCP Severity:  3 - Normal
Item Group:  Feature Request Status:  Fixed
Privacy:  Public Assigned to:  goldsimon
Open/Closed:  Closed Planned Release:  None
lwIP version:  git head

Jump to the original submission

Fri 16 Jan 2015 04:49:25 PM UTC, comment #47: 

Fixed by implementing the formula from RFC3390.

Simon Goldschmidt <goldsimon>
Group administrator
Fri 16 Jan 2015 04:31:15 PM UTC, comment #46: 

I'm reopening this because the fix does not follow RFC 3390, which says:
"The upper bound for the initial window is given more precisely in (1): min (4*MSS, max (2*MSS, 4380 bytes))"

Simon Goldschmidt <goldsimon>
Group administrator
Thu 23 Aug 2007 07:07:14 AM UTC, comment #45: 

Dear Simon,

Yes, This bug is closed as nobody has problem with this now.


> Which problem?


I have reported the problem as bug #20640 (Titled "Requests getting lost in TCP Layer"). We, Kieran and me, has observed some strange behaviour in the ethereal traces(also attached with the bug).

Can you please throw some light on that bug also?

A BUNDLE OF THANKS to you people for looking into the issues.

Vipin <vipin_dce>
Wed 22 Aug 2007 08:06:17 PM UTC, comment #44: 


> I have found some other problem in LWIP stack which i will submit
> later after confirming.


Which problem?

Anyway, since this task doesn't seem to be a problem to anyone any more, I'm closing it as fixed.

Simon Goldschmidt <goldsimon>
Group administrator
Tue 31 Jul 2007 08:55:42 AM UTC, comment #43: 

I have checked the patch and its working fine at my end. May be it can be closed.
Thanks for the patch.

I have found some other problem in LWIP stack which i will submit later after confirming.

Vipin <vipin_dce>
Wed 25 Jul 2007 07:32:15 AM UTC, comment #42: 

OK, I've checked in the patch. I'll leave this open for a while (as 'Ready For Test'), but I'd regard it as fixed.

Simon Goldschmidt <goldsimon>
Group administrator
Tue 24 Jul 2007 12:37:30 PM UTC, comment #41: 

Yes (it's the same test than comment #20, and capture files are attached)

Frédéric Bernon <fbernon>
Group Member
Tue 24 Jul 2007 10:42:02 AM UTC, comment #40: 

What do you mean with 'hack'? Splitting the packets so that 2 packets are transmitted with total amount of data < mss?

Simon Goldschmidt <goldsimon>
Group administrator
Tue 24 Jul 2007 09:42:44 AM UTC, comment #39: 

Ok, for passive open (the case of the http server), it seems to be good. With or without the hack, result is the same (~4ms), with an advantage to "without hack" solution, since the number of packets is smaller.

Now, I test an "active connection"...

Frédéric Bernon <fbernon>
Group Member
Tue 24 Jul 2007 07:54:45 AM UTC, comment #38: 


>I think in both cases, we should check if the previous cwnd is 1 (in case of state SYN_RCVD: before calling tcp_receive, because that changes the cwnd), set it to 2 mss if so and set it to 1 mss (like it is now) if it is != 1.


Oups, sorry, I have read that in a wrong way (need coffee)...

Ok, I will test your patch and give you my results...


Frédéric Bernon <fbernon>
Group Member
Tue 24 Jul 2007 07:49:48 AM UTC, comment #37: 

Sorry I didn't post a patch yet. It's a bit different to your patch (see the attached patch file): It should be 2*mss or 1*mss for both, not 2*mss or cwnd.

Also, the old value (before calling tcp_process) has to be checked in line 567 or else cwnd will never be 1.

It's all in the patch below.

(file #13448)

Simon Goldschmidt <goldsimon>
Group administrator
Tue 24 Jul 2007 07:35:44 AM UTC, comment #36: 

Can you confirm the patch I attach is what you want we test?

(file #13447)

Frédéric Bernon <fbernon>
Group Member
Tue 24 Jul 2007 06:18:50 AM UTC, comment #35: 

Yes. Thanks for testing it.
I'll check in the change these days and close the bug when I'm done.

Simon Goldschmidt <goldsimon>
Group administrator
Tue 24 Jul 2007 06:12:37 AM UTC, comment #34: 

May be the issue can be closed with the solution suggested by Simon & Kieran (if this is what all need to be done).

I just increase the window size as suggested in comment 29 and its completely working fine but to avoid comatibility issue its better if you send me the files with the changes done by you. 

Thanks very much for handling the issue so nicely and providing the solution.

Vipin <vipin_dce>
Fri 20 Jul 2007 03:01:54 PM UTC, comment #33: 

The point to change it is line 567 for passive open: ACK (probably including data) received after SYN - SYNACK in state SYN_RCVD.

For active open, this is line 151: SYN ACK received in state SYN_SENT.

I think in both cases, we should check if the previous cwnd is 1 (in case of state SYN_RCVD: before calling tcp_receive, because that changes the cwnd), set it to 2 mss if so and set it to 1 mss (like it is now) if it is != 1.

With that solution, we make sure all other cwnd = mss situations (restart...) are kept like they are, since only the initial window (when a connection is not in ESTABLISHED) is changed.

Any objections?

Simon Goldschmidt <goldsimon>
Group administrator
Fri 20 Jul 2007 09:36:38 AM UTC, comment #32: 

I'll try that, but it can take a while until I find the time.

Simon Goldschmidt <goldsimon>
Group administrator
Fri 20 Jul 2007 09:23:44 AM UTC, comment #31: 

Yes, set initial cwnd to 2 mss. 

Hmm, interesting point about the SYNs getting lost - with the change you suggest we'd set cwnd to 1 (really 1, not 1 mss) when the SYN is sent, 1 mss when retransmitting the SYN and then up to 2 mss when the connection was established.  I wonder if that's what it means?

An alternative would be to set it to 2 mss when sending the SYN, then back to 1 MSS if we retransmit (and don't change it in tcp_in.c line 515) but this might have its own problems.  For example, the SYN and SYN-ACK aren't supposed to inflate the congestion window as far as I remember, and this scheme might lead to that.  I'm also not sure what effect the change from 1 (really 1 not 1 mss) to 2 mss when sending the SYN would be - i.e. it could be 1 for a reason there!

Perhaps we could, in tcp_in.c line 515, look and see what the congestion window is currently.  If we've had a retransmission I think it will already be 1 MSS (as that's what the tcp timer sets it to on retransmission).  If we've not had a retransmission I think it will still be 1.  We can use this to decide whether to set it to 2 MSS.  Could you see if that would work?  I'm suggesting this from memory rather than looking at the code.

Kieran Mansley <kieranm>
Group Member
Fri 20 Jul 2007 09:20:51 AM UTC, comment #30: 

To conclude: The speed will be improved if i increase the Congestion window size to 'pcb->cwnd = pcb->mss*2'.

Can you tell me in which files and functions this change need to be done ? There are many lines in code which changes the pcb->cwnd.

Vipin <vipin_dce>
Fri 20 Jul 2007 09:00:07 AM UTC, comment #29: 

That's a good link, indeed. It describes our problem very accurate:

"1.  When the initial window is one segment, a receiver employing
delayed ACKs [RFC1122] is forced to wait for a timeout before
generating an ACK.  With an initial window of at least two
segments, the receiver will generate an ACK after the second data
segment arrives.  This eliminates the wait on the timeout (often
up to 200 msec, and possibly up to 500 msec [RFC1122])."

So, do we set the initial window to 2 mss instead of 1 mss?

If so, would this be in tcp_in.c line 515 (state SYN_SENT) or line 567 (state SYN_RCVD)? And how do we make sure to set it to 1 mss if a SYN or SYN ACK is lost? (RFC 3390 says "If the SYN or SYN/ACK is
lost, the initial window used by a sender after a correctly
transmitted SYN MUST be one segment consisting of MSS bytes.")

Simon Goldschmidt <goldsimon>
Group administrator
Fri 20 Jul 2007 08:34:18 AM UTC, comment #28: 

This looks rather useful:

http://www.ietf.org/rfc/rfc3390.txt

To summarise, initial window of two segments is fine, but the value must be allowed to fall to 1 MSS after loss.

Kieran Mansley <kieranm>
Group Member
Fri 20 Jul 2007 08:27:31 AM UTC, comment #27: 


> That's a partial solution


If we set it to 2 mss at the beginning, we can set it back to 2 mss later, too. There is not much difference there...

In fact, that's what I did in my local changes.

> However, if we could find something (i.e. an RFC) to justify this
> new minimum value it would certainly be an easy fix.


That's my problem right now. I tried some internet hosts and they all sent 2 packets minimum as a start. One of them happens to be a linux server (I know that because my site is running on it).

I'd love to have such a document, but compatibility is also a strong issue here. What's the point in staying with 1 mss if all the world is behaving different? If the browser was running on lwIP, too, there'd be no problem since it would directly ACK the first packet.

But I'm in no hurry to change this, I'll be searching for a more recent document that sheds some light on this.

Simon Goldschmidt <goldsimon>
Group administrator
Fri 20 Jul 2007 07:39:14 AM UTC, comment #26: 

That's a partial solution as there are other places in the stack where it will get set back to 1 mss, e.g. after a packet loss.  However, if we could find something (i.e. an RFC) to justify this new minimum value it would certainly be an easy fix.

Kieran Mansley <kieranm>
Group Member
Thu 19 Jul 2007 08:24:53 PM UTC, comment #25: 


> What disturbs me is that other stacks seem to slow-start with a
> cwnd of 2mss (instead of 1mss like lwIP does). This also better
> fits with the delayed-ack mechanism of other stacks...


When I change the initialization of 'pcb->cwnd = pcb->mss' to 'pcb->cwnd = pcb->mss*2', everything works fine!

Simon Goldschmidt <goldsimon>
Group administrator
Thu 19 Jul 2007 08:19:22 PM UTC, comment #24: 

Another limitation of the example webserver is that it lacks sending the 'Content-Length'. We should include that (actually, it must be included when generating the 'file system')!

If not, short files (< mss) have a delay also, since the client (browser) doesn't know when the file has finished and the fin (from lwIP) is only sent after an ACK from the client (and thus with the same delay of about 200ms).

Simon Goldschmidt <goldsimon>
Group administrator
Thu 19 Jul 2007 06:18:48 PM UTC, comment #23: 

Haha, I just discovered tcp_output() is called: it's called from tcp_input() after calling http_recv() (where tcp_write() is called).

So while this is kind of special (since it is called from tcp_input), it still works and the summary of this bug is wrong. But no need to change it now.

What disturbs me is that other stacks seem to slow-start with a cwnd of 2*mss (instead of 1*mss like lwIP does). This also better fits with the delayed-ack mechanism of other stacks... Does anyone know the 'most recent' document on slow start? Is it RFC 2001?

Simon Goldschmidt <goldsimon>
Group administrator
Thu 19 Jul 2007 02:38:01 PM UTC, comment #22: 

Just to give my lwipopts.h values for this test:

#define TCP_MSS          (1460)
#define TCP_SND_BUF      (4 * TCP_MSS)
#define TCP_SND_QUEUELEN (8 * (TCP_SND_BUF/TCP_MSS))
#define TCP_WND          (TCP_SND_BUF)
#define TCP_SNDLOWAT     (TCP_SND_BUF/2)

Frédéric Bernon <fbernon>
Group Member
Thu 19 Jul 2007 02:29:22 PM UTC, comment #21: 

Hmm, that is odd; the size of the application's sends shouldn't make a difference, although I agree that from the traces there it is!

We need to look into why in one case lwIP has to wait, but in the other it can send data fine.

In fact, I think I understand this specific example:
 - initial congestion window is 1 MSS (eg. 1500 bytes)
 - lwIP sends the HTTP OK packet (#6 in your traces) which uses about 200 bytes of this window
 - in the "with hack" case, the next packet in the send queue will be short (1024 bytes) because we don't do very effective merging of packets and so this fits in the rest of the available congestion window and can be sent
 - in the "without hack" case the next packet in the send queue is large (approx 1500 bytes) and so can't fit in the remaining congestion window.

The right solution to this is to do better merging and splitting of packets in the send queue.  In particular, splitting of the packet at the head.

I'd prefer to avoid having splitting in all cases as otherwise you can end up with some really bad examples (like when you have a single small packet followed by many large packets).  Perhaps a good thing to try would be only split a packet to send it if we are in slow start.

Kieran Mansley <kieranm>
Group Member
Thu 19 Jul 2007 02:01:39 PM UTC, comment #20: 


>Splitting into two is a dirty hack and shouldn't even be tried. It won't work with many stacks (who count bytes rather than packets for the delayed ack algorithm) so at best is just making your end less efficient.


You're right !!! But... (yes there is a "but") I would like to understand this little experience I did (if you can help me, once again, I'm not expert of tcp part). I think the answer is in "It slowly increases the bandwidth each round trip by increasing the number of packets it can have in flight, but it starts off very conservatively incase the network is already loaded.".

So, for application like web server, but is you have to reduce latency on a LAN (by example, your device implement a xml web services interface to drive a device or something like that), this dirty hack seems to be useful. To give a measure on my target, I do this test:

My IE7 do a GET of a GIF file. In both tests, NAGLE is disabled (note I use the socket interface). In both, I send first my Http headers, and next, the file. But, in one test, to send datas, I do multiple calls of "send()" with max. 1024 bytes (what I call, "with hack", not it's done at application level, and not in the stack), and in the second test, I call the "send" with the full size (here, the file has a 11488 bytes size). I attach the capture trace I got. 192.168.1.85 is my IE7/WINDOWS/PC, 192.168.1.246 my lwIP web server.

If I understand what I see, "with hack", the first data of the GIF file is sent before receiving the first data ACK. "without_ack," we wait (~190ms) the first data ACK (something like Vipin). After that, there is no "big" wait in both cases.

Statistics give:

- "with hack" 24 packets for the GET completion (I don't count the FIN sequence, I look between SYN in packet #1 and the HTTP end in #24), in ~5ms.

- "without hack" 18 packets (for same result) in ~196ms.

Ok, once again, I'm agree, it's really dirty !!! And I'm sure it's not working with all stacks. But... it's an interesting experience... I even think I could improve that with send the http headers in two calls, and data file in one call.

Perhaps it's a little bit "outside" the problem, but I find funny this experience (since it's only done at application level)...


(file #13397, file #13398)

Frédéric Bernon <fbernon>
Group Member
Thu 19 Jul 2007 12:45:10 PM UTC, comment #19: 


> If the other side does have a large window then turning off nagle
> might stand more chance of working


Remember the httpd we're talking about uses the RAW API under lwIP 1.2.0. There is no nagle algorithm there for that version.

Maybe tcp_output is not called often enough...

As I said, I'll try and report what I find.

Simon Goldschmidt <goldsimon>
Group administrator
Thu 19 Jul 2007 12:40:14 PM UTC, comment #18: 

Splitting into two is a dirty hack and shouldn't even be tried.  It won't work with many stacks (who count bytes rather than packets for the delayed ack algorithm) so at best is just making your end less efficient.

If the other side does have a large window then turning off nagle might stand more chance of working, but this really is just the way that TCP is supposed to work.  It slowly increases the bandwidth each round trip by increasing the number of packets it can have in flight, but it starts off very conservatively incase the network is already loaded.

I'll leave this open for now in case there turns out to be a real problem, but as I understand it and as reported at the moment this is not a bug.

Kieran Mansley <kieranm>
Group Member
Thu 19 Jul 2007 12:37:03 PM UTC, comment #17: 


> As you know, I'm not export about tcp part, but, the WINDOWS
> start with 65535 bytes for its "window size". In packet #11,
> isn't it the lwip "window size"?


Oh, seems like someone read the trace better than me :-( You're right, of course. The windows recv_wnd size is big enough... So this might be the slow start algorithm of lwIP?

Splitting into two packets might work, but it's rather unusual for a webserver, isn't it? I try that at home, only to see if it is our httpd or a misconfiguration of that target.

Simon Goldschmidt <goldsimon>
Group administrator
Thu 19 Jul 2007 12:29:47 PM UTC, comment #16: 


>why does your WINDOWS pc start with such a small receive window?


As you know, I'm not export about tcp part, but, the WINDOWS start with 65535 bytes for its "window size". In packet #11, isn't it the lwip "window size"?

To "force" the ack send, I remember there is a tips like split the data you send in two packets, or something like that :

http://lists.gnu.org/archive/html/lwip-users/2006-11/msg00008.html


Frédéric Bernon <fbernon>
Group Member
Thu 19 Jul 2007 12:26:51 PM UTC, comment #15: 

Sounds like normal behaviour for me.  Working out why the small initial window would be interesting, but even a larger one might not make any difference as at the beginning of a connection you'll be in TCP's slow-start phase, and more likely congestion window limited rather than receive window limited.

You could try turning off the nagle algorithm at the lwIP end (by setting the NODELAY socket option), but I'd be surprised if that made a difference in this case.

Kieran Mansley <kieranm>
Group Member
Thu 19 Jul 2007 12:16:17 PM UTC, comment #14: 

This really is a problem. But it's not the internet explorer that
responds slow, it's the WINDOWS tcpip stack! But this is normal
behaviour known as 'delayed ack'. WINDOWS waits for another packet
so that it doesn't send an ack for every packet but only for every
second packet. Most of the time, this works OK.

But for some reason, the initial window size is 1500 (frame 11 in
'tcp_output_http'). lwIp then responds with 1460 bytes and thus the
window has only 40 bytes left, which is not enough to send another frame (lwIP chunks the data into mss-size frames if it can). So it has to wait for a window update, which is only sent after some 200ms from the WINDOWS side...

Now the question is if this is normal (slow start algorithm in WINDOWS tcpip stack?) - if so, we should do something about it in lwIP - or not (in which case the question is: why does your WINDOWS pc start with such a small receive window?).

Kieran, what do you think?

Simon Goldschmidt <goldsimon>
Group administrator
Thu 19 Jul 2007 11:53:27 AM UTC, comment #13: 

Hello all,

1. I have analysed ethereal traces deeply and i found that when client (PC) request for 4500 bytes(img4.jpg) the following behaviour happens:
a. The server (my board with IP 192.168.2.1) sends first 1460 bytes of data in a very short time after which the client PC(192.168.2.2) ACKs the data after approx 100 -150ms.
b. After which the remaining bytes are send in a very short period of time.


My Conclusion about trace(kindly analyse):
1. My hardware is not slow.
2. Neither there is problem with lwip or http
3. Its the internet explorer which is responding very slow to the first data packet.

Is it a correct understanding of mine? If yes, then there is a problem with the explorer. How to handle this?


Thanks,

Vipin

Vipin <vipin_dce>
Thu 19 Jul 2007 08:14:18 AM UTC, comment #12: 

I conducted tests with these changes(ethereal traces are attached):

1. TCP_SND_INTERVAL changed to 100ms. No change in performance.
TCP_TMR_In100 trace attached

2.tcp_output function called (in http file) in send data function
  do {
    err = tcp_write(pcb, hs->file, len, 0);
    if (err == ERR_MEM) {
      len /= 2;
    }
  } while (err == ERR_MEM && len > 1);

  if (err == ERR_OK) {
    tcp_output(pcb);
    hs->file += len;
    hs->left -= len;
}. No change in performance. tcp_output_http is the trace.

3. TCP_SND_BUFFER size changed to 6000 bytes along with change in point 2. No effect. tcpout_Buffer trace attached.

4. The old trace named initial is also attached.

I don't know any thing about NAGLE algo but i think it is disabled in my code.

Vipin <vipin_dce>
Wed 18 Jul 2007 12:50:46 PM UTC, comment #11: 


Indeed, we had discussed some restructuring of the contrib area on CVS.

Something like      contrib/ports/{platform}/{os}/{toolchain}
And in parallel:    contrib/apps/raw-api
                    contrib/apps/netconn-api

I do think that would make things easier for new users - right now it's pretty hard to tell which port might most closely match an intended target. It may also make it easier for maintainers, if the structure was more rigorous.

One caveat - this particular example (simple httpd serving pages from ROM) isn't too bad, but others (such as an ftpd) could prove more difficult to write completely platform independently since it depends on a file system... and of course that is very differently implemented (if implemented at all) across platforms.

We'll have to think about how to handle that issue if we segregate the apps from the ports (though I still think we should).

Jim Pettinato <jim_pettinato>
Group Member
Wed 18 Jul 2007 12:37:22 PM UTC, comment #10: 


> To my point of view, it should be better to move such samples from "ports" to "contrib/apps".


That's one of the things we have on our todo-list for the contrib CVS part, if I understood Kieran right. Since we were going to move the httpd there to use it from both unix and msvc6 sim (maybe jared commits the httpd and the I can adopt msvc6), this bug should be solved in that code then.

Simon Goldschmidt <goldsimon>
Group administrator
Wed 18 Jul 2007 12:18:35 PM UTC, comment #9: 


>But in the contrib module, there are 2 httpd.c servers (in contrib/ports/msvc6 and contrib/ports/unix/proj/unisxsim/apps), and they have the same problem (those are RAW api servers). It looks like they are exactly the same as the example httpd in vipin's zip file.


To my point of view, it should be better to move such samples from "ports" to "contrib/apps". Of course, there is a problem with the initialization part which can be specific, but most of the code is "independant" of any ports (except the "leds" called in httpd.c). Perhaps we could propose a kind of "framework" for sample, where each sample have to implement a lwip_sample_main() function, and each sample_port have to call it ?

>That would probably change something but it's not the cause of the problem!


Yes, it was the "enough" side :)

>It is rather mandatory to call tcp_output (tcp_output_nagle is not available in v1.2.0) after having called tcp_write!


That's why I provide the define at the end of comment #6 :). Note the tcp_output is really not "mandatory" since it work without but it's not efficient (so saying "strongly recommanded" :) .

Vipin, can you confirm us if one of these solutions work for you ?

Frédéric Bernon <fbernon>
Group Member
Wed 18 Jul 2007 11:16:28 AM UTC, comment #8: 


> think the httpd.c is not the same as lwpi contrib


No, it's not! We currently only have a netconn httpd there.

But in the contrib module, there are 2 httpd.c servers (in contrib/ports/msvc6 and contrib/ports/unix/proj/unisxsim/apps), and they have the same problem (those are RAW api servers). It looks like they are exactly the same as the example httpd in vipin's zip file.

> If it's not enough, try to reduce TCP_TMR_INTERVAL to 100


That would probably change something but it's not the cause of the problem!

> I really like lwip code but i observed that for a http request the time taken from the starting of tcp_input function to function of http layer is 150ms which I think can be improved


Is that really true? If so, why does the first "HTTP OK" response packet come less than 3ms after the HTTP GET packet was sent by the client???

> More, you could call tcp_output to "boost" the send.
> But example, if tcp_write return ERR_OK, you could call:
> err = tcp_output_nagle(conn->pcb.tcp);


It is rather mandatory to call tcp_output (tcp_output_nagle is not available in v1.2.0) after having called tcp_write!

Simon Goldschmidt <goldsimon>
Group administrator
Wed 18 Jul 2007 11:10:02 AM UTC, comment #7: 

I think the httpd.c is not the same as lwpi contrib (/contrib/apps/httpserver), but it was in a "obsolete" port I think...


Frédéric Bernon <fbernon>
Group Member
Wed 18 Jul 2007 11:02:41 AM UTC, comment #6: 

If I understand you capture (not it will be better to attach the .cap/.acp file), the problem is the delay between the first packet of the HTTP reply and the second :

- First request: packet #19, 0.269278, and the next one : #21 0.389463, ~0.120 between. Between #21,22,24, there is only ~0.003.

- Same for the second request: packet #31 0.509113, and packet #33 0.689868, ~0.180 between. Between #33,34,36, there us only ~0.004.

I don't have check the code, but do you have disable Nagle algorithm (if you use socket api layer)? Or is your server is the httpd.c file in FreeRTOS\Demo\ARM9_STR91X_IAR\lwip\lwipWebServer?


Some remarks about your lwipopts.h. Can you change some values to test ?

#define TCP_SND_BUF 1500 for you
#define TCP_SND_BUF (4 * TCP_MSS) to test

If it's not enough, try to reduce TCP_TMR_INTERVAL to 100 (if it was possible in 1.2.0 release, Im' not sure).

>I really like lwip code but i observed that for a http request the time taken from the starting of tcp_input function to function of http layer is 150ms which I think can be improved


Your target is very slow !!! :) I feel strange in httpd.c to directly call tcp_sent AFTER your send_data. More, you could call tcp_output to "boost" the send. But example, if tcp_write return ERR_OK, you could call :

err = tcp_output_nagle(conn->pcb.tcp);

where tcp_output_nagle is this define:

/**

  • This is the Nagle algorithm: inhibit the sending of new TCP
  • segments when new outgoing data arrives from the user if any
  • previously transmitted data on the connection remains
  • unacknowledged.

 */
#define tcp_output_nagle(tpcb) ((((tpcb)->unacked == NULL) || \
                            ((tpcb)->flags & TF_NODELAY) || \
                            (((tpcb)->unsent != NULL) && ((tpcb)->unsent->next != NULL))) ? \
                                tcp_output(tpcb) : ERR_OK)

It's from the current CVS HEAD code (I think it can be "easily" use like this, but not sure at 100%)...





Frédéric Bernon <fbernon>
Group Member
Wed 18 Jul 2007 10:57:26 AM UTC, comment #5: 

Oh, and I'm changing the subject to better reflect the source of the problem.

Simon Goldschmidt <goldsimon>
Group administrator
Wed 18 Jul 2007 10:56:07 AM UTC, comment #4: 

BTW: Attaching an ethereal/wireshark capture file is worth much more than only seeing the image you attached. E.g. we can see the MAC/IP/TCP headers, maybe there are problems there?

> i observed that for a http request the time taken from the starting of tcp_input function to function of http layer is 150ms which I think can be improved.


That is definitively not normal! As it works fine on other platforms, I suggest this might be a memory limitation (problem with your memory settings in lwipopts.h)

Anyway, looking at your trace, the only real gap is between the first and second HTTP response packet for each JPG. That might be a limitation of the http server or of your memory.

What disturbs me is that there is no call to tcp_output in the whole httpd.c file. Calling tcp_write simply enqueues the data to send, without calling tcp_output, no data will be sent right away. Data will only be sent by timer functions, which are running about every 250 ms for tcp. This is the best explanation for your delay, after all.

This is not an error in lwIP itself, but in the http server!
As this http server is also included in our contrib module, I'm setting this to 'Faulty Behaviour' in module 'Contrib'.

BTW: thanks for reporting this.

Simon Goldschmidt <goldsimon>
Group administrator
Wed 18 Jul 2007 10:09:41 AM UTC, comment #3: 


1. Ethereal Capture: I am attaching the trace here. Please look at /img4.jpg and /img5.jpg request answer.

2. LWIP Release: 1.2.0

3. API Layer : The code which I am using is available on this link. http://mcu.st.com/mcdfiles/STR9_dongle_IAR_alpha_0-6.zip

4. Hardware Details:
 I am using STR9 microcontroller and running my code on STR9 eval board available from ST. STE100P configured at 100 Mbps is the phy used. Data transfer time is same whether i configured it at 10 or 100 Mbps :-(

5. Comparison Measurement: No as such but i think time is consumed in software especially in TCP layer. I checked UDP communication which was quite fast than TCP.

6. Clarification:
I really like lwip code but i observed that for a http request the time taken from the starting of tcp_input function to function of http layer is 150ms which I think can be improved.

Thanks for the reply.

Vipin <vipin_dce>
Tue 17 Jul 2007 12:25:31 PM UTC, comment #2: 

Can you also provide us your Ethereal capture, and tell us which lwIP release you use (CVS HEAD, 1.2.0...) and the API layer yyou use (socket, netconn, "raw"...)?

Frédéric Bernon <fbernon>
Group Member
Tue 17 Jul 2007 11:11:41 AM UTC, comment #1: 

Could you provide us with a little more details about your hardware and the port you are using? Do you have any comparison measurements that would show us how fast your hardware platform could be?

It is kind of hard (and I wouldn't really see it as a bug in the lwIP code, the way you posted it), to see what is the limitating factor here!

Simon Goldschmidt <goldsimon>
Group administrator
Tue 17 Jul 2007 10:43:15 AM UTC, original submission:  

Hello all,
 
I am using LWIP stack for building up my application. During development, I observed some limitation of LWIP stack. I think if it can be improved it will be much better.
 
My Application is:
 
1. To request 4000 bytes of data from my board through web browser running on the PC.
2. I am requesting this much data from browser after every 250ms.
3. The request is handled by http layer of my firmware. 
 
Observation:
1. As checked on Ethereal, every 4000 bytes of data is taking 180ms to transfer (to come to web browser).
2. Most of the time is consumed by tcp layer of lwip stack.
 
Conclusion:
1. I can't request data at a higher rate ( at max after every 200ms).
 
Problem:
1. I need to increase the speed of this communication such that i can request data after every 100ms.
 
Kindly let me know how should I amend the code so as to increase the through put.
 
There is no speed issue in my code. Kindly suggest.
 
 
Thanks & regards,
Vipin

Vipin <vipin_dce>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #13448:  tcp_in.c.patch added by goldsimon (2KiB - application/octet-stream)
file #13447:  tcp_in.patch added by fbernon (601B - text/x-patch)
file #13397:  with_hack.acp added by fbernon (14KiB - application/octet-stream)
file #13398:  without_hack.pcap added by fbernon (14KiB - application/octet-stream)
file #13391:  initial added by vipin_dce (93KiB - application/octet-stream)
file #13392:  TCP_TMR_In100 added by vipin_dce (57KiB - application/octet-stream)
file #13393:  tcp_output_http added by vipin_dce (57KiB - application/octet-stream)
file #13394:  tcpout_Buffer added by vipin_dce (260KiB - application/octet-stream)
file #13381:  Error.bmp added by vipin_dce (385KiB - image/bmp - Ethereal Image)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by kieranm (Posted a comment)
  • -email is unavailable- added by jim_pettinato (Posted a comment)
  • -email is unavailable- added by fbernon (Posted a comment)
  • -email is unavailable- added by goldsimon (Posted a comment)
  • -email is unavailable- added by vipin_dce (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 25 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2015-01-16 goldsimon StatusIn Progress Fixed
        Open/ClosedOpen Closed
    2015-01-16 goldsimon StatusFixed In Progress
        Open/ClosedClosed Open
    2007-08-22 goldsimon StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2007-07-25 goldsimon StatusIn Progress Ready For Test
    2007-07-24 goldsimon Attached File- Added tcp_in.c.patch, #13448
    2007-07-24 fbernon Attached File- Added tcp_in.patch, #13447
    2007-07-24 goldsimon StatusNone In Progress
        Assigned toNone goldsimon
        SummaryHTTP server doesn\'t call tcp_output Initial congestion window is very small
    2007-07-19 goldsimon StatusConfirmed None
    2007-07-19 fbernon Attached File- Added with_hack.acp, #13397
        Attached File- Added without_hack.pcap, #13398
    2007-07-19 vipin_dce Attached File- Added initial, #13391
        Attached File- Added TCP_TMR_In100, #13392
        Attached File- Added tcp_output_http, #13393
        Attached File- Added tcpout_Buffer, #13394
    2007-07-18 goldsimon StatusNone Confirmed
        SummaryLWIP: TCP Layer Performance HTTP server doesn't call tcp_output
    2007-07-18 fbernon CategoryContrib TCP
        Item GroupFaulty Behaviour Feature Request
        StatusConfirmed None
        SummaryHTTP server doesn\'t call tcp_output LWIP: TCP Layer Performance

    Back to the top

    Powered by Savane 3.13-02a9.
    Corresponding source code