tasklwIP - A Lightweight TCP/IP stack - Tasks: task #7865, implement non-blocking SEND...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

task #7865: implement non-blocking SEND operation (socket)

Submitter:  piero <piero74>
Submitted:  Thu 20 Mar 2008 02:08:02 PM UTC
Votes: 8
 
Category:  None Should Start On:  Thu 20 Mar 2008 12:00:00 AM UTC
Should be Finished on:  Thu 20 Mar 2008 12:00:00 AM UTC Priority:  5 - Normal
Status:  Done Privacy:  Public
Assigned to:  goldsimon Percent Complete:  100%
Open/Closed:  Closed Planned Release:  1.4.0
Effort:  0.00

Jump to the original submission

Sun 21 Feb 2010 11:37:24 AM UTC, comment #8: 

Done since noone seems to care about the tiny limitation mentioned at the end of the previous comment.

Simon Goldschmidt <goldsimon>
Group administrator
Sat 13 Feb 2010 05:12:30 PM UTC, comment #7: 

As described in last comment, I've added nonblocking write (for netconn and sockets). Once such a nonblocking write failed, sent_tcp and poll_tcp will check if there's enough buffer and mark the socket writable again.

Also, lwip_send() checks if the data would ever fit into the buffers and returns EMSGSIZE if not - I'm not sure what's the correct error here...

There's one lwIP-related problem: when a nonblocking write fails and poll_tcp marks the socket writable again, the next call could fail again if 'size > TCP_SNDLOWAT' (because TCP_SNDLOWAT is used when marking the socket writable). I don't know how other stacks solve this, but for us, the next call to write would return EAGAIN again.

Simon Goldschmidt <goldsimon>
Group administrator
Sun 24 Jan 2010 10:03:50 AM UTC, comment #6: 

Thinking about this (especially in combination with the patch for bug #28605), it might be enough to return EAGAIN and repeat the check for available buffer space (that we do in sent_tcp) in poll_tcp (depending on if we have returned EAGAIN before).

I might then still happen that we return EAGAIN again (e.g. we're still out of a memp pool), but that's OK I guess.

Simon Goldschmidt <goldsimon>
Group administrator
Fri 09 Oct 2009 08:39:56 PM UTC, comment #5: 

Being one of the supporters of a 'new' (redesigned) sockets implementation, I'd implement this only half: return EAGAIN (or wouldblock or whatever) if tcp_write() returns != ERR_OK but not the part waking up an application task from select() when memory comes available.

I think that should be good enough: the 'proper' solution would be too big for lwIP, and anyway, we would have to either try periodically or monitor the memp pools or the heap to know when memory is available...

If we do it like that, the solution for this is a simple code change in api_msg.c:do_writemore() (for RAW or UDP, the stack doesn't wait, anyway), only a flag forwarding for MSG_DONWAIT or O_NONBLOCK from socket to netconn.

Simon Goldschmidt <goldsimon>
Group administrator
Tue 25 Mar 2008 01:22:38 PM UTC, comment #4: 

This one makes me a little nervous.  Doing the job properly, including being able to wake up select when space becomes available to write, will involve a fair bit of extra code.  I wonder if this would be best left for the "new" sockets API that Frederic was working on (Task6994) or simply saying that if users need more advanced API features such as non-blocking operation they'll need to use the raw API instead.

Kieran Mansley <kieranm>
Group Member
Thu 20 Mar 2008 03:00:28 PM UTC, comment #3: 

2008/3/20, Pettinato, Jim <Jim.Pettinato@fmcti.com>:


>    Do you really need to implement another queue, or could you just have it
>    return an error eg ERR_WOULD_BLOCK if there is no longer room on the
>    existing TCP send queue, and let the task retry later?



First, i think that this  could be very usefull, and can avoid task caller blocks.

After, lwip should implement:
- manage O_NONBLOCK in the sockets layer,  for tx.
- manage flag MSG_DONTWAIT for tx
- lwip_select() triggered by tx event.

Now i'm using only a select function, with socket listener for incoming connection and rx data from connected socket. It is really very efficient, and could be close to perfect if application can manage send operation in select too.

piero <piero74>
Thu 20 Mar 2008 02:54:30 PM UTC, comment #2: 

Jim Pettinato wrote:

> Do you really need to implement another queue, or could you just
> have it return an error eg ERR_WOULD_BLOCK if there is no longer
> room on the existing TCP send queue, and let the task retry
> later?


Well that's a tricky thing I should probably have highlighted. A full send queue is not the only thing that can cause tcp_enqueue to return ERR_MEM. It can also happen if you run out of MEMP_TCP_SEGs, pbufs of the needed sort, having a resulting tcp queue which ends up too long once this write is added, and more. Have a look.

Superficially you might think that that's ok if you just try the tcp_write and look for ERR_MEM, but how does an application then know when more resources become available? ie. an app waiting in lwip_select() has to be blocked (so tcp_write returning ERR_MEM would imply :
      API_EVENT(conn, NETCONN_EVT_SENDMINUS, len);
So what would be calling:
      API_EVENT(conn, NETCONN_EVT_SENDPLUS, len);
in that case since the lack of resource may be unconnected to the send data enqueued for this tcp pcb.


Jonathan Larmour <jifl>
Group Member
Thu 20 Mar 2008 02:25:01 PM UTC, comment #1: 

As background, Piero is primarily concerned with the BSD sockets interface. lwIP does implement O_NONBLOCK in the sockets layer, but only for rx. Not for tx.

It's easy to make the first write non-blocking just by always waking up the caller, but to allow subsequent ones to be non-blocking before the first write's data is completely sent would require being able to store a list of pending writes. I don't think the implementation would be meaningful until the code does that.

Obviously you need to check against tcp_sndbuf before deciding whether to return EAGAIN immediately or append to that list. I'm not sure whether there should also be a check against the send queue length (TCP_SND_QUEUELEN). If there was, there would have to be some consideration of when to trigger:
      API_EVENT(conn, NETCONN_EVT_SENDPLUS, len);
for the use of lwip_select().

As a result, to keep things simple I think it's probably best just to accept the data for writing anyway. At least for now. Other opinions welcome. (Especially since I at least won't be able to work on this for some time).

Jonathan Larmour <jifl>
Group Member
Thu 20 Mar 2008 02:08:02 PM UTC, original submission:  


lwip should manage in socket implementation non-blocking IO for both rx and tx operations.

It should implement MSG_DONTWAIT flag for send function, non-blocking behaviour for send operation if the socket is set as non-blocking, and return with error if after a  blocking call of send function, the peer closes the connection.


piero <piero74>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by goldsimon (Posted a comment)
  • -email is unavailable- added by sapym (Voted in favor of this item)
  • -email is unavailable- added by kieranm (Posted a comment)
  • -email is unavailable- added by jifl (Posted a comment)
  • -email is unavailable- added by piero74 (Submitted the item)
  •  

    There are 8 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2010-02-21 goldsimon StatusReady For Test Done
        Percent Complete90% 100%
        Open/ClosedOpen Closed
    2010-02-13 goldsimon Planned ReleaseNone 1.4.0
    2010-02-13 goldsimon StatusNone Ready For Test
        Percent Complete0% 90%
        Assigned toNone goldsimon
    2008-05-05 sapym Carbon-Copy- Added sapym

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code