buglwIP - A Lightweight TCP/IP stack - Bugs: bug #33128, Socket send() and sendto() do not...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #33128: Socket send() and sendto() do not use bound interface.

Submitter:  Ken MacKay <kmackay>
Submitted:  Tue 19 Apr 2011 08:33:53 PM UTC
   
 
Category:  sockets/netconn Severity:  3 - Normal
Item Group:  Faulty Behaviour Status:  Invalid
Privacy:  Public Assigned to:  None
Open/Closed:  Closed Planned Release:  None
lwIP version:  CVS Head

Jump to the original submission

Mon 25 Apr 2011 06:22:42 PM UTC, comment #8: 

Didn't I essentially post this bug back in Oct. 2010?

https://savannah.nongnu.org/bugs/index.php?31367

Bill Auerbach <billauerbach>
Mon 25 Apr 2011 06:04:43 PM UTC, comment #7: 


>Now that's what I call a bug: the code selects an interface to
>send to and some lines below rejects its own choice :-) We should
>fix that...


I think that test (line 560) should be deleted and I've done so on my lwIP versions built for speed.  It does save some slight runtime overhead.

lwIP supports the Window's (and other OSes?) function to specify a route so sending to any subnet from any interface can be done.  If one calls etharp_add_static_entry then any netif should be able to send to the static IP address regardless of the netif source address.  I've had to do this in Windows to talk to a device on a different subnet.  I know it's MAC address and IP address due to discovery but I must talk to (i.e. configure) it by unicast UDP.

Bill Auerbach <billauerbach>
Thu 21 Apr 2011 05:25:37 AM UTC, comment #6: 


> It seems like the correct solution would be to set the default
> interface based on whether or not the landline is available.


Exactly.

> I guess lwIP doesn't have a SO_BINDTODEVICE ioctl right now; I
> guess that is sort of what we implemented (for the sending
> half). Maybe that would be useful?


Well, it might be useful and there have already been one or two people asking about it. However, few people really need it. You mainly need something like this to implement a "discovery" service using sockets. And if you want to implement this with lwIP, you can still use the UDP raw API (udp_sendto_if()), so I guess not implementing BINDTODEVICE would fall under the favour-small-size-over-features rule...

Simon Goldschmidt <goldsimon>
Group administrator
Wed 20 Apr 2011 09:33:58 PM UTC, comment #5: 

OK. You are correct that we have a bad routing setup issue.

In our case we actually have two interfaces that can access the Internet (in addition to the one that only talks to things on the local network); one goes over a landline (may not be connected) and the other goes over a satellite connection (which is always available, but has very high latency/cost). So when we send a packet to the internet over the landline interface, we want to make sure that it doesn't actually get sent over the satellite connection. Similarly, if the landline is not present, we don't want to try to send over it.

It seems like the correct solution would be to set the default interface based on whether or not the landline is available.

I guess lwIP doesn't have a SO_BINDTODEVICE ioctl right now; I guess that is sort of what we implemented (for the sending half). Maybe that would be useful?

Ken MacKay <kmackay>
Wed 20 Apr 2011 06:53:23 PM UTC, comment #4: 


> The reason is that there is a check to see if the packet's
> source address is equal to the netif's address (around line 560 in udp.c


Now that's what I call a bug: the code selects an interface to send to and some lines below rejects its own choice :-) We should fix that...

> This would be bad because the 10.0.0.254 gateway is not
> connected to the internet


This is a common routing problem: you just have the default-netif wrong. If you have 2 netifs where one should be used for unknown (internet-) traffic, than that should be the default netif, not the other one. You get that problem under windows or linux, too. On windows, for example, you have to adjust the interface metric to adjust the routing table. And yes, you do get these strange wireshark logs where you see the source address of interface A while logging interface B.

To me that's not an lwIP problem but simply a routing issue. However, due to lwIP's limited routing "table", routing is more limited than on other stacks.

Simon Goldschmidt <goldsimon>
Group administrator
Wed 20 Apr 2011 04:38:32 PM UTC, comment #3: 

Sorry, I don't have a reference, so possibly it's not supposed to work.

The destination is not on the same subnet as any of the device interfaces, so the selected interface must be used so that the correct gateway will be used.

So for example, suppose we have 2 interfaces on the device: 10.0.0.2 and 10.0.1.2. Both have a netmask of 255.255.255.0. We open a UDP socket bound to each interface.

From the socket bound to 10.0.0.2, we want to send to 10.0.0.24. This works fine because ip_route() finds the correct interface.

From the socket bound to 10.0.1.2, we want to send to 74.125.127.147. We would expect that the packet would be sent to the gateway for the 10.0.1.2 interface (10.0.1.254 in this case), and the gateway would then route the packet appropriately.

The main issue with the way it currently works is that the packet actually never gets sent at all if the incorrect interface is selected. The reason is that there is a check to see if the packet's source address is equal to the netif's address (around line 560 in udp.c, if (!ip_addr_cmp(&(pcb->local_ip), &(netif->ip_addr))) { ... ). This means that whenever the interface selected is not the same one as what the socket was bound to, the packet will not get sent. This happens for all the packets destined for 74.125.127.147 over the 10.0.1.2 interface. I think this is a general issue?

The current behaviour is to use the default interface (since the destination IP is not on any interface's subnet), and then send to whatever the gateway is for the default interface. So it would (for example) choose the 10.0.0.2 interface to send over, and then the packet would get sent to that interface's gateway (10.0.0.254). This would be bad because the 10.0.0.254 gateway is not connected to the internet (for example). This could just be an issue for us because of our crazy network setup (since only one gateway can actually route to the destination).

Ken MacKay <kmackay>
Wed 20 Apr 2011 07:52:24 AM UTC, comment #2: 


> Something that doesn't quite make sense to me is that for lwIP
> to be selecting the wrong interface, you must have more than one
> interface that can reach the destination. I.e. overlapping IP
> subnets. This isn't something that lwIP aspires to support.


That's what makes me think the current behaviour is correct: why should routing consider the source address (wouldn't that make it stateful, which it isn't?):
a) If there's more than one route than that's currently not supported by lwIP (or we would need a more complex routing algorithm).
b) If there's only one route to the (remote) address in question, then why would the local socket be trying to send to that address (if it's not reachable via its "interface").

My impression of this is that this is not a problem for TCP since a TCP socket bound to a specific interface simply would not get a connection into a running state.

Also, the solution with the "bound_if" sounds very much like SO_BINDTODEVICE to me!

Simon Goldschmidt <goldsimon>
Group administrator
Wed 20 Apr 2011 07:36:58 AM UTC, comment #1: 

For completeness, do you have a reference for the behaviour you require?  I expect it is correct, but having a reference would save me testing and looking for it, so make the fix much quicker.

Something that doesn't quite make sense to me is that for lwIP to be selecting the wrong interface, you must have more than one interface that can reach the destination.  I.e. overlapping IP subnets.  This isn't something that lwIP aspires to support.

If we do want to do this, rather than caching the bound interface (which would be more appropriate for SO_BINDTODEVICE) I think we just need to give preference to the interface that matches the source address as well as the destination address when routing.

Kieran Mansley <kieranm>
Group Member
Tue 19 Apr 2011 08:33:53 PM UTC, original submission:  

This is an issue we have noted with UDP sockets, but I believe it applies to all socket types. We have multiple IP interfaces on our device, and we open one socket per interface (and bind the socket to the interface). When we call sendto() on a socket, we expect that the packet will be sent over the interface that we bound the socket to. However, since udp_sendto() (called by the sockets API) uses ip_route() to find an interface, the packet will always be sent either over the first netif with a matching address/netmask, or the default netif if there is no match.

We have fixed this by adding a 'struct netif *bound_if' to the struct netconn. This is set to NULL by default in netconn_alloc(). Then, in do_bind(), we added code to iterate through all netifs and set the bound_if member to the netif with the matching IP address (if any). Finally, in do_send(), we check to see if msg->conn->bound_if is non-NULL; if so, we call udp_sendto_if()/udp_sendto_if_chksum() instead.

You could do a similar thing for TCP sockets but there was no convenient interface-specific functions to call (and TCP is not required for our application). I guess you would need to use the bound netif on connect, and then also on write.

Ken MacKay <kmackay>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by billauerbach (Posted a comment)
  • -email is unavailable- added by goldsimon (Posted a comment)
  • -email is unavailable- added by kieranm (Posted a comment)
  • -email is unavailable- added by kmackay (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2011-04-21 kieranm StatusNone Invalid
        Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code