Mon 08 Dec 2014 02:33:17 PM UTC, original submission:
We ran across a fairly esoteric issue with the LwIP socket implementation.
The issue stems from using a TCP socket as a server. If the client sends data to the server and is done, the client sends a "FIN" with the last packet. As with all network stacks, in LwIP this results in select() notifying the application of data on the socket and then a EOF on the server side draining the socket. This is the correct behavior. In this situation you see this in the debug:
State: ESTABLISHED
tcp_receive: pcb->rttest 0 rtseq 6537 ackno 6538
tcp_receive: received FIN.
State: CLOSE_WAIT
However, in a very specifically crafted case where the packet right before the FIN came from the client was dropped, a TCP retransmission must occur. The FIN packet was received and queued. Then the retransmitted packet is received, and the FIN is dequeued. You see this in the debug log:
State: ESTABLISHED
tcp_receive: pcb->rttest 0 rtseq 657 ackno 6538
tcp_receive: dequeued FIN.
State: CLOSE_WAIT
In this situation, the socket layer will notify select() and the server app will read the socket. In this situation, the last bytes of the transfer are delivered to the application, but not the EOF. The server side never receives a direct notification of the EOF.
In the sockets.c file its this function: lwip_recvfrom(). A FIN results in a NULL msg sent to the netconn.
This results in this call returning ERR_CLSD:
if (netconn_type(sock->conn) == NETCONN_TCP) {
err = netconn_recv_tcp_pbuf(sock->conn, (struct pbuf **)&buf);
You see this in the log:
lwip_recvfrom: netconn_recv err=-12, netbuf=0
However, then below:
if (err != ERR_OK) {
if (off > 0) {
...
If off is greater than 0 then the leftover data is sent to the application and returned. The knowledge of the error (in this case CLOSED) is lost, and select() is never rewoken on the socket. The workaround I have come up with is if this happens, grab a lock and increment the rcvevent for the socket so that select() will wake up the thread one more time and the application will recall lwip_recvfrom() and this time it will grab the EOF:
if (err != ERR_OK) {
if (off > 0) {
/* update receive window */
netconn_recved(sock->conn, (u32_t)off);
/* already received data, return that */
sock_set_errno(sock, 0);
// workaround begin
SYS_ARCH_PROTECT(lev);
sock->rcvevent++; /* signal one more receive event for NULL */
SYS_ARCH_UNPROTECT(lev);
// workaround end
return off;
}
Again, this has to be a specific dropped packet - we ran across this because of a MAC layer issue causing that particular packet to be dropped. It appears that if any other packet in the transmission is dropped (including the last packet) than there are no issues.
|