buglwIP - A Lightweight TCP/IP stack - Bugs: bug #27377, "op_completed" semaphore...


You are not allowed to post comments on this tracker with your current authentication level.

bug #27377: "op_completed" semaphore mechanism crash error

Submitted by:  hanhui <hanhui03>
Submitted on:  Fri 04 Sep 2009 02:38:06 AM UTC  
Category:  TCP Severity:  3 - Normal
Item Group:  Crash Error Status:  Wont Fix
Privacy:  Public Assigned to:  Simon Goldschmidt <goldsimon>
Open/Closed:  Closed Planned Release:  None
lwIP version:  CVS Head

Mon 05 Oct 2009 08:00:18 PM UTC, comment #5: 

I guess what hanhui initially posted is a (potential) bug report that op_completed might be posted by err_tcp and not checked by another task afterwards.

However, since op_completed is only posted in states NETCONN_WRITE, NETCONN_CLOSE and NETCONN_CONNECT, there is another task waiting and the errors he saw are (supposedly) only related to the fact the he used multiple threads on one netconn, which is not supported (and will not be unless that API is redesigned).

Simon Goldschmidt <goldsimon>
Project AdministratorIn charge of this item.
Fri 04 Sep 2009 07:44:31 AM UTC, comment #4: 

There is several posts about the multithreadinng question. You could find these previous chats in bugs/tasks sections.

By example:

The current status for the feature request is "wont do".

But feel free to propose a full patch file with your solution...

Good luck

Frédéric Bernon <fbernon>
Project Member
Fri 04 Sep 2009 07:25:27 AM UTC, comment #3: 

I recommend a solution: all thread which will call socket, has it's own synchronic semaphore. and tcpip thread use this semaphore synchronic user thread.

sys_arch_timeouts() get thread own timeout.
sys_arch_semaphore() get thread own semaphore.

hanhui <hanhui03>
Fri 04 Sep 2009 07:22:08 AM UTC, comment #2: 


Just for information, lwIP got several known limitations about multithreading: you cannot read/write a same socket by two threads in the same time.

Take a look to http://lwip.wikia.com/wiki/LwIP_and_multithreading


Frédéric Bernon <fbernon>
Project Member
Fri 04 Sep 2009 07:08:34 AM UTC, comment #1: 

another BUG found: user thread and tcpip thread synchronization mechanism.

for example: here are 3 thread: recv_thread() send_thread() tcpip_thread(). recv_thread() has the highest priority. tcpip_thread() has the middle priority, send_thread() has the lowest priority of all.

one time, send_thread call tcpip_apimsg() to add a do_send() job to tcpip_thread, after mbox post, tcpip_thread run immediately. now send_thread do not wait for op_completed semaphore. at this time send_thread does not wait for op_completed semaphore, then tcpip_thread execute do_send() job... at this time recv_thread() wake up, preempt all other thread, and is call tcpip_apimsg() to add a do_recv() job to tcpip_thread, mbox post do not cause a preempt, and then recv_thread wait for op_completed semaphore. then tcpip_thread resume execute do_send()... when do_send() finish, it will signal op_completed semaphore. NOW! not send_thread get is signal, but recv_thread.

hanhui <hanhui03>
Fri 04 Sep 2009 02:38:06 AM UTC, original submission:  

lwip application layer use tcpip_apimsg()[or other similar functions] send a message to tcpip thread. for example: netconn_XXX() usually call tcpip_apimsg() to execute do_XXX() function.

tcpip_apimsg() first call sys_mbox_post() send a message, and then wait tcpip thread execute this message. if user thread priority higher than tcpip thread, sys_mbox_post() do not make a scheduling. and then user thread wait a conn->op_completed semaphore block itself.

if before user thread call sys_mbox_post(), conn->op_completed semaphore already been signaled. user thread will not have any block. and tcpip_apimsg() will return immediately. but the message which tcpip_apimsg() send, is a LOCAL VARIABLE, the message must be damaged, and then tcpip thread fetch this message pointer. the message body is DEAD. Crash!

The above situation really happened! because not all conn->op_completed semaphore signals are activated by do_XXX() function. for example: net interface have a receive event, then tcpip thread are going to execute. if this packet is a tcp packet, then tcp_input() will be called. in tcp_input() function will call TCP_EVEN_SEND() or TCP_EVENT_RECV(). and then will call send_tcp() or poll_tcp(). then do_writemore() or do_close_internal() will be called. and then conn->op_completed semaphore will be signaled. if above case are happening in tcpip_apimsg() are going to send a message. Crash!

I tested using the two thread: one recv() and another send(), using one socket. these two thread priority higher than tcpip thread. use of large amounts of data test. and than crashed. When the system crashes, I see tcpip_apimsg() do not make any scheduling. in other words sys_mbox_post() and sys_arch_sem_wait() hasn't been blocked, conn->op_completed semaphore already been signaled.

but I using one thread first recv() and then send(), do not crash.

tcpip_apimsg(struct api_msg *apimsg)
  struct tcpip_msg msg;


  if (mbox != SYS_MBOX_NULL) {
    msg.type = TCPIP_MSG_API;
    msg.msg.apimsg = apimsg;
    // tcp_input() ->  TCP_EVENT_SENT()  ->  (pcb)->sent()  -> 
    // sent_tcp()  ->  do_writemore()  ->  sys_arch_sem_signal() !!!! error !!
    sys_mbox_post(mbox, &msg);
    sys_arch_sem_wait(apimsg->msg.conn->op_completed, 0);
    return ERR_OK;
  return ERR_VAL;

hanhui <hanhui03>


No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by goldsimon (Posted a comment)
  • -email is unavailable- added by fbernon (Posted a comment)
  • -email is unavailable- added by hanhui03 (Submitted the item)

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.




    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2009-10-05 goldsimon Open/ClosedOpen => Closed
        StatusNone => Wont Fix
        Assigned toNone => goldsimon

    Back to the top

    Powered by Savane 3.5