patchlwIP - A Lightweight TCP/IP stack - Patches: patch #9579, unit: increase test timeout

 
 

You are not allowed to post comments on this tracker with your current authentication level.

patch #9579: unit: increase test timeout

Submitted by:  Joel Cunningham <jcunningham>
Submitted on:  Sat 03 Mar 2018 11:15:40 PM UTC  
 
Category:  None Priority:  5 - Normal
Status:  Wont Do Privacy:  Public
Assigned to:  Joel Cunningham <jcunningham> Open/Closed:  Closed
Planned Release:  None

Sun 04 Mar 2018 04:52:20 PM UTC, comment #5: 

XML output enabled in b28e9797394beda8dcf35f1fd9cb3f0bcbda5f91

And yes, by default check uses a 4 second timeout per test.  I found more info about it here: https://libcheck.github.io/check/doc/check_html/check_4.html#Test-Timeouts

Joel Cunningham <jcunningham>
Project MemberIn charge of this item.
Sun 04 Mar 2018 09:53:56 AM UTC, comment #4: 

Check has a timeout? I wasn't even aware of that :-) And I don't know why it is set to 4 seconds. Enabling the xml output is a good idea.

Simon Goldschmidt <goldsimon>
Project Administrator
Sun 04 Mar 2018 08:01:27 AM UTC, comment #3: 

OK for me

Dirk Ziegelmeier <dziegel>
Project Administrator
Sun 04 Mar 2018 07:47:46 AM UTC, comment #2: 

If anyone is interested, I figured out that check can write out the duration of each test to an xml file. On my MacBook Air (core 2 duo from 2012) the switch to memcpy reduced the duration from 2.335124s to 0.004089s, on my win10 box (core 2 duo from 2009) it is even slower.

I think the speed up is coming from not calling fail_unless on each byte.  fail_unless is macro'd to ck_assert_msg, which calls _mark_point when the assertion is true.

https://github.com/libcheck/check/blob/ed4f6d65024510131bbcf56eab88993986ba67ce/src/check.h.in#L484

It would be useful to always output the xml output when running lwip_unittests.  Any thoughts?

For the Unix port, I have it writing lwip_unittests.xml in the same dir as the lwip_unittest executable:

diff --git a/test/unit/lwip_unittests.c b/test/unit/lwip_unittests.c
index be74077..c33e9d6 100644
--- a/test/unit/lwip_unittests.c
+++ b/test/unit/lwip_unittests.c
@@ -86,6 +86,7 @@ int main(void)
#endif

   sr = srunner_create((suites[0])());
+  srunner_set_xml(sr, "lwip_unittests.xml");
   for(i = 1; i < num; i++) {
     srunner_add_suite(sr, ((suite_getter_fn*)suites[i])());
   }

Joel Cunningham <jcunningham>
Project MemberIn charge of this item.
Sat 03 Mar 2018 11:49:18 PM UTC, comment #1: 

Actually, I was able to speed up the unit tests by replacing the byte-by-byte compare loops with memcmp.  Now the unit tests are completing within the 4 second default.  See b1258bf8e64d1bd71bbcbd33af0e5da2262e5a78

Joel Cunningham <jcunningham>
Project MemberIn charge of this item.
Sat 03 Mar 2018 11:15:40 PM UTC, original submission:  

Any objections to increasing the unit test timeout?

From attached patch:

Increase test timeout to 10 seconds from default of 4 seconds. See: https://github.com/libcheck/check/blob/9950edf2a7ba76ad33b47d27bcd237ea1be91612/src/check.c#L36

On my Windows 10 box running WSL, the Unix port unit test fails in the pbuf tests which do binary compares on 65KB buffers:

../../../../lwip/src/../test/unit/core/test_pbuf.c:169:E:PBUF:test_pbuf_queueing_bigger_than_64k:0: (after this point) Test timeout expired

The Windows 10 box has an old core 2 duo, which maybe the cause of the slow down. Could also be WSL or Windows defender, but 10 seconds is enough to pass unit tests

Joel Cunningham <jcunningham>
Project MemberIn charge of this item.

 

Attached Files

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by goldsimon (Posted a comment)
  • -email is unavailable- added by dziegel (Posted a comment)
  • -email is unavailable- added by jcunningham (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-03-03 jcunningham StatusNone => Wont Do
        Open/ClosedOpen => Closed
    2018-03-03 jcunningham Attached File- => Added 0001-unit-increase-test-timeout.patch, #43446

    Back to the top


    Powered by Savane 3.5