taskCERTI - Tasks: task #6909, Varying tests programs

 
 

You are not allowed to post comments on this tracker with your current authentication level.

task #6909: Varying tests programs

Submitter:  Eric NOULARD <erk>
Submitted:  Wed 16 May 2007 11:21:06 AM UTC
   
 
Should Start On:  Tue 15 May 2007 10:00:00 PM UTC Should be Finished on:  Tue 15 May 2007 10:00:00 PM UTC
Category:  Testsuite/Autotest Priority:  5 - Normal
Status:  In Progress Privacy:  Public
Assigned to:  approx Percent Complete:  10%
Open/Closed:  Closed Planned Release:  CERTI 3.3.0
RTI Service Groups:  None

Jump to the original submission

Thu 11 Nov 2010 11:24:30 AM UTC, comment #27: 

Enhancing HLA Test Suite is a long running task.
However I think it is too "wide".
I close this one and we will open "smaller"
task each time it's needed otherwise
this current task may remain open "forever".

Eric NOULARD <erk>
Group administrator
Fri 12 Dec 2008 10:01:09 AM UTC, comment #26: 

Hi Eric,

a clean-up and a recomile did the job! Thanks.

Christian Stenzel <approx>
Group Member
Fri 12 Dec 2008 08:10:42 AM UTC, comment #25: 

Hi Christian,

Current build is broken only on Windows due to circular ref.
Petr did begin the work with a big (working) patch
I'm now working on it for the FedTime.

I've just clean+recompile+run CERTI + HLA_TestsSuite on Linux
without trouble.

You may be need to remove your old CERTI (and test suite)
 installation and your old build tree.

Because with the libraries renaming and the move
from FindCERTI.cmake to generic FindRTI.cmake
the build may be broken with finding old-now-renamed library.


Eric NOULARD <erk>
Group administrator
Fri 12 Dec 2008 07:36:33 AM UTC, comment #24: 

Hello,

at time I cannot compile the testssuite.

Could it be that the current work at libFedtime introduces some other/new linker dependencies?

Christian Stenzel <approx>
Group Member
Thu 17 Jul 2008 09:38:56 AM UTC, comment #23: 

Hello JY,

your small change works. Now I get the callback
receive Interaction without a timestamp.

I adapted the federate test_HugeAVPV in the new branch
created by Eric.

There is still one more problem:
When sending a timestamped Interaction/Update and the
sender is not time regulating then the time stamp should
be deleted.

If you start two instances of the federate
test_HugeAVPV you can see, that the sending
of timestamped messages in a non-time-regulating federate
causes the invocation of a receive/reflect callback
with a timestamp parameter.

I suppose the easiest way to solve that problem is to
check at sender side if the time regulation flag is set.
If the sending federate isn't time regulating that
information (timestamp) can be safely deleted.

Perhaps I should open a bug report so we can't forget
that problem.

Christian Stenzel <approx>
Group Member
Tue 15 Jul 2008 01:41:53 PM UTC, comment #22: 

I think,

NMessage means "Network" Message :=)
This error message comes from:
NetworkMessage.cc: line 116+


NetworkMessage::getValue(int Rank, unsigned long *length, char *Value)
    throw (RTIinternalError)
{
    // Pre-Checking
    if ((Rank < 0) || (Rank >= (int)ValueArray.size()))
        throw RTIinternalError("(getValue) Bad Rank in NMessage.");


meaning that the message is either corrupted or
badly built because one is trying to get more values
than the number stored in the message.

Eric NOULARD <erk>
Group administrator
Tue 15 Jul 2008 01:38:24 PM UTC, comment #21: 

Bonjour Rousse,

I see. Do you know more about that error log message.
I assume that this has something to do with nullmessages
(->Nmessage).


To my observation that I only get receive Interaction with a time
stamp:

The reason is that I never receive Interaction which were sended without a time stamp.

...


Christian Stenzel <approx>
Group Member
Tue 15 Jul 2008 01:36:47 PM UTC, comment #20: 

As noted by JYR on the ML there seem
to be a lot of "Bad Rank NMessage"

0 1 1 46 2 70 IntID = 3, date = 0(setValue) Bad Rank for Nmessage.
0 1 1 46 2 70 IntID = 4, date = 0(setValue) Bad Rank for Nmessage.
0 1 1 46 2 70 IntID = 5, date = 0(setValue) Bad Rank for Nmessage.
0 1 1 46 2 70 IntID = 6, date = 0(setValue) Bad Rank for Nmessage.

which means
that there must be some trouble.

Concerning timestamped or not time stamped message
NetworkMessage are time stamped implicitely if
the builder of the message calls
NetworkMessage::setDate

concerning Message
I think it is timestamped if
Message::setBoolean is called (May be JYR may confirms this).

Eric NOULARD <erk>
Group administrator
Mon 14 Jul 2008 02:34:47 PM UTC, comment #19: 

Hello

I've just commited a new version of the test federate
test_HugeAVPV in the HLA_TestsSuite folder.

Start the first (sender) federate with
test_HugeAVPV [Packet Size in Bytes].

Do that for each receiver too.

The execution of the sender federate is divided in three parts:

1)
Send attribute and parameter updates with and without
timestamp.
No time regulation nor time constrained flag is set.

The receiver is also time regulating nor time constrained
and should receive all message without a timestamp.

Indeed, this is currently not the case. I supoose that this
is a bug.

2)
Send attribute and parameter updates with and without
timestamp.
The sending federate sets his time regulating flag. The
receiving federates have no time constrained flag set.

Again, the receivers should get all messages without a time
stamp.

This is not the case!

3)
Send attribute and parameter updates with and without
timestamp.
The sending federate sets his time regulating flag. The
receiving federates set their constrained flag.

All messages with a timestamp shoul be received as messages
with a timestamp. That is the case. All messages without
time stamp should be received without timestamp. This is the
case.

Another problem I'm currently investigating is that all
parameters are always received with a time stamp. This seems
to be also true for parameters transmitted wo a timestamp.
I suppose that I've made a mistake somewhere. So I need
some further time ...



 



Christian Stenzel <approx>
Group Member
Mon 23 Jun 2008 02:21:23 PM UTC, comment #18: 

I have added dtest scripts for both
 
  test_Sync and test_TRTCCallbacks

Those scripts uses DTest v0.5
which may be found here
http://download.savannah.gnu.org/releases/tsp/dtest/dtest-0.5.tar.gz

Now if you checkout HLA_TestsSuite module
configure with CMake
after

make build
you may do
make test

and the registered dtest scripts will be launched by CTest.

(before launching make test you may need to update your
 PYTHONPATH using export PYTHONPATH=<path/to/dtest)


Eric NOULARD <erk>
Group administrator
Mon 05 May 2008 11:41:31 AM UTC, comment #17: 

Hello,

I just added two small test programs.

test_Sync tests the functioning of synchronization points.
Start it with ./test_Sync numberOfSyncPoints.

The second one test_TRTCCallbacks tests the invocation
of the callbacks timeRegulationEnabled and timeConstrainedEnabled.

This test is needed for bug #1795.

Christian Stenzel <approx>
Group Member
Sat 03 May 2008 09:29:14 PM UTC, comment #16: 

Hi Christian,

Thank you for the elementary tests,
I will use it.

I did update the CMakeLists.txt since I added
HLATestsSuite_ADD_TEST macro today.

This should make it easier to maintain the test suite.
Concerning synchro callback, do not hesitate to open
a bug.

The relativily big changes which was necessary for heterogeneous
handling may be responsible for this.
I apologize for the eventual breakage but I (we) really need
a test suite in order to capture regression.

Eric NOULARD <erk>
Group administrator
Sat 03 May 2008 04:34:22 PM UTC, comment #15: 

Hi,

just added test_HugeAVPV program in HLA_TestsSuite folder.

The intention of this test program is to transmit a specified
attribute resp. parameter value size.

Now I have among others the problem that I programmed the
little test at home using CERTI 3.2.6. All worked well. But when updating to the current cvs version I do not get some important synchronization callbacks.

Until now, I haven't time to figure out the problem, that's why
I still generated no bug report.

Christian Stenzel <approx>
Group Member
Fri 02 May 2008 06:49:23 PM UTC, comment #14: 

Christian,

I did create an "HLA_TestsSuite" module
which may be well suited for adding such test programs:
http://cvs.savannah.nongnu.org/viewvc/applications/HLA_TestsSuite/?root=certi

cvs -dXXX co applications/HLA_TestsSuite -d HLA_TestsSuite

For the moment I'd rather keep the HLA_TestsSuite directory
flat with only files in it (i.e. no subdir)
the only restriction for this is to have unique
name for source files and fed file.

Once we have a "big" set of tests we may think of
a hierarchical structure.


Eric NOULARD <erk>
Group administrator
Fri 02 May 2008 04:15:54 PM UTC, comment #13: 

After that I will go on and write a dtest script.

Christian Stenzel <approx>
Group Member
Fri 02 May 2008 04:14:03 PM UTC, comment #12: 

Hello,

here comes a test program regarding this discussion :
http://lists.nongnu.org/archive/html/certi-devel/2008-04/msg00014.html

The first federate sends uav resp si with a payload size specified
at the startup.

Other federates receive interactions resp. reflect attributes.

I cannot send more than 117 bytes. Further on, sth seems wrong
with the receiving of ts-messages. If the first federate (time-
regulating) does an uav with a ts, federates which aren't
constrained reflect attributes with a ts (same for interactions).



(file #15587)

Christian Stenzel <approx>
Group Member
Fri 07 Mar 2008 12:29:47 PM UTC, comment #11: 

Thanks for your helpful and needed hints :). Now, it works very well for me.

I saw that you are also an admin of the TSP project. How many hours have your day? :=)

It is impressive how relatively easy such a test can be implemented. I'm sure I will use it as pattern for testing
future federate implementations.

I very appreciate the work of Lionel!

I have one bad news: I suppose that I can not start with my announced CppUnit experiments before the next two weeks. Sorry about that.

Christian Stenzel <approx>
Group Member
Fri 07 Mar 2008 10:04:13 AM UTC, comment #10: 

The post of Lionel was a bit raw :=),

DTest is a sub-project of TSP:
https://savannah.nongnu.org/projects/tsp

the source are readily available from the CVS:

http://cvs.savannah.nongnu.org/viewvc/dtest/?root=tsp

you may check out the CVS moduel anonymously:

cvs -z3 -d:pserver:anonymous@cvs.savannah.nongnu.org:/sources/tsp co dtest

Then there is a README in inside the source which explain
how to installa and use dtest.

Note that this is a "work in progress" :-)
But this is already working for the billard and the HLA Tutorial.

Eric NOULARD <erk>
Group administrator
Fri 07 Mar 2008 09:21:18 AM UTC, comment #9: 

Hello Lionel,

thanks for your attached python script. Now I've a better idea
of the term "functional tests" mentioned by Eric in previous posts.

I suppose the package dtest is not a standard package provided by python, so probably you are the author of dtest. Is it possible to attach the package at the tracker so that I and anyone else can run your dtest_tutorial_hla script?


Christian Stenzel <approx>
Group Member
Tue 04 Mar 2008 10:05:19 AM UTC, comment #8: 

Okay
I will start with a little test at the end of the week.

So I can test task 6896.

I attach the code at the tracker.

Christian Stenzel <approx>
Group Member
Fri 29 Feb 2008 09:36:41 AM UTC, comment #7: 

Hi Christian,

I think there is several level of testing needed.
CppUnit as stated in its name is well-suited for
UNIT tests.

Unit-testing distributed software is not easy
if not impossible at all.

In my mind "varying tests programs" was targeting
more "integration & functionnal tests".

I've just started to work with a student
(Lionel DUROYON) which
works on "scriptable" distributed test framework
called DTest.

He is able to launch a fully automated billard test
and he will work on other CERTI example from Pierre
and may be your HLA Tutorial.

I CC him in the tracker such that we may split the
"testing work" on the unit testing on one side
and "integration testing" on the other.

I will be pleased if you volunteer for the unit testing part
(beginning with the task
 https://savannah.nongnu.org/task/?6896
 as you proposed)
and I invite Lionel to discuss with you the
integration testing part.


Eric NOULARD <erk>
Group administrator
Fri 29 Feb 2008 09:14:25 AM UTC, comment #6: 

Is someone currently working on that task?

If not, I would try to implement first tests on basis of CppUnit.

So perhaps I can test task 6896 with such a test program.

Christian Stenzel <approx>
Group Member
Thu 28 Feb 2008 09:01:57 AM UTC, comment #5: 

Here is another project called miniCppUnit:

http://www.iua.upf.es/~parumi/MiniCppUnit/

Christian Stenzel <approx>
Group Member
Thu 28 Feb 2008 08:19:54 AM UTC, comment #4: 

Hello,

as you know yesterday I had a talk in Magdeburg.

There one of the participents used CppUnit to test
their so-called PSISA implementation.

Perhaps this could be also an option for us:

http://www.evocomp.de/tutorials/tutorium_cppunit/howto_tutorial_cppunit_en.html

Christian Stenzel <approx>
Group Member
Fri 25 Jan 2008 01:17:20 PM UTC, comment #3: 

I'm coming back on this:
something fully scriptable as test programs
Could you specify your idea please?

I was think the (standard) output of a test program
may be simple to understand or parse.
May be we should adopt something like TAP:
http://en.wikipedia.org/wiki/Test_Anything_Protocol

This way we may
(and we will since I have a student that will start
 to work on this in february)
develop some scripts which will launch every needed programs
(including rtig) and may monitor standard output of each
running federate in order to get OK - NOK results.

Eric NOULARD <erk>
Group administrator
Tue 15 Jan 2008 01:16:24 PM UTC, comment #2: 

Yes I will tell a bit more about that very soon.
The basic idea to keep in mind is that the test programs
should:

A) get all their inputs from either:

A.1) command line options
A.2) stdin
A.3) file that may be specified with command line options

B) put their output to either

B.1) stdout for normal output
B.2) stderr for ERROR output
B.3) outputs should be "parsable"
     and in particular the test program should
     outputs which may be easily used to tell
     if the test is functionnally OK or not.

C) always return a meaningfull retcode from main
   that is EXIT_SUCCESS --> OK no error
   EXIT_FAILURE --> NOK there was an error

I will give some examples in a forthcoming message
but with those requirement I must be able to write a set
of scripts which will be able to launch a bunch of
tests programs and automatically tell if the test is OK or NOK.

Eric NOULARD <erk>
Group administrator
Tue 15 Jan 2008 01:04:56 PM UTC, comment #1: 

Eric, you've mentioned in task https://savannah.nongnu.org/task/?6911 something fully
scriptable as test programs. Could you specify your idea please?


Christian Stenzel <approx>
Group Member
Wed 16 May 2007 11:21:06 AM UTC, original submission:  

We should implement several tests programs or improve
the billard (CERTI test applications) in order to tests
HLA features.

This task is link to other tasks concerning the
particular feature implementation and we may start
subtasks for each test program.

here comes the TODO list:


  • Save/Restore Federation test

  We may include save/restore for the billard app.

  • Notification services test

  Billard add-on?

  • Asynchronous message delivery test


  • Time Handling test


  • Ownership transfer test


  • DDM test

  Billard option?

Eric NOULARD <erk>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #15587:  Huge_AV_PV_federate.tgz added by approx (6KiB - application/x-compressed-tar)
file #15197:  dtest_sequence_deployment_diagrams.pdf added by lduroyon (440KiB - application/pdf - dtest sequence diagrams explain how is realized a distributed test sequence, dtest deployment diagram shows how it is physically realized)
file #15196:  dtest_class_diagram.pdf added by lduroyon (121KiB - application/pdf - dtest class diagram, it explains how dtest is structured)
file #15194:  dtest_tutorial_hla.py added by lduroyon (11KiB - text/x-python - a script to test the HLA tutorial available here http://cvs.savannah.nongnu.org/viewvc/applications/HLA_Tutorial/?root=certi, an example of launching is described in top of it)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by lduroyon (Updated the item)
  • -email is unavailable- added by erk (For DTest discussion)
  • -email is unavailable- added by approx (Posted a comment)
  • -email is unavailable- added by erk (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 13 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2010-11-11 erk Open/ClosedOpen Closed
    2008-06-25 erk StatusDesign In Progress
        Planned ReleaseNone CERTI 3.3.0
    2008-06-23 erk Percent Complete0% 10%
    2008-05-02 approx Attached File- Added Huge_AV_PV_federate.tgz, #15587
    2008-03-06 lduroyon Attached File- Added dtest_sequence_deployment_diagrams.pdf, #15197
    2008-03-06 lduroyon Attached File- Added dtest_class_diagram.pdf, #15196
    2008-03-06 lduroyon Attached File- Added dtest_tutorial_hla.py, #15194
    2008-02-29 erk Carbon-Copy- Added lduroyon
    2008-02-29 approx StatusNone Design
    2007-09-20 erk Priority1 - Later 5 - Normal
        Assigned toNone approx
    2007-05-16 erk CategoryHLA 1.3 compliance Testsuite/Autotest

    Back to the top

    Powered by Savane 3.13-caa5.
    Corresponding source code