bugTransport Sample Protocol - Bugs: bug #19008, Cannot subscribe to large arrays

 
 

bug #19008: Cannot subscribe to large arrays

Submitted by:  None
Submitted on:  Fri 09 Feb 2007 10:11:38 AM UTC  
 
Category: CoreSeverity: 3 - Normal
Status: In ProgressPrivacy: Public
Assigned to: Eric NOULARD <erk>Open/Closed: Closed
Release: 0.8.2Fixed Release: 0.8.3

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Tue 27 Mar 2007 12:45:43 PM UTC, comment #5:

Hey we are at the Workshop

Eric NOULARD <erk>
Project AdministratorIn charge of this item.
Mon 12 Feb 2007 12:40:57 PM UTC, comment #4:

Hi Eric,

Thanks for your late night hacking. :-)
It seems to work for us. Large arrays with 10000 elements work as expected.

Robert de Vries <rhdv>
Project Member
Sun 11 Feb 2007 09:46:37 PM UTC, comment #3:

I commited a preliminary fix in CVS.
The bug is mainly due to old single type and scalar
handling of TSP symbols.

The buffer sizing method assumption did not hold with
todays multisized TSP sample symbol.

The current fix is robust but not as nice as
I want. I should improve it.

Nevertheless it is ready for testing.

Eric NOULARD <erk>
Project AdministratorIn charge of this item.
Fri 09 Feb 2007 01:25:13 PM UTC, comment #2:

Hi Robert,

The buffer sizing is a bit intricated and has
been heavily modified when adding support for multiple
type and array (in early days TSP only knew scalar double value).

We already had some issue related to oversized buffer
https://savannah.nongnu.org/bugs/?16629
now its looks undersized :((

We need a thorough review of buffer sizing.
We should be able to "good-sized" those buffer
since we have all informations needed for sizing the
request_sample/request_sample_init.

I will try to review this next week.
Let me know if you are blocked with this issue.
And/or if you already tried some patches.

I may not be able to go and have a look for
this until the middle of next week.

If you are in a hurry/blocked you may bump the
max_nb_items to your maximum array size.

In the end you are right counting symbol array as
1 item seems wrong to me too.

More on this issue soon.

Eric NOULARD <erk>
Project AdministratorIn charge of this item.
Fri 09 Feb 2007 10:13:27 AM UTC, comment #1:

Sorry that I posted this bug report as Anonymous. I am the guilty party.

Robert de Vries <rhdv>
Project Member
Fri 09 Feb 2007 10:11:38 AM UTC, original submission:

When a clients requests the data of a large array (>150 doubles), the server will produce error messages indicating that a buffer is too small.
tsp_encoder.c##TSP_data_channel_double_encoder##70: buffer is too small

The cause is traced to a line where buffer sizes are determined depending on the number of items. An array appears to count for 1 item (this seems wrong).
The buffer size is determined using the macro TSP_DATA_STREAM_MAX_BUFFER_SIZE used in TSP_data_sender_create() in tsp_data_sender.c
The definition of this macro looks strange to me too as it defines the buffer size to be max_nb_items * sizeof(double) * 10 + header size (=1024) (if max_nb_items == 1, this is 1104 bytes)
If I want to send an array of 150 doubles I need about 1200 bytes buffer space. This obviously will not fit in the allocated buffer and thus an error message is generated.

The source of the problem is located in the calculation of the number of items (in TSP_group_algo_allocate_group_table() in tsp_group_algo.c)

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by erk (Posted a comment)
  • -unavailable- added by rhdv (Posted a comment)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 5 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Fri 27 Apr 2007 11:11:35 AM UTCerkOpen/ClosedOpen=>Closed
      Fixed ReleaseNextRelease=>0.8.3
    Sun 11 Feb 2007 09:46:37 PM UTCerkStatusNone=>In Progress
      Assigned toNone=>erk
      Fixed ReleaseUnknown=>NextRelease

    Back to the top


    Powered by Savane 3.1-cleanup