bugTransport Sample Protocol - Bugs: bug #19008, Cannot subscribe to large arrays

 
 

bug #19008: Cannot subscribe to large arrays

Submitter:  None
Submitted:  Fri 09 Feb 2007 10:11:38 AM UTC
   
 
Category:  Core Severity:  3 - Normal
Status:  In Progress Privacy:  Public
Assigned to:  erk Open/Closed:  Closed
Release:  0.8.2 Fixed Release:  0.8.3
* Mandatory Fields

Add a New Comment Rich Markup
   

Tue 27 Mar 2007 12:45:43 PM UTC, comment #5: 

Hey we are at the Workshop

Eric NOULARD <erk>
Group administrator
Mon 12 Feb 2007 12:40:57 PM UTC, comment #4: 

Hi Eric,

Thanks for your late night hacking. :-)
It seems to work for us. Large arrays with 10000 elements work as expected.

Robert de Vries <rhdv>
Group Member
Sun 11 Feb 2007 09:46:37 PM UTC, comment #3: 

I commited a preliminary fix in CVS.
The bug is mainly due to old single type and scalar
handling of TSP symbols.

The buffer sizing method assumption did not hold with
todays multisized TSP sample symbol.

The current fix is robust but not as nice as
I want. I should improve it.

Nevertheless it is ready for testing.

Eric NOULARD <erk>
Group administrator
Fri 09 Feb 2007 01:25:13 PM UTC, comment #2: 

Hi Robert,

The buffer sizing is a bit intricated and has
been heavily modified when adding support for multiple
type and array (in early days TSP only knew scalar double value).

We already had some issue related to oversized buffer
https://savannah.nongnu.org/bugs/?16629
now its looks undersized :((

We need a thorough review of buffer sizing.
We should be able to "good-sized" those buffer
since we have all informations needed for sizing the
request_sample/request_sample_init.

I will try to review this next week.
Let me know if you are blocked with this issue.
And/or if you already tried some patches.

I may not be able to go and have a look for
this until the middle of next week.

If you are in a hurry/blocked you may bump the
max_nb_items to your maximum array size.

In the end you are right counting symbol array as
1 item seems wrong to me too.

More on this issue soon.

Eric NOULARD <erk>
Group administrator
Fri 09 Feb 2007 10:13:27 AM UTC, comment #1: 

Sorry that I posted this bug report as Anonymous. I am the guilty party.

Robert de Vries <rhdv>
Group Member
Fri 09 Feb 2007 10:11:38 AM UTC, original submission:  

When a clients requests the data of a large array (>150 doubles), the server will produce error messages indicating that a buffer is too small.
tsp_encoder.c##TSP_data_channel_double_encoder##70: buffer is too small

The cause is traced to a line where buffer sizes are determined depending on the number of items. An array appears to count for 1 item (this seems wrong).
The buffer size is determined using the macro TSP_DATA_STREAM_MAX_BUFFER_SIZE used in TSP_data_sender_create() in tsp_data_sender.c
The definition of this macro looks strange to me too as it defines the buffer size to be max_nb_items sizeof(double) 10 + header size (=1024) (if max_nb_items == 1, this is 1104 bytes)
If I want to send an array of 150 doubles I need about 1200 bytes buffer space. This obviously will not fit in the allocated buffer and thus an error message is generated.

The source of the problem is located in the calculation of the number of items (in TSP_group_algo_allocate_group_table() in tsp_group_algo.c)

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by erk (Posted a comment)
  • -email is unavailable- added by rhdv (Posted a comment)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2007-04-27 erk Open/ClosedOpen Closed
        Fixed ReleaseNextRelease 0.8.3
    2007-02-11 erk StatusNone In Progress
        Assigned toNone erk
        Fixed ReleaseUnknown NextRelease

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code