bugmldonkey, a multi-networks file-sharing client - Bugs: bug #17202, Webinterace:overnet stat gives:...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #17202: Webinterace:overnet stat gives: "Unknown type for content: Exception Failure("Buffer.add: cannot grow buffer")"

Submitted by:  Fabiano <fabtar>
Submitted on:  Tue 25 Jul 2006 04:07:51 PM UTC  
 
Category:  HTTP interface Severity:  3 - Normal
Item Group:  Program malfunction Status:  In Progress
Assigned to:  None Open/Closed:  Closed
Release:  Release:  2.7.7
Operating System:  Linux Binaries Origin:  CVS / Self compiled
CPU type:  Intel x86

( Jump to the original submission)

Fri 01 Sep 2006 10:29:57 AM UTC, comment #19: 

After 4 days there is no buffer error :-)
I have experiemced slowdown in accessing kad/ovr stats but , no errors..
Plus, about the other problem, the kas has re-collected miracously 60 boot nodes :-) , it is never happened first.

It' a good behaviour as I can see.
I have 500000 ate hthe moment.

Fabiano <fabtar>
Tue 29 Aug 2006 09:38:02 PM UTC, comment #18: 

>Do you use webinterface frequently?


Yes

The buffer error is not appeared after more than 2 days.this is good.
Also I have noted that the stats's speed raises when kad and ovr starts working badly.
It could be only a perception issue but I was really sure about the  better ovr/kad behaviour. :-)

About your patch.. yes.. things are improved surely, this patch is OK.. A greaer value 200000 .. could be considered as fine too.
Currently I'm using 500000.
Cheers

Fabiano <fabtar>
Tue 29 Aug 2006 06:02:34 PM UTC, comment #17: 

MLDonkey has a known bug regarding Overnet, it looses nodes rather quickly without regaining them, this patch has nothing to do with it.

This patch is only aimed at solving the ov_stats bug but, as Pango already pointed out, its not a real solution.

Either the webinterface procedures have to be changed to stream
content instead of putting everything into a buffer (difficult)
or some HTML code can be removed from ov_stats output to have
more content displayed.

Fabtar wrote:
"I think that increased buffer has solved performace issues too on my machine."

Do you use webinterface frequently?

Pango wrote:
"The only thing it changes is that it decreases the number of reallocations needed, so it may be easier on memory management, but that's all."

If the patch improves webinterface-speed for you it should be
applied, but to really solve the bug different steps are needed.

spiralvoice <spiralvoice>
Project Administrator
Tue 29 Aug 2006 05:14:10 PM UTC, comment #16: 

I have rised buffers(allocation block?) to 500000 but I cannot obtain more than 2 days with a functional kad/ovr.
My final report is:
Increasing buffers(allocation) from 10000 to 100000 has revived my poor&died kad/ovr permitting 2 days of good working.
By risng buffers from 100000 to 500000  I haven't gained more.
I 'll use the 100000 value (I manually patch and compile) and I'll close mldonkey after 3 days uptime.

Fabiano <fabtar>
Mon 28 Aug 2006 09:31:02 AM UTC, comment #15: 

thanks pango for replying.
I have to point the fact that this change has solved my mldonkey's issues with kad and overnet.
Perhaps there si something which triggers that problem and having bigger buffers helps in avoiding the triggerin' stuff.(My hypotesis).

I'm not opposing your opinion(I agree on all points) but I have reported impressions based on observations and results.

Fabiano <fabtar>
Mon 28 Aug 2006 07:54:39 AM UTC, comment #14: 

I doubt that changing the parameter given to Buffer.create can change anything.
It is a "size hint" (the initial size of the Buffer.t), but Buffers are automatically resized whenever they become full. And then the upper limit is always the same : 16M chars, because of the Ocaml string used to implement them.

The only thing it changes is that it decreases the number of reallocations needed, so it may be easier on memory management, but that's all.

pango <mlpango>
Project Member
Sun 27 Aug 2006 08:58:52 PM UTC, comment #13: 

In this precise moment , things have started collapsing.
Boot nodes have disappeared(there are 1 boot nodes per kad and ovr) and  buckets are somehow empty (there are few lines).
I think that kad and ovr 'll stop working in few minutes.
For my personal point a view is a sucess.. raising the buffers has permitted  to my mldonkey to operate correctly for 2 days.
I think that in few minutes the buffer overflow 'll came up again to sign the complete collapse. :-(
Personally I 'll try rsing the buffers to 500000 (this 'll hopefully grant 10 days of good behaviour :-) ).
But I think there is something with those buffers which fills without release and then nothing could work and nodes are lost... boot nodes, buckets, nothing

Fabiano <fabtar>
Sun 27 Aug 2006 10:30:23 AM UTC, comment #12: 

Compiled last CVS with your proposed/mini-changes-patches.
Great!, I hope this is not a placebo-effect but Kad and overnet have started immediately with no problems, boot nodes are been  collected correctly and  both boot peers list and buckets and stats are FULLY POPULATED.
There are high source/search hits on ovr/kad networks.
I think that increased buffer has solved performace issues too on my machine.
Session is one day old and there is no buffer problem (at the moment).
I'm confident this 'll delay the problme or solve the issue :-D .
I'll report back later to confirm (hopefully) this good behaviour.

Fabiano <fabtar>
Fri 18 Aug 2006 01:05:56 PM UTC, comment #11: 

Ok, I have verified the issue.. the "Unknown type for content: Exception Failure("Buffer.add: cannot grow buffer")"
Original Submission: " is here again :)
There is no correlation between pango's patch and this error, the only thing triggerin the error is the amount of activity.

I have to apply your patch to look deeper

Fabiano <fabtar>
Thu 17 Aug 2006 11:19:23 AM UTC, comment #10: 

My last CVS core without any patch( I havenot installed the pango's patch equivalent to 5310, nor your suggested patch cause my inability to apply) is running fine, I have no buffer problems.
I'll try with the LAST CVS tomorrow with 5310, and I'll let you informed if the problem is triggered again.

Ovr is has worked fine for a while but currently no boot nodes are collected :-O
Kad is working fine but also at the moement zero boot nodes are colected too.
It seems that kad and ovr too much easily get out of nodes.
Cheers

Fabiano <fabtar>
Wed 16 Aug 2006 10:49:25 AM UTC, comment #9: 

@spiral
Please  explain me how to correctly apply your patch.

About ovr/kad, I have 3 kad boot nodes after 3 days of uptime, and 0 ovr oot nodes.
The good thing is that ovr plus kad are effectivley working and searching :-) .
So, I'm happily waiting for the bug to be triggered again.

If this bug is not triggered this time there could be a chance it is related to pango's patch.
Thanks

Fabiano <fabtar>
Sun 13 Aug 2006 08:11:17 AM UTC, comment #8: 

I have tried again a unpatched cvs.
I have again the problem with overnet and kademlia wich losts progressively boot nodes witout effectively doing searches.
I think this is a local problem of mine.I'll try a statically compiled core of yours on monday.
As far as kad and ovr doesn't boot properly and search suceccfully (I have 0 searches and zero hits in statistics,) the buffer's bug could not be triggered.

I fear somewhere in my network there is an udp problem, but the other applications using udp work perfectly.
Perhaps the boot peers has to be mantained longer with higher timeouts.. ??? I have also lots of download recently on my box, so I have great activity.

on monday I'll also update the system a bit.

thanks for your patience

Fabiano <fabtar>
Sat 12 Aug 2006 09:05:26 PM UTC, comment #7: 

I have tryed aaplying your patch at cvs in different ways, I get always malformed patch or similar errors.. could you please explain me the right way to apply this patch please?
( i have copied in a file and then using the patch command as usual)

I have noted that the other patches has the diff command too, for example:
diff -u -r1.27 commonClient.ml

does your patch lacks this? Have I to write this commnad manually?I have always applied patches in automatic way.Can you give me some hints?

Fabiano <fabtar>
Sat 12 Aug 2006 08:39:02 PM UTC, comment #6: 

The old core is not collecting the boot nodes properly :-(.
I'll try compiling this core now,
I have simply to point the fact i'm using too the pango's patch located here(I have pointed in the first post):

ftp://ftp.berlios.de/pub/mldonkey/pango/no_auto_query_view_files.patch

I find this patch  necessary, a must cause this doesn't abuse the friend's feature by asking the friend's continuosly.
I'll compile a new fresh CVS-bi-patched core immediately and if this fail again ..I'll try one without pango's patch.

Thanks a lot for interesting in this bug.I'll report soon.

Fabiano <fabtar>
Sat 12 Aug 2006 06:03:47 PM UTC, comment #5: 

I do not have a clue what triggers this bug, but we can test a bit.
Please try this patch:

--- ././src/daemon/driver/driverControlers.ml   2006-08-10 19:41:19.000000000 +0200
+++ ././src/daemon/driver/driverControlers.ml   2006-08-12 20:02:26.000000000 +0200
@@ -1510,7 +1510,7 @@
     else s

 let http_options = {
-    conn_buf = Buffer.create 10000;
+    conn_buf = Buffer.create 100000;
     conn_output = HTML;
     conn_sortvd = NotSorted;
     conn_filter = (fun _ -> ());

spiralvoice <spiralvoice>
Project Administrator
Mon 07 Aug 2006 05:07:49 PM UTC, comment #4: 

I have also the impression that meanwhile the bug is triggered  kad and overnet stop working properly (Overnet peers counters in ed2k stats doesn't rise), and currently, rebooting after new long session, I have kad and overet without overnet boot peers.
I'll update the status tomorrow..I'll also try compiling a new CVS and post again about.

Fabiano <fabtar>
Mon 07 Aug 2006 05:01:01 PM UTC, comment #3: 

I have verified again this behaviour.
This time , this bug has affected the kad stats (the overnet is not affected this time).
The same error about buffers.. I suppose when the log's limit is reached the errors began.

Fabiano <fabtar>
Thu 27 Jul 2006 10:19:49 PM UTC, comment #2: 

I have verified that the memory consuption is a bit higher than normal (about 77 megs) . (but it looks not so abnormal)
My mlnet has about a 15 days long session at the moment without a real crash.
the exception is still there...

Fabiano <fabtar>
Thu 27 Jul 2006 09:57:34 PM UTC, comment #1: 

All what that means is that output grow past 16M characters, the maximum allowed size for ocaml's strings.

Maybe the exception could be caught, and partial result displayed ?

At least that would help understanding what and why display became that large.

pango <mlpango>
Project Member
Tue 25 Jul 2006 04:07:51 PM UTC, original submission:  

I have this error when using overnet stat from webinterface:

Code:
Unknown type for content :
Exception Failure("Buffer.add: cannot grow buffer")

and overnet stats are not visualized..I'm quite sure this thing has not appeared days ago with a younger session.
Mldonkey keeps working, the error is not critical and seems an interface-only issue.

at the moment:

Quote:
Buildinfo
Version: MLNet Multi-Network p2p client version 2.7.7.CVS
SCM version: 2006-07-11 12:20:38
Networks: Global Shares Gnutella FileTP BitTorrent Donkey (SUI)
Ocaml version: 3.08.3 - C compiler version: 3.3.5 - C++ compiler version: 3.3.5
Build on: Linux i686 2.6.8-2-386 (little endian) with glibc 2.3.2
Configure args: --enable-multinet --disable-fasttrack --disable-gnutella2 --disable-gui
Features: threads zlib-1.2.2 bzip2-1.0.2 gd(jpg/png-1.2.Cool iconv magic(active) no-check-bounds

Runinfo
User: admin (Warning: empty Password) - uptime: 10d 17h 24m 15s
Enabled nets: Donkey Overnet Kademlia BitTorrent FileTP
Server usage: enabled
Geoip: enabled, GeoLite data created by MaxMind, available from http://maxmind.com/
IP blocking: local: 0 ranges - web: 96007 ranges
Libmagic: file-type recognition database present
System info: Linux pigrigw.tana.net 2.6.8-2-386 #1 Tue Aug 16 12:46:35 UTC 2005 i686
language: EN - locale: ISO-8859-1 - UTC offset: +0200
max_string_length: 16777211 - word_size: 32 - max_array_length: 4194303
max file descriptors: 1024 - max useable file size: 2^63-1 bits (do the maths ;-p)

Self compiled cvs + pango's patch for browse share requests (I hope to see this patch in CVS in the future).

Cheers

Fabiano <fabtar>

 

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

CC list is empty

 

Do you think this task is very important?
If so, you can add your encouragement to it.
This task has 0 encouragements so far.

Only logged-in users can vote.

 

 

 

Follow 2 latest changes.

Date Changed by Updated Field Previous Value => Replaced by
2006-09-01 spiralvoice Reassign ItemFrom group mldonkey, a multi-networks file-sharing client, bugs tracker => To group mldonkey, a multi-networks file-sharing client, patch tracker
2006-08-29 spiralvoice StatusNone => In Progress

Back to the top


Powered by Savane 3.5