"I'll use console commands via sancho to acquire information about overnet and kademlia sources/activity.Is there another way?"

No

This item has been reassigned from the project mldonkey, a multi-networks file-sharing client patch tracker to your tracker.

The original report is still available at patch #5360

Following are the information included in the original report:

[field #0]                  Item ID: 5360
[field #1]                 Group ID: 1409
[field #2]              Open/Closed: Open
[field #3]                 Severity: 3 - Normal
[field #4]                  Privacy: Public
[field #6]                 Category: None
[field #7]             Submitted by: fabtar
[field #8]              Assigned to: None
[field #9]             Submitted on: Freitag 01.09.2006 um 18:25
[field #10]                  Summary: HTML: Output bigger than 16MB can not be displayed due to buffer size limits
[field #11]      Original Submission: I have this error when using overnet stat from webinterface:

Code:
Unknown type for content :
Exception Failure("Buffer.add: cannot grow buffer")


and overnet stats are not visualized..I m quite sure this thing has not appeared days ago with a younger session.
Mldonkey keeps working, the error is not critical and seems an interface-only issue.

at the moment:

Quote:
Buildinfo
Version: MLNet Multi-Network p2p client version 2.7.7.CVS
SCM version: 2006-07-11 12:20:38
Networks: Global Shares Gnutella FileTP BitTorrent Donkey (SUI)
Ocaml version: 3.08.3 - C compiler version: 3.3.5 - C++ compiler version: 3.3.5
Build on: Linux i686 2.6.8-2-386 (little endian) with glibc 2.3.2
Configure args: --enable-multinet --disable-fasttrack --disable-gnutella2 --disable-gui
Features: threads zlib-1.2.2 bzip2-1.0.2 gd(jpg/png-1.2.Cool iconv magic(active) no-check-bounds

Runinfo
User: admin (Warning: empty Password) - uptime: 10d 17h 24m 15s
Enabled nets: Donkey Overnet Kademlia BitTorrent FileTP
Server usage: enabled
Geoip: enabled, GeoLite data created by MaxMind, available from http://maxmind.com/
IP blocking: local: 0 ranges - web: 96007 ranges
Libmagic: file-type recognition database present
System info: Linux pigrigw.tana.net 2.6.8-2-386 #1 Tue Aug 16 12:46:35 UTC 2005 i686
language: EN - locale: ISO-8859-1 - UTC offset: +0200
max_string_length: 16777211 - word_size: 32 - max_array_length: 4194303
max file descriptors: 1024 - max useable file size: 2^63-1 bits (do the maths ;-p)


Self compiled cvs + pango s patch for browse share requests (I hope to see this patch in CVS in the future).

Cheers
[field #12]                Closed on: Dienstag 05.09.2006 um 23:49
[field #13]               Item Group: None
[field #14]                   Status: None
[field #15]        Component Version: None
[field #16]         Operating System: None
[field #17]          Reproducibility: None
[field #18]               Size (loc): None
[field #19]            Fixed Release: None
[field #20]          Planned Release: None
[field #21]                   Effort: 0.00
[field #25]                 Priority: 5 - Normal
[field #28]         Percent Complete: 0%
[field #30]                  Release: None
[field #55]     Custom Select Box #1: None
[field #56]     Custom Select Box #2: None
[field #57]     Custom Select Box #3: None
[field #58]     Custom Select Box #4: None
[field #59]     Custom Select Box #5: None
[field #60]     Custom Select Box #6: None
[field #61]     Custom Select Box #7: None
[field #62]     Custom Select Box #8: None
[field #63]     Custom Select Box #9: None
[field #64]    Custom Select Box #10: None

Unfortunately I don't see how you can be "pretty sure the results are real", nor how it can be a workaround.

stdlib/buffer.ml

let resize b more =
  let len = b.length in
  let new_len = ref len in
  while b.position + more > !new_len do new_len := 2 * !new_len done;
  if !new_len > Sys.max_string_length then begin
    if b.position + more <= Sys.max_string_length
    then new_len := Sys.max_string_length
    else failwith "Buffer.add: cannot grow buffer"
  end;
  let new_buffer = String.create !new_len in
  String.blit b.buffer 0 new_buffer 0 b.position;
  b.buffer <- new_buffer;
  b.length <- !new_len

As you can see, a buffer will fail to grow ONLY if it exceeds Sys.max_string_length.  That is the only way it can fail.

If you initialize the buffer to 1, 150, 150305 or 3949392, it will double in size each resize until it exceeds Sys.max_string_length.  That is not what I feel, nor what I guess is happening, nor the impression I get after using mldonkey for a week or two. It is what the cold hard code says that it is doing! :-)

The larger point that I am trying to make is that some code might be going into CVS that doesn't make any logical sense. 

This patch doesn't make sense to me.  Can someone please explain why it was added to cvs?

As I understand it, a Buffer is a Buffer.  You can initialize it to 1, or 100000 and it is still a buffer that grows when needed [1]. One specific user says that it helps, but this doesn't carry much weight since it could be an illusion just like when people attribute extraneous causes to increased download speed on a P2P network.

If you don't understand why something happens is it right to apply it to CVS?  My rational for saying "No": If I can see a bug quickly, I can fix it, but if you "paper over" or "hide" a bug, I can't experience it as easily and it is much harder to find and fix the real bug.  I don't think increasing the size at which a Buffer is initialized could possibly fix any "real" bug.  Does anyone think that it does?  Also, if you don't understand why something happens, you probably also don't understand the possible consequences of changing the behaviour of some code (this is more of a general argument for a lot of patches I've seen go in, not really about this specific patch, but I had to get it off my chest :-) ). 

Just my 2cents.



[1]
module Buffer: sig .. end
Extensible string buffers.
This module implements string buffers that automatically expand as necessary. It provides accumulative concatenation of strings in quasi-linear time (instead of quadratic time when strings are concatenated pairwise).

Applied 2006/09/01

This item has been reassigned from the project mldonkey, a multi-networks file-sharing client bugs tracker to your tracker.

The original report is still available at bugs #17202

Following are the information included in the original report:

[field #0]                  Item ID: 17202
[field #1]                 Group ID: 1409
[field #2]              Open/Closed: Open
[field #3]                 Severity: 3 - Normal
[field #4]                  Privacy: Public
[field #6]                 Category: HTTP interface
[field #7]             Submitted by: fabtar
[field #8]              Assigned to: None
[field #9]             Submitted on: Dienstag 25.07.2006 um 18:07
[field #10]                  Summary: Webinterace:overnet stat gives: "Unknown type for content: Exception Failure("Buffer.add: cannot grow buffer")"
[field #11]      Original Submission: I have this error when using overnet stat from webinterface:

Code:
Unknown type for content :
Exception Failure("Buffer.add: cannot grow buffer")


and overnet stats are not visualized..I m quite sure this thing has not appeared days ago with a younger session.
Mldonkey keeps working, the error is not critical and seems an interface-only issue.

at the moment:

Quote:
Buildinfo
Version: MLNet Multi-Network p2p client version 2.7.7.CVS
SCM version: 2006-07-11 12:20:38
Networks: Global Shares Gnutella FileTP BitTorrent Donkey (SUI)
Ocaml version: 3.08.3 - C compiler version: 3.3.5 - C++ compiler version: 3.3.5
Build on: Linux i686 2.6.8-2-386 (little endian) with glibc 2.3.2
Configure args: --enable-multinet --disable-fasttrack --disable-gnutella2 --disable-gui
Features: threads zlib-1.2.2 bzip2-1.0.2 gd(jpg/png-1.2.Cool iconv magic(active) no-check-bounds

Runinfo
User: admin (Warning: empty Password) - uptime: 10d 17h 24m 15s
Enabled nets: Donkey Overnet Kademlia BitTorrent FileTP
Server usage: enabled
Geoip: enabled, GeoLite data created by MaxMind, available from http://maxmind.com/
IP blocking: local: 0 ranges - web: 96007 ranges
Libmagic: file-type recognition database present
System info: Linux pigrigw.tana.net 2.6.8-2-386 #1 Tue Aug 16 12:46:35 UTC 2005 i686
language: EN - locale: ISO-8859-1 - UTC offset: +0200
max_string_length: 16777211 - word_size: 32 - max_array_length: 4194303
max file descriptors: 1024 - max useable file size: 2^63-1 bits (do the maths ;-p)


Self compiled cvs + pango s patch for browse share requests (I hope to see this patch in CVS in the future).

Cheers
[field #13]               Item Group: Program malfunction
[field #14]                   Status: In Progress
[field #15]        Component Version: None
[field #16]         Operating System: Linux
[field #17]          Reproducibility: None
[field #18]               Size (loc): None
[field #19]            Fixed Release: None
[field #20]          Planned Release: None
[field #21]                   Effort: 0.00
[field #25]                 Priority: 5 - Normal
[field #28]         Percent Complete: 0%
[field #30]                  Release: 2.7.7
[field #55]          Binaries Origin: CVS / Self compiled
[field #56]                 CPU type: Intel x86
[field #57]                      CPU: None
[field #58]     Custom Select Box #4: None
[field #59]     Custom Select Box #5: None
[field #60]     Custom Select Box #6: None
[field #61]     Custom Select Box #7: None
[field #62]     Custom Select Box #8: None
[field #63]     Custom Select Box #9: None
[field #64]    Custom Select Box #10: None

>Do you use webinterface frequently?

Yes

The buffer error is not appeared after more than 2 days.this is good.
Also I have noted that the stats's speed raises when kad and ovr starts working badly.
It could be only a perception issue but I was really sure about the  better ovr/kad behaviour. :-)

About your patch.. yes.. things are improved surely, this patch is OK.. A greaer value 200000 .. could be considered as fine too.
Currently I'm using 500000.
Cheers

I have rised buffers(allocation block?) to 500000 but I cannot obtain more than 2 days with a functional kad/ovr.
My final report is:
Increasing buffers(allocation) from 10000 to 100000 has revived my poor&died kad/ovr permitting 2 days of good working.
By risng buffers from 100000 to 500000  I haven't gained more.
I 'll use the 100000 value (I manually patch and compile) and I'll close mldonkey after 3 days uptime.

I doubt that changing the parameter given to Buffer.create can change anything.
It is a "size hint" (the initial size of the Buffer.t), but Buffers are automatically resized whenever they become full. And then the upper limit is always the same : 16M chars, because of the Ocaml string used to implement them.

The only thing it changes is that it decreases the number of reallocations needed, so it may be easier on memory management, but that's all.

Compiled last CVS with your proposed/mini-changes-patches.
Great!, I hope this is not a placebo-effect but Kad and overnet have started immediately with no problems, boot nodes are been  collected correctly and  both boot peers list and buckets and stats are FULLY POPULATED.
There are high source/search hits on ovr/kad networks.
I think that increased buffer has solved performace issues too on my machine.
Session is one day old and there is no buffer problem (at the moment).
I'm confident this 'll delay the problme or solve the issue :-D .
I'll report back later to confirm (hopefully) this good behaviour.

My last CVS core without any patch( I havenot installed the pango's patch equivalent to 5310, nor your suggested patch cause my inability to apply) is running fine, I have no buffer problems.
I'll try with the LAST CVS tomorrow with 5310, and I'll let you informed if the problem is triggered again.

Ovr is has worked fine for a while but currently no boot nodes are collected :-O
Kad is working fine but also at the moement zero boot nodes are colected too.
It seems that kad and ovr too much easily get out of nodes.
Cheers

I have tried again a unpatched cvs.
I have again the problem with overnet and kademlia wich losts progressively boot nodes witout effectively doing searches.
I think this is a local problem of mine.I'll try a statically compiled core of yours on monday.
As far as kad and ovr doesn't boot properly and search suceccfully (I have 0 searches and zero hits in statistics,) the buffer's bug could not be triggered.

I fear somewhere in my network there is an udp problem, but the other applications using udp work perfectly.
Perhaps the boot peers has to be mantained longer with higher timeouts.. ??? I have also lots of download recently on my box, so I have great activity.

on monday I'll also update the system a bit.

thanks for your patience

The old core is not collecting the boot nodes properly :-(.
I'll try compiling this core now,
I have simply to point the fact i'm using too the pango's patch located here(I have pointed in the first post):

ftp://ftp.berlios.de/pub/mldonkey/pango/no_auto_query_view_files.patch

I find this patch  necessary, a must cause this doesn't abuse the friend's feature by asking the friend's continuosly.
I'll compile a new fresh CVS-bi-patched core immediately and if this fail again ..I'll try one without pango's patch.

Thanks a lot for interesting in this bug.I'll report soon.

I have also the impression that meanwhile the bug is triggered  kad and overnet stop working properly (Overnet peers counters in ed2k stats doesn't rise), and currently, rebooting after new long session, I have kad and overet without overnet boot peers.
I'll update the status tomorrow..I'll also try compiling a new CVS and post again about.

I have verified that the memory consuption is a bit higher than normal (about 77 megs) . (but it looks not so abnormal)
My mlnet has about a 15 days long session at the moment without a real crash.
the exception is still there...

I have this error when using overnet stat from webinterface:

Code:
Unknown type for content :
Exception Failure("Buffer.add: cannot grow buffer")


and overnet stats are not visualized..I'm quite sure this thing has not appeared days ago with a younger session.
Mldonkey keeps working, the error is not critical and seems an interface-only issue.

at the moment:

Quote:
Buildinfo
Version: MLNet Multi-Network p2p client version 2.7.7.CVS
SCM version: 2006-07-11 12:20:38
Networks: Global Shares Gnutella FileTP BitTorrent Donkey (SUI)
Ocaml version: 3.08.3 - C compiler version: 3.3.5 - C++ compiler version: 3.3.5
Build on: Linux i686 2.6.8-2-386 (little endian) with glibc 2.3.2
Configure args: --enable-multinet --disable-fasttrack --disable-gnutella2 --disable-gui
Features: threads zlib-1.2.2 bzip2-1.0.2 gd(jpg/png-1.2.Cool iconv magic(active) no-check-bounds

Runinfo
User: admin (Warning: empty Password) - uptime: 10d 17h 24m 15s
Enabled nets: Donkey Overnet Kademlia BitTorrent FileTP
Server usage: enabled
Geoip: enabled, GeoLite data created by MaxMind, available from http://maxmind.com/
IP blocking: local: 0 ranges - web: 96007 ranges
Libmagic: file-type recognition database present
System info: Linux pigrigw.tana.net 2.6.8-2-386 #1 Tue Aug 16 12:46:35 UTC 2005 i686
language: EN - locale: ISO-8859-1 - UTC offset: +0200
max_string_length: 16777211 - word_size: 32 - max_array_length: 4194303
max file descriptors: 1024 - max useable file size: 2^63-1 bits (do the maths ;-p)


Self compiled cvs + pango's patch for browse share requests (I hope to see this patch in CVS in the future).

Cheers