bugmldonkey, a multi-networks file-sharing client - Bugs: bug #7967, memory leak and lockup

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #7967: memory leak and lockup

Submitter:  Mark-Oliver Wolter <m-o-w>
Submitted:  Tue 02 Mar 2004 12:37:31 AM UTC
   
 
Category:  Core Severity:  3 - Normal
Item Group:  Memory leak Status:  Wont Fix
Assigned to:  None Open/Closed:  Closed
Release:  2-5-12 Release:  2-5-12
Operating System:  Linux Binaries Origin:  CVS / Self compiled
CPU type:  None

Jump to the original submission

Sat 21 Aug 2004 12:01:04 PM UTC, comment #6: 

On my non-SMP system I also have four threads with 2-5-16t, so this is normal.

spiralvoice <spiralvoice>
Group administrator
Fri 23 Jul 2004 01:23:15 AM UTC, comment #5: 

ATM it is behaving, but I have no big downloads running currently. ulimit is set to 100MB, in the last weeks mlnet ran about a day and a half before it exhausted that limit, so the problem that mlnet apparently uses more memory for connection handling than back in 2.5-4 still exists in 2.5-16q. However, CPU usage is low, no problems with that.

Is it normal that four threads are appearing?
mow      10314  2.8 34.4 50624 43936 ?       S    Jul17 251:27 mlnet -daemon
mow      10315  0.0 34.4 50624 43936 ?       S    Jul17   0:10  \_ mlnet -daemon
mow      10316  0.0 34.4 50624 43940 ?       SN   Jul17   0:03      \_ mlnet -daemon
mow      10353  0.0 34.4 50624 43940 ?       SN   Jul17   0:47      \_ mlnet -daemon

See also what I just posted in bugs #8580.

Mark-Oliver Wolter <m-o-w>
Fri 23 Jul 2004 12:23:40 AM UTC, comment #4: 

You are not the first one mentioning problems on a SMP system. What is the situation now?

spiralvoice <spiralvoice>
Group administrator
Sat 08 May 2004 08:35:17 PM UTC, comment #3: 

Okay, following up with the information I gathered since.

Memory use does not continuously increase, but also goes back down, sometimes a lot, e.g. when mldonkeywatch is closed or a big file is finished. This might mean there is not really a memory leak, just mlnet has become a lot more bloaty somewhere since 2.5-4 (see also bugs #8187). With 2.5-16l I have even seen memory usage going down from about 80MB back to 20.

Your mldonkey-2.5-12b.static.i386-Linux.rar exhibited exactly the same behavior, aside from the multithreading which I disabled in my compile. [1]

With the 2.5-16 cores, I noticed that the behavior chanced so that mlnet didn't lockup after a while but still responded, albeit slowly due to swapping.
However, this brought a new problem: As mlnet continues to work, it also continues to grow.
After a while, this leads to the machine freezing. I even replaced the power supply before I concluded it's mlnet's fault (and the kernel's, due to inability to handle such a growing user program properly).

I then set ulimits:
ulimit -d 100000 -l 100000 -m 102400 -v 110000
(this is a bit redundant, but you never know ...)
This caused mlnet to at least shut down not too ungracefully, leaving the linux box alive. However, yesterday I noticed it triggered another problem, see bugs #8580 - and mlnet still runs just for 1-2 days before it gets so big that it runs out of memory, which apparently is a condition certain programmers thought of as something that can never happen.

I'll now go and download 2.5-16m.
Is it already safe to try your 2.5-21b?

[1] weird stuff with threads: mlnet starts with 3 threads, then after a while a fourth appears. Is this normal? I have an SMP machine.

Mark-Oliver Wolter <m-o-w>
Tue 04 May 2004 07:18:40 PM UTC, comment #2: 

Please check with current CVS versions if the memory leak is still there. Regarding the saving of ini files with low free HDD space there is a new option:
2004/03/22: Fabrice
  - Safer options-saving at exit: close all the sockets to prevent "not
     enough file descriptors" error and remove a file called
    "config_files_space.tmp" to free 'config_files_security_space' megabytes
    created at startup.

spiralvoice <spiralvoice>
Group administrator
Tue 16 Mar 2004 12:29:54 AM UTC, comment #1: 

You can get static binaries here: http://www.8ung.at/spiralvoice/

spiralvoice <spiralvoice>
Group administrator
Tue 02 Mar 2004 12:37:31 AM UTC, original submission:  

Moin,

I'm seeing something not completely unlike bugs #7222 in that my self-compiled 2-5-12 eats a lot of memory.
However, the mldonkey-2.5-4.static.i686-Linux.tar.bz2 I used before didn't.

So I think the problem lies maybe not in the mldonkey itself but in the way we compile it, ie gcc & libs ...

What happened here is:
After running since Jan 2, on Feb 25 I shut down the old mlnet 2.5-4:
  637 mow       14   0 39704  38M  2088 R     0  2.1 31.1  2278m mlnet

As you can see, memory usage is quite low. Download was about 3.4GB during that time, with about 12GB upload.

Then I installed the newly-built 2-5-12 (by the way, thankyouverymuch for rm-rf'ing the ocaml tree after copying the libs to the safe place failed due to insufficient disk space, thus having to compile the stuff a second time).
The new mldonkey quickly went to download and even appeared to perform better than the older version, and initially even the memory footprint looked better:
 8602 mow       12   0 15156  14M  2824 R     0 25.5 11.8   2:01 mlnet
However, it didn't stay like that.
Due to a power outage, I had to boot up the machine on Saturday.
This morning, I found mlnet was dead.
A straight line in mldonkeywatch stats.
Memory footprint now didn't look so good:
  892 mow        7   0  170M 106M  2472 D     1  3.5 85.9 228:33 mlnet
strace -p 892 showed ... nothing.
lsof showed 125 open thingies.
telnet to 4000 also hung.
I then murdered mlnet by sending an -IOT/ABRT.

Then tried to restart:
[...]
File ./fasttrack.ini.tmp exists
An error may have occurred during previous configuration save.
Please, check your configurations files, and rename/remove this file
before restarting
SAVING SHARED FILES AND SOURCES
SAVED
Network.save_complex_options not implemented by BitTorrent
Options correctly saved

What mlnet did, is:
a) remove said fasttrack.ini.tmp itself
b) move servers.ini, files.ini, file_sources.ini, friends.ini, shared_files_new.ini to .old and put default files in their location.

Maybe rotating .ini files and then bailing out before writing new ones is not that good an idea.

Okay, next start:
"Looks like you have no servers in your servers.ini"
... yes, thank you, you just removed them. Okay, apparently you got yourself a new list, too, so I won't complain much.

Well, mlnet starts up, all seems fine ...
... except, a bunch of file names from the downloads are missing, only the hashes show. [btw, mldonkeywatch nitpick, don't know if the problem lies with mldw or with mlnet itself: changing to another filename from the dropdown list reverts to the old filename on the next refresh]
Now, hashes got replaced by filenames as files were found.
However, "pause" settings were gone also.

Now mlnet has been running for 10h16mins.
mow      23887  9.5 65.9 87124 84016 ?       S    13:25  58:58 mlnet -daemon

That's quite a lot of memory.

Now, let's see what we have here:
# ldd /usr/local/bin/mlnet
    libz.so.1 => /usr/lib/libz.so.1 (0x4001d000)
    libm.so.6 => /lib/libm.so.6 (0x4002c000)
    libdl.so.2 => /lib/libdl.so.2 (0x4004e000)
    libc.so.6 => /lib/libc.so.6 (0x40052000)
    /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
those libs pointing to:
-rwxr-xr-x   1 root     root        55772 Mar 11  2002 /usr/lib/libz.so.1.1.4*
-rwxr-xr-x   1 root     root       561250 May 26  2001 /lib/libm-2.2.3.so*
-rwxr-xr-x   1 root     root        72701 May 26  2001 /lib/libdl-2.2.3.so*
-rwxr-xr-x   1 root     root      4787276 Jan 11  2002 /lib/libc-2.2.3.so*
-rwxr-xr-x   1 root     root       432647 May 26  2001 /lib/ld-2.2.3.so*

Compiler:
# gcc --version
2.95.3

Machine:
Linux good 2.2.26ext3 #4 SMP Thu Feb 25 05:30:18 CET 2004 i686 unknown

configure invocation:
  $ ./configure
(auto-gets and builds ocaml-3.07pl2)
... followed by make depend and make, as told.

Now, it would be really interesting whether a mldonkey-2-5-12-static built in the same way as the 2.5-4-static leaks or not. Could the one who produced those binaries please do so again, that I can test whether it's mldonkey or just my environment that acts up?

BTW: This is a machine with two DSL lines. Default route is on the PPPOE interface, thus changing IP every 24 hours.
The old 2.5-4 first ran on the PPPOE line, then kept getting more and more (Overnet?) connects over the fixed IP line, leading to, at first, IP-flapping in the mldonkeywatch Settings:Core window, then after a few weeks it stood solid on the fixed IP. A problem was, that some servers then complained "ERROR: Your userhash doesn't match the login one", so it seems mldonkey sent something based on the wrong IP. Unfortunately, I can't check that with 2-5-12 because it simply doesn't run long enough.

Hope that made sense.
MOW
(mlnet has now 100840K. Shutting down in a few.)

Mark-Oliver Wolter <m-o-w>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

CC list is empty

 

There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

 

Follow 4 latest changes.

Date Changed by Updated Field Previous Value => Replaced by
2004-08-21 spiralvoice StatusNone Wont Fix
    Open/ClosedOpen Closed
2004-03-02 t8m Carbon-Copy- Added t8m
2004-03-02 spiralvoice ReleaseNone 2-5-12

Back to the top

Powered by Savane 3.13-d3ae.
Corresponding source code