bugmldonkey, a multi-networks file-sharing client - Bugs: bug #985, NetBSD: Too many open files

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #985: NetBSD: Too many open files

Submitter:  None
Submitted:  Tue 13 Aug 2002 12:59:27 PM UTC
   
 
Category:  Core Severity:  3 - Normal
Item Group:  None Status:  None
Assigned to:  mldonkey Open/Closed:  Closed
Release:  None Release: 
Operating System:  None Binaries Origin:  None
CPU type:  None

Jump to the original submission

Wed 25 Feb 2004 01:11:07 AM UTC, comment #27: 

This patch report is old. If the bug still exists in current
versions please post a new bug report - spiralvoice

spiralvoice <spiralvoice>
Group administrator
Mon 16 Feb 2004 12:15:48 AM UTC, comment #26: 

Please check if this bug is still in current CVS version.

spiralvoice <spiralvoice>
Group administrator
Tue 25 Nov 2003 11:56:20 PM UTC, comment #25: 

Moin,

I have a similar problem on Linux 2.2.24ext3:

mlnet has been running for quite some time:
mow       4481  2.8 24.7 34224 31584 ?       S    Sep28 2418:36 mlnet -daemon
It has even survived several changes in routing without giving any problems, but now ...

I get:
socket(PF_UNSPEC, 0 /* SOCK_??? */, 0)  = -1 EMFILE (Too many open files)

Debugging shows: 84 connections on 127.0.0.1 in state CLOSE

Further looking shows:
lsof|grep mlnet|wc
   1030    9972  116758
with lots of
mlnet      4481  mow  123w   REG       3,66     24460     100400 /samba-shares/audio/mldonkey/files.ini (deleted)

More exactly:
lsof|egrep "mlnet.*deleted"|wc
    705    7050   83593
lsof|egrep "mlnet.*ini"|wc
    658    6580   78094

It seems like when mlnet rewrites its config files, it never closes the old ones.

mldonkey is:
mldonkey-2.5-3.static.i686-Linux.tar.bz2

Hope this helps.

Mark-Oliver Wolter <m-o-w>
Fri 05 Sep 2003 07:10:54 AM UTC, comment #24: 

ulimit -n 1024
before starting mldonkey fixes the problem too.

Thomas Klausner <tk>
Thu 09 Jan 2003 10:17:31 AM UTC, comment #23: 

NetBSD has a softlimit of 64 filedescriptors at maximum by d
The hardlimit is 1772. Simply set the softlimit higher with
sysctl -w proc.$$.rlimit.descriptors.soft=1024
before starting mldonkey.

Anonymous
Fri 08 Nov 2002 06:15:41 PM UTC, comment #22: 

Updated to latest CVS and the OpenBSD 3.2 issue is gone, but it seems almost impossible to connect to servers now. I will investigate.

Anonymous
Fri 08 Nov 2002 05:47:17 PM UTC, comment #21: 

Same problem on OpeBSD 3.2

Anonymous
Sun 29 Sep 2002 11:22:21 AM UTC, comment #20: 

Ok, I am trying this new cvs (from 28-09 10h (gmt+1)).

But now I have another Problem - mldonkey uses 80 % memory (I have 128 MB). Is this normal? Before this version I had ~ 50% - and downloads worked ok (except the open file bug :) ).

Because of the high memory use, downloads are extremly slow (0.1 %). The system access of course is slow, too.

Any ideas?

Regards
martin

Anonymous
Sat 28 Sep 2002 08:32:01 PM UTC, comment #19: 

I have just commited new fixes that might have an impact on this bug.

mldonkey admin <mldonkey>
Group administrator
Sat 28 Sep 2002 04:39:52 PM UTC, comment #18: 

No problem, I have the actual cvs now and this time debug_net in downloads.ini works.

So when the bug happens again I will post the logs immediately.

martin

Anonymous
Sat 28 Sep 2002 10:34:28 AM UTC, comment #17: 

Thanks, the logs helped me a lot. I found at least one bug (incoming connection timeouts are not correctly initialized, so that these sockets never get disconnected). I commited a fix for that. Could someone try it ?

For the logs, I still need more information: you must set the "debug_net" option at the beginning (best would be to start mldonkey, change the option with "set" in telnet, save, quit and restart, so that all connections FD appear in the logs). Can you try that ? Specify which version you use (ie if you use the just commited CVS, or the old beta8 :)




mldonkey admin <mldonkey>
Group administrator
Fri 27 Sep 2002 04:36:39 PM UTC, comment #16: 

Hi,

here is another log with "debug_net" set to "true" in telnet.

And I forgot to say, my ip address did not change (this and last time), when this bug happened (I have looked at pppd logs).

Are these logs ok?

Regards
martin

Anonymous
Thu 26 Sep 2002 08:26:23 PM UTC, comment #15: 

Hi,

I have the same problem under linux, too.

I have 2.4.19 Linux on K6-2 350 and 128 MB Ram. Use an cvs version from 19.09.2002. It is compiled with ocaml 3.06 (make opt).

I followed your instructions and attach the .bz2 (including ls -l, lsof, netstat, mldonkey.log, downloads.ini).

I have put the "debug_net = true" in the downloads.ini, but as you can see, it is deleted after start. "verbose = true" works ok. And the debug_socks from telnet too.

If it is important I can log it again with "debug_net = true" set from telnet. Do you need this?

Regards
martin

Anonymous
Wed 25 Sep 2002 08:56:42 AM UTC, comment #14: 

Right, here's the logs with 'debug_net' enabled this time :)

If you need anything else let me know and I'll run things again - I've switched off the debugging for now though.

Cheers,
Robin

Anonymous
Wed 25 Sep 2002 06:51:35 AM UTC, comment #13: 

OK, I will check your logs to try to understand the problem.

mldonkey admin <mldonkey>
Group administrator
Tue 24 Sep 2002 08:11:59 AM UTC, comment #12: 

Okay, I've followed the instructions given and have attached my log, downloads.ini, lsof & netstat output, etc.  I'm running it again as I found out it's actually the "debug_net" option not "net_debug" option (I set it from telnet and this doesn't give any errors with invalid options).

There were no changes on the system while this was running (it's on a static IP address and a permanently-on connection).

Anyway, hope this helps and I'll post up the others next time it stops.

Cheers,
Robin

Anonymous
Mon 23 Sep 2002 10:25:54 AM UTC, comment #11: 

I've got this bug under linux, too. System is Debian Woody with a patched 2.4.18. Those disfuntional sockets (which "strace" says are closed) show up under lsof with "can't indendify target". I have attached one log generated by

while true; do
 o=`lsof -n | grep mldonkey | grep iden  | wc -l`
 d=`date`
 t=`lsof -n | grep mldonkey | wc -l`
 echo "$d $o/$t stalled sockets" ; sleep 50
done

I'm on the mldonkey-bugs and -users list for contact

Jens

Anonymous
Sun 22 Sep 2002 01:52:27 PM UTC, comment #10: 

How long does it take to get the "too many open files" message ?

Before it happens, set the "net_debug" and "verbose" options to true (with CVS version), and keeps a log of mldonkey output (pipe it to a file).
Keep also a telnet session to mldonkey opened, so that you are sure to be able to send commands when no fd will be available.

 When it happens:

  • In mldonkey telnet, call the "debug_socks" command, verify that mldonkey has placed some debug information on sockets in the log.
  • In the shell, do a "netstat -nap", check mldonkey pid, go in /proc/<pid>/fd (if you have it), and "ls -l", use the "lsof" program on mldonkey (if you have it). Take as much information as you can (downloads.ini for example too), put them in a tar file, compress it (bzip2 or gzip) and upload it here in a bug report.


Do you have special behavior of your connection between the time you start mldonkey and the error (ppp connection reseted, IP changed, etc ?).

mldonkey admin <mldonkey>
Group administrator
Sun 22 Sep 2002 01:34:41 PM UTC, comment #9: 

I've now got two systems running the CVS version, one has this problem and the other doesn't.  I've had a look at the setup and the only difference I can see is that one is linked against libz (the working one) and the other isn't.

I've now recompiled the broken one with libz-devel installed (but it's also with a newer CVS version) so I'll see whether this makes a difference.

If anyone else suffering this problem has their mldonkey executable linked against libz, can they say so?  Or can any of the developers suggest whether this would/would not lead to these (or any other) problems?

Cheers,
Robin

Anonymous
Sun 22 Sep 2002 09:33:02 AM UTC, comment #8: 

I've got OpenBSD and this error is solved (on my system) by reducing max_open_connections to 37 (when i put there 38, error occures).

Please help us because such small number means small downloads (at least for me)!

Regards
BusteR

Anonymous
Tue 17 Sep 2002 01:09:17 PM UTC, comment #7: 

Not enough information to solve this bug. An important bug, though

mldonkey admin <mldonkey>
Group administrator
Mon 16 Sep 2002 08:26:06 AM UTC, comment #6: 

Not just NetBSD, I have this problem on Linux as well:

After running mldonkey for under 12 hours it has 995 file descriptors allocated to sockets (from /proc/mldonkey_PID/fd) and only 13 client and 2 server sockets in use (from netstat).  There are 27 file descriptors allocated to shared & config files (of which 11 are marked deleted). This is with max_opened_connections set to only 200, and mainly downloading.

Anonymous
Sun 15 Sep 2002 04:13:07 PM UTC, comment #5: 

I've got the same.. I don't know what's wrong. Since it prints out a handled error msg someone of the developers should be able to take a look on it. Hopefully ;)

Anonymous
Sun 15 Sep 2002 04:11:48 PM UTC, comment #4: 

I've got the same.. I don't know what's wrong. Since it prints out a handled error msg someone of the developers should be able to take a look on it. Hopefully ;)

Anonymous
Tue 10 Sep 2002 01:19:46 PM UTC, comment #3: 

I agree with you....
I have compiled mldonkey for OpenBSD-i386 and the same problem happens.
Just try to reduce your maximum connections...
But this will not solve your problem permanently.

Anonymous
Tue 10 Sep 2002 01:18:24 PM UTC, comment #2: 

I agree with you....
I have compiled mldonkey for OpenBSD-i386 and the same problem happens.
Just try to reduce your maximum connections...
But this will not solve your problem permanently.

Anonymous
Tue 10 Sep 2002 01:17:11 PM UTC, comment #1: 

I agree with you....
I have compiled mldonkey for OpenBSD-i386 and the same problem happens.
Just try to reduce your maximum connections...
But this will not solve your problem permanently.

Anonymous
Tue 13 Aug 2002 12:59:27 PM UTC, original submission:  

Hi,

my logfile is filled with endless repetitions of:

+++ Exception BEFORE CONNECT socket failed: Too many open files
Exception socket failed: Too many open files in client connection

What does it mean - how can I solve this problem?

Thanks,
 Andreas

mldonkey is great! I'd be happy to provide some code for it,
but why the heck did you use ocaml and not something less
exotic? :)

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #101:  ml_openfiles_logs_2.tar.bz2 added by None (106KiB - application/octet-stream - Logs from martin (linux) - now with debug_net from telnet)
file #100:  ml_openfiles_logs.tar.bz2 added by None (264KiB - application/octet-stream - Logs from martin (linux) - look at the comment, too)
file #98:  mldonkey_debug_logs2.tar.bz2 added by None (128KiB - application/bzip2 - New debug logs and socket info, with debug_net enabled)
file #96:  mldonkey_debug_logs.tar.bz2 added by None (134KiB - application/bzip2 - Logs and socket info)
file #95:  stalls.log added by None (23KiB - application/octet-stream - Defunct socket statistics for one run)

 

Depends on the following items: None found

Items that depend on this one: None found

 

CC list is empty

 

There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

 

Follow 7 latest changes.

Date Changed by Updated Field Previous Value => Replaced by
2004-02-25 spiralvoice StatusRemind None
    Open/ClosedOpen Closed
2002-09-22 mldonkey ReproducibilityHow long does it take to get the &quot;too many open files&quot; message ? Before it happens, set the &quot;net_debug&quot; and &quot;verbose&quot; options to true (with CVS version), and keeps a log of mldonkey output (pipe it to a file). Keep also a telnet session to mldonkey opened, so that you are sure to be able to send commands when no fd will be available. When it happens: * In mldonkey telnet, call the &quot;debug_socks&quot; command, verify that mldonkey has placed some debug information on sockets in the log. * In the shell, do a &quot;netstat -nap&quot;, check mldonkey pid, go in /proc/&lt;pid&gt;/fd (if you have it), and &quot;ls -l&quot;, use the &quot;lsof&quot; program on mldonkey (if you have it). Take as much information as you can (downloads.ini for example too), put them in a tar file, compress it (bzip2 or gzip) and upload it here in a bug report. Do you have special behavior of your connection between the time you start mldonkey and the error (ppp connection reseted, IP changed, etc ?).
2002-09-22 mldonkey Reproducibility
2002-09-17 mldonkey CategoryNone None
    StatusNone None
    Assigned toNone None

Back to the top

Powered by Savane 3.13-d3ae.
Corresponding source code