bugmldonkey, a multi-networks file-sharing client - Bugs: bug #12384, mld hangs and uses all cpu if max...


You are not allowed to post comments on this tracker with your current authentication level.

bug #12384: mld hangs and uses all cpu if max sources <20k

Submitted by:  None
Submitted on:  Sun 20 Mar 2005 07:54:18 AM UTC  
Category:  Core Severity:  3 - Normal
Item Group:  Program malfunction Status:  Fixed
Assigned to:  None Open/Closed:  Closed
Release:  Release:  2-5-30
Operating System:  Linux Binaries Origin:  CVS / Self compiled
CPU type:  Intel x86

( Jump to the original submission)

Mon 28 Mar 2005 11:07:56 AM UTC, comment #22: 
Amorphous <amorphous>
Sun 27 Mar 2005 01:12:18 PM UTC, comment #21: 

the patch was diffed the wrong way. i correctly applied it in my branch in svn revision 243 .

Amorphous <amorphous>
Fri 25 Mar 2005 06:06:26 PM UTC, comment #20: 

The attached file sm_fix.patch fixes the problem ... tested with max_sources = 3 amd max_sources = 0 and max_sources = -40 ;)

Fritz Mock <surround>
Wed 23 Mar 2005 03:11:29 PM UTC, comment #19: 

Ok, gave the new branch a go. Strange enough it failed to connect to any server directly on the first go as i was not using the default 4662 port. The port itself was working before. The core was running stable, but useless :-) After changing it to the default one it took 7-15 minutes to get an unresponsive core again. I've compiled the debug one again now .....

ocamldebug $(find /source/amorphous-compilation/src/ -name '.*' -prune -o -name 'gtk' -prune -o -name 'gtk2' -prune -o -name 'applets' -prune -o -type d -printf '-I %p ') ./mlnet.byte
        Objective Caml Debugger version 3.08.2

(ocd) set checkpoints off
(ocd) run
Loading program... done.
No /root/.mldonkey/installer.ini found
Resolving [legolas] ...done
Using threads
Network Global Shares registered
Network BitTorrent registered
Network Donkey registered
IP blacklist: 60669 ranges loaded
Logging in /var/log/mldonkey.log
Time : 2897090000 - pc : 991336 - module CommonSources
1643                     if Queue.length q<|a|> > 0
(ocd) bt
#0  Pc : 991336  CommonSources char 71153
#1  Pc : 14544  List char 1857
#2  Pc : 1422352  DonkeyMain char 2273
#3  Pc : 478008  BasicSocket char 16932
#4  Pc : 480220  BasicSocket char 17823
#5  Pc : 1570856  CommonMain char 1041
(ocd) rame 0
Unknown command.
Time : 2897090000 - pc : 991336 - module CommonSources
1643                     if Queue.length q<|a|> > 0
(ocd) frame 0
#0  Pc : 991336  CommonSources char 71153
1643                     if Queue.length q<|a|> > 0
(ocd) frame 1
#1  Pc : 14544  List char 1857
69   | a::l -> f a<|a|>; iter f l
(ocd) frame 2
#2  Pc : 1422352  DonkeyMain char 2273
83   DonkeySources.clean_sources ()<|a|>; (* Moved here from fivemin_timer. *)
(ocd) frame 3
#3  Pc : 478008  BasicSocket char 16932
500             try t.time_handler t<|a|> with _ -> ()
(ocd) frame 4
#4  Pc : 480220  BasicSocket char 17823
523       exec_timers !timers<|a|>;
(ocd) frame 5
#5  Pc : 1570856  CommonMain char 1041
31   BasicSocket.loop ()<|a|>



Wed 23 Mar 2005 05:41:22 AM UTC, comment #18: 

I just commited the change surround talked about, into my svn branch as revision 237. Perhaps this helps.

Amorphous <amorphous>
Tue 22 Mar 2005 09:15:29 PM UTC, comment #17: 

I don't think that's really the problem, because :
1) I had the problem just after 8 minutes one time.
2) The clean_sources never stop ? I had this during hours (6-8)

Christophe PEREZ <novazur>
Tue 22 Mar 2005 08:05:39 PM UTC, comment #16: 

Amorphous asked me to take a look at this one ... unfortunatly I am in a real hurry this week, but from "code meditation" in break to releax  between conferences ...

commmonSources.ml Line 1663:

iter nsources m.manager_sources.(new_sources_queue) (do_not_try_queue)

should be:

iter nsources m.manager_sources.(new_sources_queue) (new_sources_queue)

That's certenly not intended and a bug. If it not the solution for this problem, you have to be paitient and wait until easter-holidays for a fix.

Fritz Mock <surround>
Tue 22 Mar 2005 05:11:56 PM UTC, comment #15: 

please see the backtrace in duplicate savannah bug #12420

after a quick look at that backtrace: could it be that it is no loop but just that clean_sources realy uses much cpu cycles (if one has some 20k sources) and because we run it every half minute the core effectively hangs?

Amorphous <amorphous>
Mon 21 Mar 2005 08:41:38 PM UTC, comment #14: 

this should be fixed. sourcemanagement should not get stuck in a loop even if it would be realy inefficient for smaller values of max_sources (what happens if a souce has 40k files does it still loop with a limit of 20k?).

@spiralvoice: nope that change only supresses the log message for the exception invalid argument.

@comment #9: you should either rise the fd limit as spiralvoice said or you could decrease the limit in mldonkey.

Amorphous <amorphous>
Mon 21 Mar 2005 05:12:05 PM UTC, comment #13: 

Seems to work fine now, after an hour, and 20000 max_sources.
Thank you very much for all.

Christophe PEREZ <novazur>
Mon 21 Mar 2005 03:54:22 PM UTC, comment #12: 
spiralvoice <spiralvoice>
Project Administrator
Mon 21 Mar 2005 03:29:35 PM UTC, comment #11: 

i set now ulimit -n 800  (was 256) and will see what happens

this log entry thing appeared only with amorphous svn (i think about) 227 and later

Mon 21 Mar 2005 03:16:07 PM UTC, comment #10: 

@comment #9:
"Exception tcp_handler: accept failed: Too many open files"

Some systems have pretty low limits, try raising them according to:

spiralvoice <spiralvoice>
Project Administrator
Mon 21 Mar 2005 01:42:04 PM UTC, comment #9: 

i have a similiar problem. no high CPU load but non responsivness of the core (web interface) and network traffic is going towards zero. but it recovers mostly after some minutes.

i think it is caused by tons of log entries:
Exception poll failed: Bad file descriptor in Select.loop
Exception tcp_handler: accept failed: Too many open files - i thougt this was due to a ocaml bug and was fixed

the log file now has a size of 55 mbytes after a uptime of about 30 hours.

Mon 21 Mar 2005 10:42:07 AM UTC, comment #8: 

oh please check before you try debugging, that your max sources setting is 20000 (the default), if not try if that fixes it.
someone had a similar problem, that was fixed by this: http://mldonkey.berlios.de/modules.php?name=Forums&file=viewtopic&p=17917#17917

Amorphous <amorphous>
Mon 21 Mar 2005 07:56:51 AM UTC, comment #7: 

you could compile mldonkey from the sources (30ab or 30-1) as follows (replace "foo" with all the configure options you did use to compile):
./configure --enable-debug foo
make mlnet.byte

copy mlnet.byte to your mldonkey directory and cd into it
execute the debugger as follows and replace "/path/to/src/" with the path to the directory src in the mldonkey sources:
ocamldebug $(find /path/to/src/ -name '.*' -prune -o -name 'gtk' -prune -o -name 'gtk2' -prune -o -name 'applets' -prune -o -type d -printf '-I %p ') ./mlnet.byte

then type the following in the debugger:
set checkpoints off

then wait until the problem occurs and do the following:
press ctrl-c and type:

then for each # greater than 0 type (and replace # with the number):
frame #

example for the last part:
(ocd) bt
#0  Pc : 6160  Test char 22
#1  Pc : 6160  Test char 22
#2  Pc : 6160  Test char 22
(ocd) frame 1
#1  Pc : 6160  Test char 22
2     <|b|>test ()
(ocd) frame 2
#2  Pc : 6160  Test char 22
2     <|b|>test ()

then copy and past all this (from when you ocamldebug started) into a file and add this file to his bugreport.
i hope i can then make sense of it :) .

Amorphous <amorphous>
Mon 21 Mar 2005 03:28:42 AM UTC, comment #6: 

So, what can I do to give you more information ?

I tried again with CVS version, it took only 8 mn to give me the problem :

top - 23:25:53 up 70 days,  8:20,  1 user,  load average: 2.39, 1.06, 0.72
Tasks: 138 total,   2 running, 136 sleeping,   0 stopped,   0 zombie
Cpu(s):  2.6% us,  8.5% sy, 88.9% ni,  0.0% id,  0.0% wa,  0.0% hi,  0.0% si
Mem:    515240k total,   510200k used,     5040k free,    44080k buffers
Swap:   506008k total,   125452k used,   380556k free,   273652k cached

 5051 mldonkey  27   2 40520  21m 5576 R 89.3  4.3   1:19.76 mlnet             

I had to killproc it.

Christophe PEREZ <novazur>
Mon 21 Mar 2005 01:17:59 AM UTC, comment #5: 

i woun't help, 30-1 is not different enough from 30c. i think i can't find out why that happens without debugging the core on a system where it happens (some people reported something similar, but here the core runs fine for days). it seems like an infinite loop, if i know where the core loops, i should be able to fix it.

Amorphous <amorphous>
Mon 21 Mar 2005 12:55:47 AM UTC, comment #4: 

I used ocaml 3.08.2 yesterday and after 3.08.3 today.
I tried with 2.5.30c fron spiralvoice, and CVS yesterday, and today.
I always had the same problem with all.

Not yet tried 2.5.30-1, I will do it.

Thanks for your help.

Christophe PEREZ <novazur>
Sun 20 Mar 2005 11:26:36 PM UTC, comment #3: 

what ocaml version did you use? could you try if the same happens with 2.5.30-1 (from spiralvoice and in cvs) or 2.5.30ab (from me).

Amorphous <amorphous>
Sun 20 Mar 2005 05:25:35 PM UTC, comment #2: 

Thanks Anonymous ;-)

(I'm now registered as novazur)

Christophe PEREZ <novazur>
Sun 20 Mar 2005 01:13:42 PM UTC, comment #1: 

english translation

He has no problems in compiling and running the core but after a while (less than an hour) CPU load gets very high. He then is unable to access the core neither by web nor telnet interface. The core has to be killed. Even after hours it doesn't get responsive again.

Sun 20 Mar 2005 07:54:18 AM UTC, original submission:  


I won't be able to explain that in english, so, I'll do it in french. I somebody understand or can traduce, it will be fine.

CVS version

Compilation ok, fonctionnement ok, mais après un petit moment (moins d'heure je pense), mon processeur s'embale, impossible d'accéder au serveur, que ce soit par interface web ou par telnet, obligé de killer le process car, même après plusieurs heures, ça ne revient jamais à la normale.
Désolé de ne pouvoir expliquer ça en Anglais.
Merci à ceux qui transmettront.



Attached Files
file #2786:  sm_fix.patch added by surround (1KiB - application/octet-stream)


Depends on the following items: None found

Items that depend on this one: None found


CC list is empty


Do you think this task is very important?
If so, you can add your encouragement to it.
This task has 0 encouragements so far.

Only logged-in users can vote.




Follow 11 latest changes.

Date Changed by Updated Field Previous Value => Replaced by
2005-04-04 spiralvoice Open/ClosedOpen => Closed
2005-03-31 spiralvoice Open/ClosedClosed => Open
2005-03-30 spiralvoice Open/ClosedOpen => Closed
2005-03-27 amorphous StatusNone => Fixed
2005-03-25 surround Attached File- => Added sm_fix.patch, #2353
2005-03-21 amorphous Open/ClosedClosed => Open
2005-03-21 amorphous StatusWont Fix => None
    Summaryprocessor going mad => mld hangs and uses all cpu if max sources <20k
2005-03-21 spiralvoice StatusNeed Info => Wont Fix
    Open/ClosedOpen => Closed
2005-03-21 amorphous StatusNone => Need Info

Back to the top

Powered by Savane 3.5