taskmldonkey, a multi-networks file-sharing client - Tasks: task #11489, Incremental scan of shared files

 
 

You are not allowed to post comments on this tracker with your current authentication level.

task #11489: Incremental scan of shared files

Submitter:  Huang Hang <seakage2h>
Submitted:  Tue 25 Oct 2011 07:06:09 PM UTC
   
 
Category:  None Priority:  5 - Normal
Status:  None Assigned to:  None
Open/Closed:  Open Should Start On:  -
Should be Finished on:  -

Tue 25 Oct 2011 07:06:09 PM UTC, comment #2: 

This item has been reassigned from the project mldonkey, a multi-networks file-sharing client bugs tracker to your tracker.

The original report is still available at bugs #34110

Following are the information included in the original report:

[field #0]                  Item ID: 34110
[field #1]                 Group ID: 1409
[field #2]              Open/Closed: Open
[field #3]                 Severity: 3 - Normal
[field #4]                  Privacy: Public
[field #8]                         : Unknown bugs Field Display Type
[field #9]                 Category: Core
[field #10]             Submitted by: seakage2h
[field #11]              Assigned to: None
[field #12]             Submitted on: Mi 24 Aug 2011 06:25:53 CEST
[field #13]                  Summary: Incremental scan of shared files
[field #14]      Original Submission: At present, when mldonkey is starting up(start by init scripts or by manually called), it will do a full range scan of all shared files in the shared paths. If there're a lot of shared files, the scanning will introduced huge amount of file caches and the whole performance of the system is quite slow.

For example, in my case, I have 2000+ files and totally in 220+ GB large(some are small ebooks, while some are large movies). I've setup mldonkey auto-startup in Gentoo. After system boot up and mldonkey is running, my harddisk will keep busy for a couple of minutes, CPU usage is ~20%, and all 4G memory are occupied by file cache, and ~70M swap in use. If I don't make mldonkey auto-startup, this won't happen.

So I think mldonkey should do incremental scan instead of the full scan. Since the info of all shared files are stored in shared_files_new.ini, mldonkey can check whether the shared file is in the list or not, just fully examine those not in the list, and delay verification of the file info when the shared file is requested(e.g. recompute the MD4 sum when other clients ask for sharing the file).
[field #16]               Item Group: Program malfunction
[field #17]                   Status: None
[field #18]        Component Version: None
[field #19]         Operating System: Linux
[field #20]          Reproducibility: None
[field #21]               Size (loc): None
[field #22]            Fixed Release: None
[field #23]          Planned Release: None
[field #24]                   Effort: 0.00
[field #28]                 Priority: 5 - Normal
[field #31]         Percent Complete: 0%
[field #33]                  Release: 3.1.0
[field #34]                  Release: All
[field #58]          Binaries Origin: Gentoo ebuild
[field #59]                 CPU type: None
[field #60]                      CPU: None
[field #61]     Custom Select Box #4: None
[field #62]     Custom Select Box #5: None
[field #63]     Custom Select Box #6: None
[field #64]     Custom Select Box #7: None
[field #65]     Custom Select Box #8: None
[field #66]     Custom Select Box #9: None
[field #67]    Custom Select Box #10: None

spiralvoice <spiralvoice>
Group administrator
Wed 24 Aug 2011 04:32:13 AM UTC, comment #1: 

I'm running mldonkey on Gentoo, and this bug exists since I use mldonkey the first time almost 3 years ago.

My PC box configuration:

CPU: AMD K10 3.0G, 4 cores
Memory: 4G DDR3 1333
Disk: 1TB, 7200rpm, 32MB cache

Huang Hang <seakage2h>
Tue 25 Oct 2011 07:06:09 PM UTC, original submission:  

At present, when mldonkey is starting up(start by init scripts or by manually called), it will do a full range scan of all shared files in the shared paths. If there're a lot of shared files, the scanning will introduced huge amount of file caches and the whole performance of the system is quite slow.

For example, in my case, I have 2000+ files and totally in 220+ GB large(some are small ebooks, while some are large movies). I've setup mldonkey auto-startup in Gentoo. After system boot up and mldonkey is running, my harddisk will keep busy for a couple of minutes, CPU usage is ~20%, and all 4G memory are occupied by file cache, and ~70M swap in use. If I don't make mldonkey auto-startup, this won't happen.

So I think mldonkey should do incremental scan instead of the full scan. Since the info of all shared files are stored in shared_files_new.ini, mldonkey can check whether the shared file is in the list or not, just fully examine those not in the list, and delay verification of the file info when the shared file is requested(e.g. recompute the MD4 sum when other clients ask for sharing the file).

Huang Hang <seakage2h>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by spiralvoice (Updated the item)
  • -email is unavailable- added by seakage2h (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2011-10-25 spiralvoice Reassign itemmldonkey, a multi-networks file-sharing client, bug #34110 mldonkey, a multi-networks file-sharing client, task #11489

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code