taskmldonkey, a multi-networks file-sharing client - Tasks: task #11489, Incremental scan of shared files

 
 

You are not allowed to post comments on this tracker with your current authentification level.

task #11489: Incremental scan of shared files

Submitted by:  Huang Hang <seakage2h>
Submitted on:  Tue 25 Oct 2011 07:06:09 PM UTC  
 
Category: NonePriority: 5 - Normal
Status: NoneAssigned to: None
Open/Closed: OpenShould Start On: -
Should be Finished on: -

Tue 25 Oct 2011 07:06:09 PM UTC, comment #2:

This item has been reassigned from the project mldonkey, a multi-networks file-sharing client bugs tracker to your tracker.

The original report is still available at bugs #34110

Following are the information included in the original report:

[field #0] Item ID: 34110
[field #1] Group ID: 1409
[field #2] Open/Closed: Open
[field #3] Severity: 3 - Normal
[field #4] Privacy: Public
[field #8] : Unknown bugs Field Display Type
[field #9] Category: Core
[field #10] Submitted by: seakage2h
[field #11] Assigned to: None
[field #12] Submitted on: Mi 24 Aug 2011 06:25:53 CEST
[field #13] Summary: Incremental scan of shared files
[field #14] Original Submission: At present, when mldonkey is starting up(start by init scripts or by manually called), it will do a full range scan of all shared files in the shared paths. If there're a lot of shared files, the scanning will introduced huge amount of file caches and the whole performance of the system is quite slow.

For example, in my case, I have 2000+ files and totally in 220+ GB large(some are small ebooks, while some are large movies). I've setup mldonkey auto-startup in Gentoo. After system boot up and mldonkey is running, my harddisk will keep busy for a couple of minutes, CPU usage is ~20%, and all 4G memory are occupied by file cache, and ~70M swap in use. If I don't make mldonkey auto-startup, this won't happen.

So I think mldonkey should do incremental scan instead of the full scan. Since the info of all shared files are stored in shared_files_new.ini, mldonkey can check whether the shared file is in the list or not, just fully examine those not in the list, and delay verification of the file info when the shared file is requested(e.g. recompute the MD4 sum when other clients ask for sharing the file).
[field #16] Item Group: Program malfunction
[field #17] Status: None
[field #18] Component Version: None
[field #19] Operating System: Linux
[field #20] Reproducibility: None
[field #21] Size (loc): None
[field #22] Fixed Release: None
[field #23] Planned Release: None
[field #24] Effort: 0.00
[field #28] Priority: 5 - Normal
[field #31] Percent Complete: 0%
[field #33] Release: 3.1.0
[field #34] Release: All
[field #58] Binaries Origin: Gentoo ebuild
[field #59] CPU type: None
[field #60] CPU: None
[field #61] Custom Select Box #4: None
[field #62] Custom Select Box #5: None
[field #63] Custom Select Box #6: None
[field #64] Custom Select Box #7: None
[field #65] Custom Select Box #8: None
[field #66] Custom Select Box #9: None
[field #67] Custom Select Box #10: None

spiralvoice <spiralvoice>
Project Administrator
Wed 24 Aug 2011 04:32:13 AM UTC, comment #1:

I'm running mldonkey on Gentoo, and this bug exists since I use mldonkey the first time almost 3 years ago.

My PC box configuration:

CPU: AMD K10 3.0G, 4 cores
Memory: 4G DDR3 1333
Disk: 1TB, 7200rpm, 32MB cache

Huang Hang <seakage2h>
Tue 25 Oct 2011 07:06:09 PM UTC, original submission:

At present, when mldonkey is starting up(start by init scripts or by manually called), it will do a full range scan of all shared files in the shared paths. If there're a lot of shared files, the scanning will introduced huge amount of file caches and the whole performance of the system is quite slow.

For example, in my case, I have 2000+ files and totally in 220+ GB large(some are small ebooks, while some are large movies). I've setup mldonkey auto-startup in Gentoo. After system boot up and mldonkey is running, my harddisk will keep busy for a couple of minutes, CPU usage is ~20%, and all 4G memory are occupied by file cache, and ~70M swap in use. If I don't make mldonkey auto-startup, this won't happen.

So I think mldonkey should do incremental scan instead of the full scan. Since the info of all shared files are stored in shared_files_new.ini, mldonkey can check whether the shared file is in the list or not, just fully examine those not in the list, and delay verification of the file info when the shared file is requested(e.g. recompute the MD4 sum when other clients ask for sharing the file).

Huang Hang <seakage2h>

 

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by spiralvoice (Updated the item)
  • -unavailable- added by seakage2h (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follows 1 latest change.

    Date Changed By Updated Field Previous Value => Replaced By
    Tue 25 Oct 2011 07:06:09 PM UTCspiralvoiceReassign itemmldonkey, a multi-networks file-sharing client, bug #34110=>mldonkey, a multi-networks file-sharing client, task #11489

    Back to the top


    Powered by Savane 3.1-cleanup