bugmldonkey, a multi-networks file-sharing client - Bugs: bug #9391, Files copied to incoming dir for...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #9391: Files copied to incoming dir for sharing do not appear in upstats after hashing

Submitted by:  None
Submitted on:  Sat 19 Jun 2004 09:23:18 PM UTC  
 
Category: CoreSeverity: 3 - Normal
Item Group: Program malfunctionStatus: Fixed
Assigned to: NoneOpen/Closed: Closed
Release: Release: 2.5.30.16
Operating System: LinuxBinaries Origin: CVS / Self compiled
CPU type: Intel x86

(Jump to the original submission Jump to the original submission)

Sun 17 Jul 2005 04:35:47 PM UTC, comment #11:

upstats.patch (file #2596) is committed to CVS via patch #4180.

spiralvoice <spiralvoice>
Project Administrator
Sun 17 Jul 2005 01:31:49 PM UTC, comment #10:

Regarding comment #9 I think this is related to compute_md4_delay.
This is an option only used in the Donkey module, now there is
commonSwarming where hashing and verifying chunks takes place, so
the only purpose of compute_md4_delay is to check if there are
finished files, scan them and put them into shared_files_new.ini.

If the timer related to compute_md4_delay in the donkey module
catches a finished file before it is committed it will put
./temp/urn_ed2k_ABABABABAB into shared_files_new.ini, after
commmitting took place the timer finds a "new" file in
./incoming/files, rehashes and puts it into shared_files_new.ini
again. Raising compute_md4_delay to 60 sometimes solves the
problem, I think raising it to 180 seconds would be better as
auto_commit is called every 60 seconds. The donkey module however
will print an error message in the logfile that it could not find
a shared file but it is only present once in shared_files_new.ini
because the common module informed the donkey module about that
file already.
It would be good to remove the hashing code completely from the
donkey module because the common module has all necessary functions
for it, unfortunately shared_files_new.ini is used exclusively
in the EDK module and can“t be removed easily without deep changes
to the code. b8_bavard stopped half-way implementing the common
module...

spiralvoice <spiralvoice>
Project Administrator
Sun 17 Jul 2005 12:50:25 PM UTC, comment #9:

I think I tracked down the bug described in comment #8 but I am
not sure if I fixed that correctly because some times the files
are still present twice, see too_much_shared.patch

spiralvoice <spiralvoice>
Project Administrator
Sun 17 Jul 2005 12:48:19 PM UTC, comment #8:

I attached shared_twice.patch for debugging the fact that shared
files are present twice in shared_files_new.ini, which is only
used by the Donkey module. Files should be present there at least
twice, once as incoming/file.aaa, second as temp/ABABABABABABABAB.

spiralvoice <spiralvoice>
Project Administrator
Mon 13 Jun 2005 05:27:16 PM UTC, comment #7:

I changed that to one minute to have an up-to-date list.
Files are only checked for existance, no hashing or other stuff takes place.
But I will test it with some 1000+ files (/usr/portage on Gentoo;-))
to see how it performs.

spiralvoice <spiralvoice>
Project Administrator
Mon 13 Jun 2005 02:30:01 PM UTC, comment #6:

Is it a good idea to shared_check_files every minute?
There are users sharing hundreds (or mybe thousands) of files.

If you want to check you can always issue a "reshare"-command.

Knocker <knocker>
Sat 11 Jun 2005 07:42:54 PM UTC, comment #5:

Another update, unshare directory should also remove file entries
from upstats. Note: the files are not removed from shared_files_new.ini
so resharing those files avoids rehashing but they are not uploaded.

spiralvoice <spiralvoice>
Project Administrator
Sat 11 Jun 2005 04:59:36 PM UTC, comment #4:

Updated patch so removed shared files should also be removed from upstats

spiralvoice <spiralvoice>
Project Administrator
Sat 11 Jun 2005 02:19:02 PM UTC, comment #3:

I attached a patch for testing.

It removes a line in donkeyShare where a shared file is removed from
the list maintained by commonUploads after a new file has been hashed.

This is not necessary because one line above donkeyShare.new_file_to_share
is called which itself calls donkeyShare.must_share_file with the
old (old_impl) and the new file-data.

donkeyShare.must_share_file takes care of updating the list of shared
files
match has_old_impl with
None -> update_shared_num impl
| Some old_impl -> replace_shared old_impl impl

Here it replaces the line in upstats with the zeroed MD4 with the
new hashed data (replace_shared old_impl impl) provided by
new_file_to_share called by check_shared_files.

So why call shared_remove there when new_file_to_share already
took care of updating the list and replacing the old with the
new entry?

Some testing here showed good results, copying new files into the
share and downloading files worked. Also the double lines for
finished downloads went away one the first look.

Please report how this patch works for you.

spiralvoice <spiralvoice>
Project Administrator
Fri 06 Aug 2004 05:06:46 PM UTC, comment #2:

(from the forum http://mldonkey.berlios.de/modules.php?name=Forums&file=viewtopic&p=14335)

Hi!

A while ago I posted the following bug report:
http://savannah.nongnu.org/bugs/?func=detailitem&item_id=9391

The following behaviour maybe is related to this:
I copied three new files into my shared directory, and typed [i]reshare[/i] at the telnet interface. After that I logged out and waited for about 30 seconds (I hope that was long enough for three files with a total size of 353 MB on a 733 Mhz Celeron). After that, I stopped the core and restarted it.

The [i]uploaders[/i] command shows me that one user is downloading one of the three files, although it is neither listed under [i]upstats[/i] nor [i]links[/i].

I'm using spiralvoice's 2-5-16t under Debian Linux.

Best regards
indanet

Anonymous
Sun 20 Jun 2004 05:10:11 PM UTC, comment #1:

Maybe the files were not completely hashes before the first restart and disappeared then as they got hashed during that core run. Normal behaviour (please see bug 7925) should be like this:
- core is running
- files are being put into shared/incoming directories
- files are recognized by the core with hash MD4=000000000000000
- files are hashed (see logfile) and disappear from upstats
- when all newly copied files disappeared from upstats is it time to restart the core
- after restart of the core upstats should show correct stats

More info here: http://mldonkey.berlios.de/modules.php?name=Forums&file=viewtopic&t=2286

spiralvoice <spiralvoice>
Project Administrator
Sat 19 Jun 2004 09:23:18 PM UTC, original submission:

I wanted to share some files on the eDonkey network. I copied them into my incoming directory while mlDonkey was running, restartet mlDonkey so that it could recognize and hash the new files -- which it did. Some time after hashing new the files could be found by using mlDonkey's search function, but they didn't appear in the table of the "upstats" command. After noticing this, I restartet mlDonkey after which the new files also appeared in the table of the upstats command.

The correct behaviour should be that new files appear after the first restart in the upstats table, not after the second restart.

Anonymous

 

Attached Files
file #3243:  too_much_shared.patch added by spiralvoice (843B - application/octet-stream)
file #3242:  shared_twice.patch added by spiralvoice (1KiB - application/octet-stream)
file #3077:  upstats.patch added by spiralvoice (4KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

CC list is empty

 

Do you think this task is very important?
If so, you can click here to add your encouragement to it.
This task has 0 encouragements so far.

Only logged-in users can vote.

 

Please enter the title of George Orwell's famous dystopian book (it's a date):

 

 

Follow 13 latest changes.

Date Changed By Updated Field Previous Value => Replaced By
Mon 23 Jan 2006 12:16:58 AM UTCspiralvoiceStatusIn Progress=>Fixed
  Open/ClosedOpen=>Closed
Sun 17 Jul 2005 04:35:47 PM UTCspiralvoiceStatusNone=>In Progress
Sun 17 Jul 2005 12:50:25 PM UTCspiralvoiceAttached File-=>Added too_much_shared.patch, #2719
Sun 17 Jul 2005 12:48:20 PM UTCspiralvoiceAttached File-=>Added shared_twice.patch, #2718
Mon 13 Jun 2005 12:41:06 PM UTCspiralvoiceCategoryeDonkey-Plugin=>Core
  Release2-5-16=>2.5.30.16
  Release2-5-16o (spiralvoice)=>
Sat 11 Jun 2005 07:42:54 PM UTCspiralvoiceAttached File-=>Added upstats.patch, #2596
Sat 11 Jun 2005 07:41:38 PM UTCspiralvoiceAttached File#2594=>Removed
Sat 11 Jun 2005 04:59:36 PM UTCspiralvoiceAttached File-=>Added upstats.patch, #2594
Sat 11 Jun 2005 04:59:08 PM UTCspiralvoiceAttached File#2593=>Removed
Sat 11 Jun 2005 02:19:02 PM UTCspiralvoiceAttached File-=>Added upstats.patch, #2593

Back to the top


Powered by Savane 3.1-cleanup1