bugDirect Connect Text Client - Bugs: bug #6037, continue of previously stopped...

 
 

bug #6037: continue of previously stopped download does not check integrity of source

Submitted by:  None
Submitted on:  Sat 18 Oct 2003 04:15:26 PM UTC  
 
Category: NoneSeverity: 3 - Normal
Item Group: NoneStatus: None
Privacy: PublicAssigned to: Eric PREVOTEAU <ericprev>
Open/Closed: Closed

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Sat 15 Nov 2003 02:45:06 PM UTC, comment #1:

There is no way to verify if the new source is the same file as the one you have (DCTC has a protocol extension to do that but it requires a hub and other clients to use it).

Your solution has a big flaw because it is based on a one download process, unlike GDL which is a multi download process. I explain. If 2 downloads start on the same file (file size = S), one (X) will start at the beginning of the file (offset 0) and the second (Y) at the offset S/2. The first problem is to be sure that X[S/2]=Y[S/2]. How to do this ? open a 3rd transfer on the same file as X and download the data at S/2 position ??? a bit heavy and moreover, it will slow a lot the download, not because there is 3 running xfers but because it requires 2 slots on the same client (it is already hard to have 1). By using only 2 connections, why not, but this mean we have to open X at position S/2, download few bytes, close the connection, open X again at position 0 and start the download. This is required because DC protocol has no command to download a bloc at a given position with a given size. Because the connection is closed, the upload slot is released and may be used by another client.

The best solution (adopted by lot of other p2p and used by DCTC has protocol extension) is to compute a checksum on the file. This checksum provides 2 things:
- the most important thing is to be sure the transfered file is identical to the original one. Ethernet specification says there have at most 1 erroneous bit for 10^12 bits (~116GB) and the internet carries a lot more than 116GB a day, this means you may encounter numerous corrupted file. Some file (mp3, mpg, divx) accepts errors as long as the error does not appear at critical position (data header, ...)
- a finger print to identify nearly surely a file (collision probability is very very very low).

DCTC uses the same checksum algorithm as e-donkey because it is fairly fast and provides the data to identify the corrupted blocs and to identify file (2 level MD4 algorithm).

Eric PREVOTEAU <ericprev>
Project AdministratorIn charge of this item.
Sat 18 Oct 2003 04:15:26 PM UTC, original submission:

Would it be possible to add integrity checking when stopped download is started to continue possibly from another source by
downloading of few KB of end of already downloaded part?

Let's say that we have downlaoded first 200KB of the file, then dctc was stopped or source disconected us, so if we continue
with the download, we could validate the new source not only by the same size, but also part of the content, for example 16
KB, and compare the last 16KB of previously downlaoded part and 16KB from new source and if they match, continue, otherwise
error.

How many KB to check could be set by option.

I think this is quite important feature worth to be implemented.

Thanks a lot:-)

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

CC list is empty

 

Do you think this task is very important?
If so, you can click here to add your encouragement to it.
This task has 0 encouragements so far.

Only logged-in users can vote.

 

Please enter the title of George Orwell's famous dystopian book (it's a date):

 

 

Follow 2 latest changes.

Date Changed By Updated Field Previous Value => Replaced By
Sat 15 Nov 2003 02:45:06 PM UTCericprevAssigned toNone=>NA
  Open/ClosedOpen=>(Error - Not Found)

Back to the top


Powered by Savane 3.1-cleanup1