patchmldonkey, a multi-networks file-sharing client - Patches: patch #7448, BT: support for partial download

 
 

You are not allowed to post comments on this tracker with your current authentication level.

patch #7448: BT: support for partial download

Submitter:  Joakim Verona <jave>
Submitted:  Sat 22 Jan 2011 10:57:35 PM UTC
   
 
Category:  None Priority:  7 - High
Status:  Done Assigned to:  ygrek
Open/Closed:  Closed

Jump to the original submission

Mon 31 Jan 2011 05:38:49 PM UTC, comment #30: 

For later usage here is the not-committed white-space part of prio19.patch

(file #22574)

spiralvoice <spiralvoice>
Group administrator
Mon 31 Jan 2011 05:32:03 PM UTC, comment #29: 

committed patch without white-space part 2011/01/31

spiralvoice <spiralvoice>
Group administrator
Sun 30 Jan 2011 09:44:57 PM UTC, comment #28: 

Added progress indicator for each subfile, disabled debug output.

(file #22561)

ygrek <ygrek>
Group Member
Sat 22 Jan 2011 10:57:35 PM UTC, comment #27: 

This item has been reassigned from the project mldonkey, a multi-networks file-sharing client task tracker to your tracker.

The original report is still available at task #4755

Following are the information included in the original report:

[field #0]                  Item ID: 4755
[field #1]                 Group ID: 1409
[field #2]              Open/Closed: Open
[field #3]                 Severity: 3 - Normal
[field #4]                  Privacy: Public
[field #6]                    Votes: 335
[field #9]                 Category: Bittorrent-Plugin
[field #10]             Submitted by: jave
[field #11]              Assigned to: ygrek
[field #12]             Submitted on: Fri 07 Oct 2005 01:11:40 PM GMT
[field #13]                  Summary: BT: support for partial download
[field #14]      Original Submission: heres a little implementation plan:

- get all chunks for a file from its "swarmer" see chunk status
printing

- add "want to download" chunk list to swarmer, add it to chunk
download prioritizing logic, never ask for unintersting chunks.

- since we newer download unwanted chunks, we wil not faultily claim we have them to peers either

- when chunks are downloaded they ought to wind up in their correct file as usual. space will be wasted during download, because ill keep
space for all files anyway, and chunks spanning file boundaries will
go into the unwanted files. this is no big deal i think. and its good
if we change our mind of which files to download while downloading.

- make a command that sets the "want to download" chunk list given a
partial file id, something like:

want_partial_file A B C
where A is themain download id number , B is the sub-file number, and
C is the true/false if we want the file or not.


[field #15]                Closed on: Wed 14 Jan 2009 01:28:08 PM GMT
[field #16]               Item Group: None
[field #17]                   Status: Ready For Test
[field #18]        Component Version: None
[field #19]         Operating System: None
[field #20]          Reproducibility: None
[field #21]               Size (loc): None
[field #22]            Fixed Release: None
[field #23]          Planned Release: None
[field #24]                   Effort: 0.00
[field #28]                 Priority: 7 - High
[field #29]          Should Start On: Fri 07 Oct 2005 12:00:00 AM GMT
[field #30]    Should be Finished on: Fri 07 Oct 2005 12:00:00 AM GMT
[field #31]         Percent Complete: 0%
[field #33]                  Release: None
[field #58]     Custom Select Box #1: None
[field #59]     Custom Select Box #2: None
[field #60]     Custom Select Box #3: None
[field #61]     Custom Select Box #4: None
[field #62]     Custom Select Box #5: None
[field #63]     Custom Select Box #6: None
[field #64]     Custom Select Box #7: None
[field #65]     Custom Select Box #8: None
[field #66]     Custom Select Box #9: None
[field #67]    Custom Select Box #10: None

ygrek <ygrek>
Group Member
Sat 22 Jan 2011 10:55:57 PM UTC, comment #26: 

Patch updated to make html interface useable. Partial files are not commited - imho this is the only sane solution in case user wants to download some more files later. Location in temp dir is shown so that user can easily locate downloaded subfile.

ygrek <ygrek>
Group Member
Thu 13 May 2010 07:10:33 PM UTC, comment #25: 

prio17 - fix compilation with Makefile (I was using ocamlbuild which automatically discovered dependencies and didn't notice breakage), thx Paranoik.

(file #20532)

ygrek <ygrek>
Group Member
Wed 12 May 2010 10:01:38 PM UTC, comment #24: 

version 16:
- store/restore priority intervals from settings
- hide debug_set_download_prio command
- add rudimentary html interface to manage subfile priorities
- convert tabs to spaces in code

NB Seems like partial downloads do not get commited


(file #20526)

ygrek <ygrek>
Group Member
Mon 05 Apr 2010 09:07:20 PM UTC, comment #23: 

This patch badly needs testing and feedback!

ygrek <ygrek>
Group Member
  Spam posted by oyunlar35
Mon 12 Jan 2009 07:08:47 PM UTC, comment #21: 
Invalid User ID <#71779>
Mon 12 Jan 2009 07:08:25 PM UTC, comment #20: 
Invalid User ID <#71779>
Sun 11 Jan 2009 10:52:31 AM UTC, comment #19: 
Invalid User ID <#71767>
Sun 11 Jan 2009 10:51:41 AM UTC, comment #18: 
Invalid User ID <#71767>
Sun 11 Jan 2009 10:48:48 AM UTC, comment #17: 
Invalid User ID <#71767>
  Spam posted by fctom
  Spam posted by dimon1
Sun 16 Dec 2007 11:40:30 AM UTC, comment #14: 

so.. noone working on this i suppose?

Anton Romanov <theli>
Mon 18 Jun 2007 07:17:16 AM UTC, comment #13: 

this patch is broken after patch #6005

unease <unease>
Thu 14 Jun 2007 08:27:37 AM UTC, comment #12: 

hi I tested this patch (waiting for this so long)... but it seems priorities gone after mlnet kill... And also If I have some time I would like to make some cosmetic HTML corrections to this patch... Should I wait for apply ? or can I change now ?

unease <unease>
Sun 27 Aug 2006 02:48:41 PM UTC, comment #11: 

Minimally updated for CVS.

With WIP10 commited to CVS, I hope I didn't forget anything :/

pango <mlpango>
Group Member
Wed 28 Jun 2006 10:35:23 PM UTC, comment #10: 

Updated for CVS (minimal update, I hope everything's alright)

pango <mlpango>
Group Member
Sun 11 Jun 2006 11:49:42 AM UTC, comment #9: 

Updated for current CVS HEAD

pango <mlpango>
Group Member
Wed 07 Jun 2006 08:46:29 AM UTC, comment #8: 

Updated for current CVS

pango <mlpango>
Group Member
Sat 27 May 2006 07:52:21 AM UTC, comment #7: 

pango has updated the prio patch for wip4a, which is now in CVS.
This means the prio patch now applies over plain cvs.

This version also adds logic to make the priority bitmap be
recalculated on "merge" commands.

Note that prio '0' blocks will never download, but lower prio blocks
MIGHT download before higher prio blocks, if no lower prio blocks were
available at the time of block selection.

Also note that the UI will definitively change, probably in way to be
compatible with main file priorities.

The current patch has only recevied light testing.

Joakim Verona <jave>
Group Member
Fri 12 May 2006 11:36:17 AM UTC, comment #6: 

v8 should apply to wip3e'.
It is also more efficient, because the linear scan is avoided.


Joakim Verona <jave>
Group Member
Thu 20 Apr 2006 01:57:27 PM UTC, comment #5: 

This 6th version of the prio patch supports:
- setting of priority on individual blocks
- setting of priorities on subfiles in a multifile torrent
- better view of subfile download progress

I renamed the commands:
debug_set_download_prio - for blocks
debug_get_download_prio - return block prio and download status
debug_set_subfile_prio  - operate on subfile level instead

Please read the help strings for the commands for details.

You must apply pango:s WIP3C patch before! WIP3C contains an important
bugfix in commonswarming, and the prio patch wont work without it.
(this is of course true only until WIP3C is applied to cvs)

the prio patch is furthermore tested with the "multiuser" patch, but
probably works withouth it as well.


there are some currently known problems:
- if you "merge" 2 downloads, the priority system will most probably
crash

- block priorities doesnt survive core restarts

- a file is commited as soon as all non-zero prio blocks are
downloaded. This means that its possible for junk to be commited. You
are expected to know about this and remove junk files yourself. I'm
not sure I'm interested in "fixing" this, since there are no good
solutions in the general case.

Also, the implementation isnt the most efficient, but in practice it
doesnt seem to affect performance much for me at least. (there is a
linear block scan to determine if a block should be downloaded or not)

Furthermore the previous "download first" code, which hardcoded high
priorities for the first and last blocks of a file is removed by this
patch.

you can recreate the behaviour by setting higher prios for those
blocks with the commands.

Finally a warning: its not obvious to me that its good for the
networks and/or your average download speed to mess with block
priorities. Is it, for instance "fair" to download a single file of a
multifile torrent and then jump out?

In practice it appears to work well though.





Joakim Verona <jave>
Group Member
Wed 12 Apr 2006 07:12:37 PM UTC, comment #4: 

better bt subfile view support

Joakim Verona <jave>
Group Member
Mon 10 Apr 2006 12:01:37 PM UTC, comment #3: 

new patch applies to plain cvs as of today

Joakim Verona <jave>
Group Member
Sun 09 Apr 2006 09:19:16 PM UTC, comment #2: 

this 1st version of the patch is already usable in a primitive way.

for instance:

dllink lalala.torrent
(gets id 528)

debug_set_download_priorities 528 666666666666666666655555555555555555555555555555555554444444444444444444444444444444433333333333333333333333333333333333222222222222222222222222222222222221111111111111111111111111111111111

(the long string matches the chunk bitmap you see with "vd 528" in
telnet interface. the numbers are priorities. 6 will go before 5
etc. 0 wont download at all)

i tried a couple of files and it seems to work nicely. good for
listening to mp3:s while they are dowloading.

currently you have to figure out which chunks correspond to a certain
file in a bt multipart if thats what you want to do.


Joakim Verona <jave>
Group Member
Thu 06 Apr 2006 09:51:59 AM UTC, comment #1: 

same as #4851

Joakim Verona <jave>
Group Member
Sat 22 Jan 2011 10:57:35 PM UTC, original submission:  

heres a little implementation plan:

- get all chunks for a file from its "swarmer" see chunk status
printing

- add "want to download" chunk list to swarmer, add it to chunk
download prioritizing logic, never ask for unintersting chunks.

- since we newer download unwanted chunks, we wil not faultily claim we have them to peers either

- when chunks are downloaded they ought to wind up in their correct file as usual. space will be wasted during download, because ill keep
space for all files anyway, and chunks spanning file boundaries will
go into the unwanted files. this is no big deal i think. and its good
if we change our mind of which files to download while downloading.

- make a command that sets the "want to download" chunk list given a
partial file id, something like:

want_partial_file A B C
where A is themain download id number , B is the sub-file number, and
C is the true/false if we want the file or not.


Joakim Verona <jave>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #22574:  7448_3.patch added by spiralvoice (88KiB - text/x-diff)
file #22561:  prio19.patch added by ygrek (114KiB - text/x-patch)
file #22496:  prio18.patch added by ygrek (110KiB - text/x-patch)
file #20532:  prio17.patch added by ygrek (110KiB - text/x-diff)
file #20526:  prio16.patch added by ygrek (110KiB - text/x-diff)
file #13094:  prio15.patch added by spiralvoice (16KiB - text/x-diff - updated for current CVS)
file #12948:  prio14.patch added by spiralvoice (15KiB - text/x-diff - updated for MLDonkey 2.8.7)
file #10604:  prio13.patch added by mlpango (15KiB - text/x-patch)
file #10265:  prio12.patch added by mlpango (15KiB - text/x-patch)
file #10161:  prio11.patch added by mlpango (15KiB - text/x-patch)
file #10137:  prio10.patch added by mlpango (14KiB - text/x-patch)
file #10062:  prio9.patch added by jave (13KiB - application/octet-stream)
file #9947:  prio8.patch added by jave (11KiB - application/octet-stream)
file #9808:  prio_6.patch added by jave (11KiB - application/octet-stream)
file #9772:  prio_4.patch added by jave (8KiB - application/octet-stream)
file #9768:  prio_3.patch added by jave (8KiB - application/octet-stream)
file #9753:  prio_2.patch added by jave (6KiB - application/octet-stream)
file #9749:  prio.patch added by jave (5KiB - application/octet-stream)

 

Depends on the following items: None found

Digest:
   task dependencies.

 

Carbon-Copy List
  • -email is unavailable- added by balamutick (Voted in favor of this item)
  • -email is unavailable- added by ygrek (Posted a comment)
  • -email is unavailable- added by kindacute (Voted in favor of this item)
  • -email is unavailable- added by oyunlar35 (Posted a comment)
  • -email is unavailable- added by lapo (Voted in favor of this item)
  • -email is unavailable- added by karl (Updated the item)
  • -email is unavailable- added by fctom (Posted a comment)
  • -email is unavailable- added by dimon1 (Posted a comment)
  • -email is unavailable- added by loc (Voted in favor of this item)
  • -email is unavailable- added by theli (Posted a comment)
  • -email is unavailable- added by theli
  • -email is unavailable- added by unease (Posted a comment)
  • -email is unavailable- added by spiralvoice (Updated the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2011-01-31 spiralvoice Attached File- Added 7448_3.patch, #22574
    2011-01-31 spiralvoice StatusReady For Test Done
        Open/ClosedOpen Closed
    2011-01-30 ygrek Attached File- Added prio19.patch, #22561
    2011-01-22 ygrek StatusNone Ready For Test
        Assigned toNone ygrek
    2011-01-22 ygrek Dependencies- task #4755 is dependent
    2011-01-22 ygrek Reassign itemmldonkey, a multi-networks file-sharing client, task #4755 mldonkey, a multi-networks file-sharing client, patch #7448

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code