bugmldonkey, a multi-networks file-sharing client - Bugs: bug #8961, QoS, dynamic connection limiting...


You are not allowed to post comments on this tracker with your current authentication level.

bug #8961: QoS, dynamic connection limiting so we don't flood our connections

Submitted by:  None
Submitted on:  Fri 14 May 2004 06:25:18 PM UTC  
Category:  Core Severity:  3 - Normal
Item Group:  Feature request Status:  Invalid
Assigned to:  None Open/Closed:  Closed
Release:  Release:  None
Operating System:  None Binaries Origin:  CVS / Self compiled
CPU type:  None

Sun 16 Jul 2006 12:52:03 AM UTC, comment #5: 

This is not really a bug report, besides being from the stone-age;-)

spiralvoice <spiralvoice>
Project Administrator
Mon 18 Apr 2005 11:34:49 AM UTC, comment #4: 


spiralvoice <spiralvoice>
Project Administrator
Mon 18 Apr 2005 10:44:22 AM UTC, comment #3: 

Could sonmebody make 12471 public?

Knocker <knocker>
Sat 26 Mar 2005 08:20:48 PM UTC, comment #2: 

Bug #12471 provides a technically better solution for outbound saturation than does hard rate capping.

Mark Schreiber <mark7>
Mon 07 Jun 2004 02:17:02 PM UTC, comment #1: 

Follow up:

Network resource limitation should be tight in the application (not externally via packet queuing)

I. Rate Limiting:
A. Sending (outgoing):
1. responding to requests
2. promoting shared files
3. data
B. Receiving (incoming):
1. requests
2. answers to requests
3. data
C. Intelligent Bandwidth Determining
II. Misc Features:
A. Chunk hiding (so it looks like we don't have the full file...)
B. Robust IP blocking (certain IPv4 blocks, evil p2p ips, banned, etc)
C. Robust banning (name, ip, cipher per object blocked)
D. Prefer newer lugdunum servers (add extra rating to these)
E. Optional file ACL (allow which protocols to download, etc)

I.  Rate Limiting

Rate limiting should be implemented in the application rather than from an external packet queuer since some packets are more important than others, and rate limiting in the packet queuer may only see the differences in packet sizes or ports, while the application can see more.  We can also attempt to avoid flooding the connection if we already are utilizing the connection to its maxmimum extent by not sending more requests than we need.

A.  Sending (outgoing):

This is the communication / data going out of our computer, to another computer.  We should focus on simplicity yet prioritizing different type of communication.

Types listed by importants:

1.  responding to requests - source exchanging, adding to queues, etc (don't drop packets here)
high priority, we want clients to be able to contact us
2.  promoting shared files - so people will download from us (p2p only, since only once for client-server )
only promote every xx minutes according to the official client
-flexible scheduling based on connection speed and business?
experiment with promoting files that aren't being requested a certain number of times
-possibly hook this in with whether the local file is complete
-whether the network has a complete version somewhere
3.  sending data - only if bandwidth on the client is available
FIFO queuing of clients
trimming (IF_PINGABLE = TRUE, TCP/UDP/ICMP, etc) -- probably not required, but cleanliness is nice
intelligent slotting:
we give priority to the first slotted... if we have extra bandwidth it goes to this guy
if we have any extra upload bandwidth, lets create a new slot and give it to first in queue, etc
if we are running out of upload bandwidth, lets cut out the lowest slot and put him first in queue

B.  Receiving (incoming):

Types listed by importants:

1.  requests (don't drop packets here)
high availability and reliability are key
2.  answers to requests (don't drop packets here)
these are important so we get max sources, reliable responses, etc
3.  receiving data (drop excess packets here)
while its nice to grab packets, they are coming in too fast so we drop them accordingly
intelligent slotting:
we give priority to the file with the highest priority number (drop packets with the lowest priority)
if we have extra download bandwidth we create a new slot
if we max out our download and have extra slots, get rid of a slot
intelligent chunk selection:
download the rarest chunks from clients first

C.  Intelligent Bandwidth Determining

(Fuzzy, so it'll adapt to each user's connection)
This is important so we don't need to rely on the manually imposed limits by the users of the clients.  Hard limits set by users should be obeyed (should be 90% of connection).  Fuzzy limit should be dynamic.  We know we are flooding the connection when ???. We know our connection is not being fully utilized when ???.  We know we are in the "sweet spot" when ???.

We should save bandwidth info to file when we close the client...  Load on startup too.

I'm thinking we can base '???' off ping which may or may not be as realiable as we want...
We'd want to make sure we check our ping when we start the application... But what if the connection is already being used by some other application... Then we may not have a very good ping to go by... Maybe average too?
We increase the fuzzy limit when ping is low... bigger the connection -> bigger the increase
We decrease the fuzzy limit when ping is too high... bigger the connection -> bigger the decrease

We should also do checks to see if the hard limit seems correct (check if seemingly unused [can let the user know we can go faster maybe])

We could probably base more network tuning options off these as well.

Fri 14 May 2004 06:25:18 PM UTC, original submission:  

Control packets and other non-"data" packets should have priority over data-transfer packets.  SYN (connecting), ACK (acknowlege of a data packet), SACK, FIN, etc, packets should have a higher priority than normal data-transfer packets.  Packets to and from servers (control messages, searching, retrieval of sources) should be last to be dropped from rate limiting.  Soft and Hard limits for downloading... Hard would be manually set.  Soft would be an intelligently determined (auto-magically) limit.  When we reach our soft limits we shouldn't initiate any more download connections (unless we can get rid of the slow ones and replace them with faster more stable ones?).  Anyways, this is just an overview of what I have written down somewhere, expect a follow up.  =P



No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


CC list is empty


Do you think this task is very important?
If so, you can add your encouragement to it.
This task has 0 encouragements so far.

Only logged-in users can vote.




Follow 2 latest changes.

Date Changed by Updated Field Previous Value => Replaced by
2006-07-16 spiralvoice StatusNone => Invalid
    Open/ClosedOpen => Closed

Back to the top

Powered by Savane 3.5