bugmonotone - Bugs: bug #12080, epoch netsync is somewhat...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #12080: epoch netsync is somewhat inefficient

Submitted by:  Nathaniel Smith <njs>
Submitted on:  Mon 21 Feb 2005 09:38:54 PM UTC  
 
Category: networkingSeverity: 1 - Wish
Item Group: NoneStatus: None
Privacy: PublicAssigned to: None
Open/Closed: Closed
mtn version --full: 

Mon 21 Feb 2005 09:38:54 PM UTC, original submission:

In the initial version of epoch support, we simply send all epochs in a big lump at the beginning of each netsync. This actually scales not too badly, since there aren't that many branches in a typical netsync, and we send <40 bytes per branch. But in the long run, it will probably be a win to use merkle refinement for this; this removes the O(n) cost for no-op netsyncs.

The tricky part of doing this is in the server -- one might think we could just start doing pipelined, parallel refinement as usual, and then if it turned out that epochs mismatched, abort out, no problem, let database rollback take care of making sure nothing bad happened.

However, this doesn't work in the server -- the server commits after every netsync packet, because rollback is an app-wide operation; the server can't rollback one connection's data without killing all other simultaneous connections.

One way to solve this would be to wrap our packet_consumer in one that "has a valve" -- have it just queue up packets in memory until epoch refinement finishes, at which point we can "open the valve", and actually send packets towards the db. This should be very very cheap, since the way netsync is structured, we already queue up all our packets in memory for a long time; the valve will probably open before we would have written anything to disk anyway.

Nathaniel Smith <njs>
Project Member

 

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

CC list is empty

 

Do you think this task is very important?
If so, you can click here to add your encouragement to it.
This task has 0 encouragements so far.

Only logged-in users can vote.

 

Please enter the title of George Orwell's famous dystopian book (it's a date):

 

 

Follows 1 latest change.

Date Changed By Updated Field Previous Value => Replaced By
Sat 05 Mar 2005 11:13:56 AM UTCnjsOpen/ClosedOpen=>Closed

Back to the top


Powered by Savane 3.1-cleanup1