Mon 21 Feb 2005 09:38:54 PM UTC, original submission:
In the initial version of epoch support, we simply send all epochs in a big lump at the beginning of each netsync. This actually scales not too badly, since there aren't that many branches in a typical netsync, and we send <40 bytes per branch. But in the long run, it will probably be a win to use merkle refinement for this; this removes the O(n) cost for no-op netsyncs.
The tricky part of doing this is in the server -- one might think we could just start doing pipelined, parallel refinement as usual, and then if it turned out that epochs mismatched, abort out, no problem, let database rollback take care of making sure nothing bad happened.
However, this doesn't work in the server -- the server commits after every netsync packet, because rollback is an app-wide operation; the server can't rollback one connection's data without killing all other simultaneous connections.
One way to solve this would be to wrap our packet_consumer in one that "has a valve" -- have it just queue up packets in memory until epoch refinement finishes, at which point we can "open the valve", and actually send packets towards the db. This should be very very cheap, since the way netsync is structured, we already queue up all our packets in memory for a long time; the valve will probably open before we would have written anything to disk anyway.
|