When is fanout, not....

A feature of Groove is its ability to fanout data. As an affecionado I thought I understood the fanout concept, but it turns out I had it upside down. To me, fanout should naturally refer to Groove's ability to send the same data to all peers with whom it needs to communicate. Nothing more, and nothing less. Whereas in Jon Udell's interview of Ray Ozzie nearly four years ago (http://www.openp2p.com/pub/a/p2p/2000/10/24/ozzie_interview.html) more properly defines fanout as the indirect distribution of that same data via a certralised server - a relay server.

This difference was drawn to my attention recently as a result of our Strong Angel II participation. Scott Silvasy provides the insight, "Groove's communications service senses network connectivity and has a general idea of the connection speed. The service also works with the Relay Server to determine if fanout should/could be used as a more efficient transport method. Groove's dynamics service determines the change in tool deltas (i.e., additions, modification, or deletions of Groove tool data) and packages it as either a simple command (e.g., in SketchPad and Co-Edit of Word documents) or as a binary difference (e.g., modification to a file in the Files Tool) -- the security service encrypts and signs these delta's. Since we know the number of end-points to receive the delta, the connection speed, and the approximate connection speed, Groove uses an algorithm to determine if it is more efficient to send the delta to the Relay Server and have the Relay Server propagate the change (i.e., fanout), or to function "normally" where the client sends the delta's to all end-points."

So we can deduce from this that Groove can automatically detect & monitor connection speed and make decisions about fanout, dependent upon that information. Obviously such decisions depend on the quality of the algorithms, and needless to say these are under constant review. However there are limits to this optimisation: "You cannot squeeze a quart out of a pint pot," as the saying goes. Or can you?

If Groove Networks can move their relay code away from the current server product and embed more of it into the client application, then one can begin to attain an ideal position where the client can act as relay. The client can do the fanout at a much more granular level. The result is a self-balancing, self-tuning system that make the most of bandwidth as and when it occurs.

As an ideal I think it will remain a significant challenge for some time. Until then there is always PopG... With PopG the user can elect to make optimisation decisions on behalf of Groove: one can manipulate the fanout architecture by placing your Groove client into the bandwidth category of your choice. As with relay architecture this is a pragmatic decision that will suit people manipulating large datasets, possibly with large numbers of space members or where those members have their accounts & spaces replicated across several computers.

posted by Andy Swarbrick/PopG at 4:37