secushare Scalability

secushare Scalability

Video summary from ChaosWest: Scalable and Privacy-Respectful Distributed Systems - Our Chance to Avoid Cloud Computing?

Scalability by Multicast

Once we have a first iteration of this technology running, we can move on to add the extras that make this fully scalable. We already laid out the architecture pretty well, but it still makes sense to use multicast distribution trees (not to be confused with IP Multicast or LAN Multicast!) to make sure distribution of popular data will not get stuck in bottlenecks.

Our PSYC project has gained some experience with application-level multicast, so it would be the easiest to deploy a multicast overlay over the routing technology as we have it, by using PSYC on top of it. That's what our new pubsub and multicast APIs for GNUnet do.

Next we can dig deeper and optimize the routing technology to support such distribution trees the best and also in a safest possible way: We don't want the structure of our distribution trees to be visible to an observer. At the same time user interface designers can start making our dreams visually come true.

So this is the step that is likely to yield humanity-wide scalability. Unfortunately we cannot provide mathematical proof, we're just talking from experience so far. And it isn't rocket science really, after all BitTorrent and cloud technology scale in similar ways.

Protocol efficiency

Of course we shouldn't be wasting CPU power parsing notoriously inefficient formats like XML or JSON. It's not a bottleneck, but since we want to be energy-efficient also on small devices, let's pick a protocol that performs better under such conditions.