Scalability challenges and solutions for emerging networks
Ken Birman · 2002
Summary form only given. Computer networks are becoming increasingly common, and are used in sensitive applications in which serious damage could be done by a network failure. There is a need for design principles that would enable a new generation of solutions having the required properties. Needed are technologies that would be inherently robust, provably scalable, and sufficiently self-organizing to adapt as conditions change in the network. The Spinglass project has been successful in solving an important class of such problems. At the core of our work is a new style of gossip-based communication protocol. We are using this protocol in support of a variety of systems programming tools. The article discusses two of them: Bimodal Multicast, a scalable reliable multicast protocol having probabilistic reliability properties, and Astrolabe, a virtual distributed database constructed entirely through peer-to-peer interactions among the components of a large system. Both technologies are shown to be stable under stress, arbitrarily scalable without growth in communication or processing loads, and to have real-time properties.