Scaling content-based services using p2p technology
Purvi Shah · 2008
Peer-to-Peer (P2P) technology overcomes many limitations of traditional client-server architectures by letting clients serve each other. As a result, it has been widely used to build inexpensive self-scalable distributed systems that can serve very large populations of end-users. In our research we address the challenges in P2P systems to implement services difficult to deploy at the network (IP) level. We first analyzed P2P systems for the distribution of large videos. We observed that P2P systems scale much better than standard client-server architectures, and can potentially significantly decrease the cost of distributing videos to many peers simultaneously. We observed the behavior of the BitTorrent (BT) system, and showed that the downloading time scales logarithmically in the number of peers. We proposed a modified solution to BT based distribution mechanism that focuses on locality awareness and includes the ability to manage the bandwidth usage over expensive or constrained links. Next, we focus on the two main limitations of existing BT system, namely, their vulnerability to malicious and selfish attacks, and their very poor performance in streaming applications. The proposed solutions include a new data chunk selection and distribution policy, a new neighbor selection policy and a partially decentralized trust management policy. We also examined the performance of other P2P streaming systems namely, a directed mesh letting peers dynamically replace poorly performing parents. Finally, we present a content delivery infrastructure for distributing and maintaining software packages in a large organization. Our work is based on a trace-based analysis of an existing software delivery system that we conducted to find general principles and properties that could be used to devise a better solution. Our proposal consists of supplementing a conventional server with volunteer nodes that expand its scalability. Our design is unique in that it combines the concept of volunteering with the P2P technology. We rely on P2P technology to speed up content synchronization among the volunteer nodes while maintaining a conventional client/server interface for the service customers. Our system includes a novel load balancing mechanism that considers both the synchronization workload and the customer-generated workload of the volunteer nodes.