Improving data consistency management and overlay multicast in internet-scale distributed systems
Hong Jiang, Ying Lü, Yijun Lu · 2007
In the past decade, the Internet has become the dominant platform for doing research and business. Compared to other platforms, the Internet promises to manage shared data efficiently and deliver the data to the consumer/user in real-time and in a transparent way. A wide variety of applications, including online collaboration and e-business, have already been implemented on the Internet. However, providing services on the Internet poses two challenges. First, an Internet-based collaboration often has to replicate the shared data in many geographically distant locations to improve efficiency and availability. This replication makes the shared data difficult to manage. Second, the Internet is usually loosely connected due to its autonomous and decentralized nature and thus the transmission is inherently unreliable. Even when a message gets delivered ultimately, it is not uncommon for the message to be delayed for a significant amount of time. These challenges significantly hinder effective data management and efficient data delivery. In this dissertation research, we focus on improving data consistency management, from the data management perspective, and overlay multicast, from the data delivery perspective, in Internet-scale distributed systems. The current data consistency management mechanisms often enforce a predefined consistency level, which cannot cater to the needs of multiple applications with different consistency requirements and incur high communication cost when there are a large number of participants. To improve data consistency management, we first propose IDF (Inconsistency Detection Framework) that provides a unified framework to maintain adaptive consistency control for online distributed collaborations; second, we propose CVRetrieval (Consistency View Retrieval), a new system that greatly improves the scalability of consistency control by reducing the communication cost. These two systems tackle the consistency control problem from two angles: IDF provides adaptive consistency control and CVRetrieval improves the scalability of consistency control. Together, they help scale a variety of distributed online collaboration applications to the Internet level. To encourage participation of online collaboration and e-business, a main focus of current research on overlay multicast is to improve the fairness among all participants' contributions. However, current mechanisms to enforce fairness only considers one multicast session. In this dissertation, we propose FairOM (Fair Overlay Multicast) that provides a new definition to fairness in overlay multicast to support multiple simultaneous multicast sessions. FairOM hence enables scalable, fair, and high performance overlay multicast of shared data in an Internet-scale distributed system. IDF, CVRetrieval, and FairOM are fully evaluated through analysis, simulation, and deployment of prototypes on Planet-Lab, a widely used test-bed for distributed Internet services that is constructed on top of the Internet. The results show that (1) IDF achieves an adaptive and efficient consistency maintenance that can benefit a wide variety of applications; (2) CVRetrieval greatly improves consistency control by removing passive participants from the expensive consistency maintenance protocol and instead satisfies their needs on demand; and (3) FairOM can support more simultaneous multicast sessions than other protocols without sacrificing significant multicast performance and incurring unacceptable communication overhead.