Fault-Tolerant Distributed Information Retrieval for Publius Servers and Mobile Peers
K. M. Hanna, Brian Neil Levine, Raghaven Manmatha · 2002
Text of abstract We show how dividing a database and replicating documents and indicies in an overlapping manner provides resilience in the face of node failures, malicious attacks, censorship attempts, and network partitions. This property of fault tolerance is beneficial for a wide range of scenarios, and we examine it in the context of two applications: an IR collection distributed over a set of mobile peers with wireless interfaces; and an IR collection distributed over servers set up to support censorship-resistant peer-to-peer file sharing and web publishing systems, such as Publius. Our use of random replication and split document sources makes it difficult for attackers to remove specific indexed content from the system. Moreover, we show the system is able maintain high IR performance even when 45 out of 50 nodes are unavailable. For mobile nodes, we have show that our design manages the randomness of mode mobility. Nodes are able to contact only direct neighbors who change frequently, not use ad hoc routing protocols, and still maintain good IR performance. This makes our design applicable to mobility situations where routing partitions are common. Our evaluation show nodes require only low additional storage on average.