On investigating scalability and robustness in a self-organizing retrieval system
Jan Sedmidubský, Vlastislav Dohnal, Pavel Zezula · 2011
We introduce a self-organizing similarity search system for a large-scale unstructured peer-to-peer network, called the Metric Social Network. This system does not rely on any centralized control and does not define any policy for distributing data to individual peers. It combines multiple strategies into a single system which results in abilities to scale to a large number of peers, to adapt to different data distributions, and to be robust to abrupt peer disconnections. We demonstrate these abilities by running various experimental trials on a real-life, as well as, a synthetic data set stored on up to 2,000 peers. Additionally, different data distributions among the peers, ranging from clustered to totally non-clustered and real-life data distributions, are also considered.