An Efficient Multidimension Metadata Index and Search System for Cloud Data
Yang Yu, Yongqing Zhu, Willie Ng, Juniarto Samsudin · 2014
The ever increasing amounts of digital data being stored in public and private clouds are challenging users to access and manage the data. With the corresponding storage system reaches Petabyte-scale, or even Exabyte-scale, metadata access will become a severe performance bottleneck. Hence, this paper proposes an efficient multi-dimensional metadata index and search solution for cloud data. By proposing some new mechanism for K-D-B tree based index/search and implementing index partitioning technique, our system can achieve optimized performance in terms of memory utilization and search speed. Experiments show that our system performs much better as compared with existing solutions. In addition, our system can safely scale out in a distributed manner with guaranteed performance.