Searching XML data by SLCA on a MapReduce cluster
Mengjie Zhou, Haoji Roland Hu, Minqi Zhou · 2010
XML keyword search is a popular topic in research field, and the Smallest Lowest Common Ancestor (SLCA) concept is fundamental for XML keyword search algorithms. With the rapid growth of XML data in internet, we are confronted with big data issues, it's becoming a new research direction for managing massive XML data now. Conventional centralized data management technologies are limited in the aspects of efficiency, throughout and maintenance cost. MapReduce framework is a recent trend to process large-scale data. It is implemented on clusters built by numbers of business machines, to conquer limitations mentioned above by parallel computation. In this paper, we provide a SLCA-based keyword search implementation for large-scale XML data sets on a MapReduce cluster. Main steps of our implementation include XML data partition, parse and sort, index setup and SLCA computation. We conduct some experiments to evaluate the effectiveness of the proposed method.