PERFORMANCE STUDY OF LUCENE IN PARALLEL AND DISTRIBUTED ENVIRONMENTS
Kishore Sajja · 2011
Performance study is a common way to assess the usability of a component in a software application. Lucene is a library for information retrieval. For this project, we are going to study the performance of some available searching features of Lucene in commonly available environments: serial, distributed and parallel. However there is no information available as to how various features perform in these environments. In this project we compare distributed searching versus parallel searching using Lucene. Distributed searching is easily implemented with the Remote Method Invocation hooks provided by Lucene. Parallel implementation requires more work. The hypothesis was that if Lucene performs reasonably well in distributed mode then we don’t need a parallel implementation. We found that Lucene in distributed environments using the RMI hooks did not perform reasonably as well as expected in our experiment setup. This is because communication overhead dominates the actual searching time in the distributed environment. Next we experimented with Lucene in a parallel implementation. We designed the experiment to keep the overhead from communication and other system operations at a minimum. We got reasonably good speedups for this setup.