Performance improvement of multi-dimensional indexing system for big data analysis
Kazuto Nakanishi, Teruhisa Hochin, Hiroki Nomiya · 2016
This paper proposes the methods for improving performance of multi-dimensional index system. In the previous system, data are inserted into a reception-node, and are stored in a reception-index. When a reception-index stores a certain number of data, the reception-index is moved to a normal-node, and becomes a partial-index. At this time, insertion is paused. In addition, if the numbers of data in a normal-node are skewed, the retrieval performance degrades. This paper proposes an architecture enabling data to be inserted without a pause. Moreover, when a reception-index is moved to a normal-node, all normal-nodes are tried to store partial-indexes equally. It is experimentally clarified that the insertion time decreases by the proposed architecture. It is also clarified that partial-indexes can be distributed equally based on the proposed method, and that the retrieval performance can be improved.