Distributed high-dimensional index creation using Hadoop, HDFS and C++
Gylfi Por Gudmundsson, Laurent Amsaleg, Björn Þór Jónsson · 2012
This paper describes an initial study where the open-source Hadoop parallel and distributed run-time environment is used to speedup the construction phase of a large high-dimensional index. This paper first discusses the typical practical problems developers may run into when porting their code to Hadoop. It then presents early experimental results showing that the performance gains are substantial when indexing large data sets.