nbodykit
Nick Hand, Yu Feng · 2017
We present nbodykit, an open source, massively parallel Python toolkit for cosmology simulations and data analysis developed for high performance computing machines. We discuss the challenges encountered while designing parallel and scalable software in Python that still exploits the unique interactive tools provided by the Python stack. Using the mpi4py library, nbodykit implements a fully parallel, canonical set of algorithms in the field of large-scale structure cosmology and also includes a set of distributed data containers, insulated from the algorithms themselves. We use the dask library to provide a straightforward method for users to manipulate data without worrying about the costs of parallel IO operations. We take advantage of the readability of Python as an interpreted language by implementing nbodykit in pure Python, while ensuring high performance by relying on external, compiled libraries, optimized for specific tasks. We demonstrate the ease of use and performance capabilities of nbodykit with several real-world scenarios in the field of cosmology.