Elephas: Distributed Deep Learning with Keras & Spark
Max Pumperla, Daniel E. Cahall · The Journal of Open Source Software · 2022
Elephas is an extension of Keras, which allows you to run distributed deep learning models at scale with Apache Spark.It was built to allow researchers and developers to distribute their deep learning experiments as easily as possible on a Spark computer cluster.With elephas, researchers can currently run data-parallel training of deep learning models with distribution modes as suggested in (Dean et al., 2012), (Recht et al., 2011) and (Noel & Osindero, 2014).Additionally, elephas supports distributed training of ensemble models.Until version 2.1.,elephas also supported distributed hyper-parameter optimization of Keras models.Elephas keeps the simplicity and high usability of Keras, thereby allowing for fast prototyping of distributed models.When ready, researchers can then scale out their experiments on massive data sets.Elephas comes with full API documentation and examples to get you started.Initiated in late 2015, elephas has been actively maintained since then and has reached maturity for distributed deep learning on Spark.