Efficient Bulk Data Replication for the Earth System Grid
Alex Sim, Dan Gunter, Vijaya Natarajan, Arie Shoshani, Dean Williams, Jeff Long, Jason Hick, Jason Lee, Eli Dart · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · 2010
The Earth System Grid (ESG) community faces the difficult challenge of managing the distribution of massive data sets to thousands of scientists around the world. To move data replicas efficiently, the ESG has developed a data transfer management tool called the Bulk Data Mover (BDM). We describe the performance results of the current system and plans towards extending the techniques developed so far for the upcoming project, in which the ESG will employ advanced networks to move multi-TB datasets with the ultimate goal of helping researchers understand climate change and its potential impacts on world ecology and society.