Towards scalable array-oriented active storage
Viet-Trung Tran, Bogdan Nicolae, Gabriel Antoniu · ACM SIGOPS Operating Systems Review · 2012
The recent explosion in data sizes manipulated by distributed scientific applications has prompted the need to develop specialized storage systems capable to deal with specific access patterns in a scalable fashion. In this context, a large class of applications focuses on parallel array processing: small parts of huge multi-dimensional arrays are concurrently accessed by a large number of clients, both for reading and writing. A specialized storage system that deals with such an access pattern faces several challenges at the level of data/metadata management. We introduce Pyramid, an active arrayoriented storage system that addresses these challenges. Experimental evaluation demonstrates substantial scalability improvements brought by Pyramid with respect to state-ofart approaches both in weak and strong scaling scenarios, with gains of 100% to 150%.