PetaCache: A memory-Based Data-Server System
C. Boeheim, Stephen J Gowdy, Andrew Bohdan Hanushevsky, D. Leith, R.E. Melen, R. D. MOUNT, T. Pulliam, Bill Weeks · 2006
Scientific advances depend increasingly on agility in the analysis of data, along with access to massive computation. The PetaCache project addresses the data-access issue by recognizing that the future for intense, non-sequential, data access must be based on low latency solid-state storage. The PetaCache architecture aims at a minimum unit cost, highly scalable hardware and software approach that can take advantage of existing and emerging solid-state storage technologies providing data-access latencies in the range 10-100 microseconds. A prototype system has been constructed as a cluster of 64 nodes hosting a total of one terabyte of memory. Client processors retrieve data from the data-server nodes over a switched Ethernet infrastructure using SLAC's xrootd data-server software. The system is in use for performance testing, optimization and trial deployments for scientific data analysis. It also provides an excellent platform for testing new data access paradigms