7. Data Storage and Distribution
Michael Fehler, P. Joseph Keliher · Society of Exploration Geophysicists eBooks · 2011
Data Volume Sizes and the Decision to Compress One of the painful lessons learned from the earlier SEG/EAGE 3D modeling work was that provision needs to be made quite early in the process for long-term storage and distribution so that the data that are generated don't become inaccessible within a few years. For example, the old SEG/EAGE 3D model shot records are sitting on D2 tapes in the basement of the Tulsa SEG headquarters. It is now cheaper for someone to regenerate those shot records from scratch on a moderate Linux cluster than it is to try to recover the data from those outmoded tapes. In line with the overall SEAM business model to put out for bid contracts for the various aspects and phases of the project, the Storage and Distribution Committee was formed to research commercial storage options, estimate budget impact, and help draft bid specifications. The first step was to assess the amount of data that would need to be stored. As the model and acquisition design were constantly in flux, this proved a challenge. At various points, proposed data acquisition schemes ranged from the low hundred terabytes to more than two petabytes. Clearly, as the volume of data would be a large component in the cost of storage and distribution, the committee assembled a list of companies that would have the capability and potential interest in storing and distributing SEAM data. These covered a range of business models: traditional E&P data storage and transcription, PC data backup and restore, and corporate data center hosting. Unsurprisingly, their business models covered a range as well, with different weights on cost of storage versus distribution.