The sqlLoader Data-Loading Pipeline
Alexander S. Szalay, Ani Thakar, Jim Gray · Computing in Science & Engineering · 2008
Using a database management system (DBMS) is essential to ensure the data integrity and reliability of large, multidimensional data sets. However, loading multiterabyte data into a DBMS is a time-consuming and error-prone task that the authors have tried to automate by developing the sqlLoader pipeline-a distributed workflow system for data loading.