Lessons Learned from the SDSS Catalog Archive Server
Ani Thakar · Computing in Science & Engineering · 2008
The Sloan Digital Sky Survey is one of the first very large archives in astronomy and other sciences. Experiences gained over the past decade of SDSS Catalog Archive Server development in software, hardware, and operational issues could provide useful advice for future archive projects. The Sloan Digital Sky Survey, or SDSS, is a multi-institution, internationally funded project to map about half of the northern sky in unprecedented detail with a dedicated 2.5-m telescope and special-purpose instruments. The SDSS Science Archive resulting from the survey contains the calibrated science data and is published in two formats: a raw image archive consisting of approximately 15 Tbytes of binary image files and accessible via an rsync/wget interface, and an approximately 6-Tbyte catalog archive that is loaded into a relational database management system (DBMS) and is accessible via specialized Web-based tools and SQL query interfaces.