OPUS-97: A Generalized Operational Pipeline System
R. Albrecht, R. N. Hook, H. Bushouse, James F. Rose · ASPC · 1998
OPUS is the platform on which the telemetry pipeline at the Hubble Space Telescope Science Institute is running currently. OPUS was developed both to repair the mistakes of the past, and to build a system which could meet the challenges of the future. The production pipeline inherited at the Space Telescope Science Institute was designed a decade earlier, and made assumptions about the environment which were unsustainable. While OPUS was developed in an environment that required a great deal of attention to throughput, speed, eciency, flexibility, robustness and extensibility, it is not just a \big science machine. The OPUS platform, our baseline product, is a small compact system designed to solve a specic problem in a robust way. The OPUS platform handles communication with the OPUS black- board; individual processes within this pipeline need have no knowledge of OPUS, of the blackboard, or of the pipeline itself. The OPUS API is an intermediate pipeline product. In addition to the pipeline platform and the GUI managers, the OPUS object libraries can give your mission ner control over pipeline processing. The OPUS platform, including a sample pipeline, is now available on CD-ROM. That package, designed to be run on the Solaris operating system, can help you decide whether OPUS can be used for your own mission. OPUS was developed in an environment which demanded attention to pro- ductivity. A ten-fold increase in the volume of data was anticipated with the new instrumentation to be flown on the Hubble, and the ground system required a pipeline which could accommodate that load. OPUS has met these challenges. A distributed pipeline system which allows multiple instances of multiple processes to run on multiple nodes over multiple paths may seem like an opera- tional nightmare. To resolve this, OPUS includes two pipeline managers: Motif GUI applications which assist operations sta in monitoring the system. The Process Manager not only assists with the task of conguring the system, but monitors what processes are running on which nodes, and what they are doing currently. The Observation Manager provides a dierent view of the pipeline ac- tivities, monitoring which datasets are in which step in the pipeline and alerting the operator when observations are unable to complete the pipeline.