Scalable Analysis of Distributed Workflow Traces
Dan Gunter, Brian L. Tierney, S. Bailey · eScholarship (California Digital Library) · 2005
Bacterial response to nitric oxide (NO) is of major importance since NO is an obligatory intermediate of the nitrogen cycle. Transcriptional regulation of the dissimilatory nitric oxides metabolism in bacteria is Large-scale workflows are becoming increasingly important in both the scientific research and business domains. Science and commerce have both experienced an explosion in the sheer amount of data that must be analyzed. An important tool for analyzing these huge datasets is a compute cluster of hundreds or thousands of machines. However, debugging and tuning clusters requires specialized tools. Current cluster performance tools are more oriented towards tightly coupled parallel applications. We describe how the NetLogger Toolkit methodology is more appropriate for this class of cluster computing, and describe our new automatic work flow anomaly detection component. We also describe how this methodology is being used in the Nearby Supernova Factory (SN factory) project at Lawrence Berkeley National Laboratory.