RT-ROC: a near-real-time performance evaluation tool
James M. Baumann, James L. Jackson, Gregory D. Sterling, Erik Blasch · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 2005
While modern sensors allow vast amounts of data to be collected in seconds, it can take weeks or months to analyze the data and determine sensor performance. Because of this lag between data collection and usable results, issues such as sensor calibration and algorithm biases are not detected until well after an experiment or test, when it is too late to correct them. Recognizing that a rapid performance snapshot would be extremely valuable in many situations, the AFRL COMPASE Center developed tools to produce receiver operating characteristic (ROC) curves in near real-time for an advanced technology demonstration (ATD) program. These tools, called real-time (RT-ROC) and identification (ID-ROC), gave the evaluation team timely insight into overall system performance, and when substandard results were obtained, diagnostic tests were initiated to determine the underlying causes. These tools have been experiment demonstrated, allowing the COMPASE team to find and fix a sensor error in a matter of hours. This paper will concentrate on RT-ROC and will address analysis tool requirements, operation of the tool during experiments, a walkthrough of the tool using simulated data, and future uses for this application.