Sequential Hypothesis Testing Based on Machine Learning
Ryan Harvey, Paolo Braca, Leonardo Maria Millefiori, Peter Willett · 2024
With the rapid proliferation of Machine-Learning (ML) and Deep Learning (DL) based decision systems, properly characterizing their often unpredictable performance is a key challenge. In this work we introduce the notion of a Sequential Data-Driven Decision Function (S-D3F), as a data-driven analogue to the Sequential Probability Ratio Test (SPRT). Key performance metrics for sequential analysis are shown suitable for use in analyzing the S-D3F’s performance both in terms of error probabilities and average stopping times. The notion of rate function from large deviations theory is extended to this S-D3F test, and it is shown that with a sequential approach the S-D3F can outperform its Fixed Sample-Size (FSS) counterpart in the D3F as the average number of samples needed to make a decision diverges.