Uncertainty evaluation for machine learning
A Thompson, K Jagan, Agnideven Palanisamy Sundar, Rahul Khatry, J Donlevy, Spencer A. Thomas, Philip J. Harris · 2022
We explore the challenge of uncertainty evaluation in machine learning regression problems. We begin by outlining how regression problems typically arise in a metrology setting. We then perform a numerical investigation of three methods for performing uncertainty-aware machine learning regression, namely Gaussian Processes (GPs), Monte-Carlo Dropout and Deep Ensembles. Based upon our investigations, we identify a list of metrology requirements that methods for uncertainty evaluation in machine learning regression must satisfy, and we evaluate the three methods against these requirements. We conclude by describing how the requirements give rise naturally to several open challenges which constitute an agenda for future work.