Efficient Estimation of Generative Models using Tukey Depth
Minh-Quan Vo, Thu Nguyen, Michael Alexander Riegler, Hugo L. Hammer · Research Square · 2023
Abstract Generative models have recently received a lot of attention. However, a challenge with such models is that it usually is not possible to compute the likelihood function, making parameter estimation or training of the models challenging. The most commonly used alternative strategy is called likelihood-free estimation, based on finding values of the model parameters such that a set of selected statistics have similar values in the dataset and in samples generated from the model. However, a challenge is how to select statistics that are efficient in estimating unknown parameters. The most commonly used statistics are the mean vector, variances, and correlations between variables, but may be less relevant in estimating the unknown parameters. We suggest utilizing Tukey depth contours (TDCs) as statistics in likelihood-free estimation. TDCs are highly flexible and can capture almost any property of multivariate data, in addition, they seem not have been explored for likelihood-free estimation. We demonstrate that TDC statistics are able to estimate the unknown parameters more efficiently than mean, variance, and correlation in likelihood-free estimation. We further apply the TDC statistics to estimate the properties of requests to a computer system, demonstrating its real-life applicability. The suggested method is able to efficiently find the unknown parameters of the request distribution, and quantify the estimation uncertainty.