Test et estimation multiple de moyennes en grande dimension avec applications à l'apprentissage automatique
Jean-Baptiste Fermanian · theses.fr (ABES) · 2024
In this thesis, we study the influence of high dimension in testing and estimation problems. We analyze the dimension dependence of the separation rate of a closeness test and of the quadratic risk of multiple vector estimation. We complement existing results by studying these dependencies in the case of non-isotropic distributions. For such distributions, the role of dimension is played by notions of effective dimension defined from the covariance of the distributions. This framework covers infinite-dimensional data such as kernel mean embedding, a machine learning tool we will be seeking to estimate. Using this analysis, we construct methods for simultaneously estimating mean vectors of different distributions from independent samples of each. These estimators perform better theoretically and practically than the empirical mean in unfavorable situations where the (effective) dimension is large. These methods make explicit or implicit use of the relative ease of testing compared with estimation. They are based on the construction of estimators of distances and moments of covariance, for which we provide non-asymptotic concentration bounds. Particular interest is given to the study of bounded data, for which a specific analysis is required. Our methods are accompanied by a minimax analysis justifying their optimality. In a final section, we propose an interpretation of the attention mechanism used in Transformer neural networks as a multiple vector estimation problem. In a simplified framework, this mechanism shares similar ideas with our approaches, and we highlight its denoising effect in high dimension.