A scale-dependent notion of effective dimension
Oksana Berezniuk, Alessio Figalli, Raffaele M. Ghigliazza, Kharen Musaelian · arXiv (Cornell University) · 2020
We introduce a notion of "effective dimension" of a statistical model based on the number of cubes of size $1/\sqrt{n}$ needed to cover the model space when endowed with the Fisher Information Matrix as metric, $n$ being the number of observations. The number of observations fixes a natural scale or resolution. The effective dimension is then measured via the spectrum of the Fisher Information Matrix regularized using this natural scale.