Estimation of intrinsic dimensionality of samples from noisy low-dimensional manifolds in high dimensions with multiscale SVD
Anna V. Little, Jason D. Lee, Yoon-Mo Jung, Mauro Maggioni · 2009 IEEE/SP 15th Workshop on Statistical Signal Processing · 2009
The problem of estimating the intrinsic dimensionality of certain point clouds is of interest in many applications in statistics and analysis of high-dimensional data sets. Our setting is the following: the points are sampled from a manifold M of dimension k, embedded in RopfD, with k Lt D, and corrupted by D-dimensional noise. When M is a linear manifold (hyperplane), one may analyse this situation by SVD, hoping the noise would perturb the rank k covariance matrix. When M is a nonlinear manifold, SVD performed globally may dramatically overestimate the intrinsic dimensionality. We discuss a multiscale version SVD that is useful in estimating the intrinsic dimensionality of nonlinear manifolds.