Intrinsic Dimensionality Estimation of High-Dimension, Low Sample Size Data withD-Asymptotics
Kazuyoshi Yata, Makoto Aoshima · Communication in Statistics- Theory and Methods · 2010
High-dimension, low sample size (HDLSS) data are becoming common in various fields such as genetic microarrays, medical imaging, text recognition, finance, chemometrics, and so on. Such data have surprising and often counterintuitive geometric structures because of the high-dimensional noise that dominates and corrupts the local neighborhoods. In this article, we estimate the intrinsic dimension (ID) that allows one to distinguish between deterministic chaos and random noise of HDLSS data. A new ID estimating methodology is given and its properties are studied by using a d-asymptotic approach.