K-Deep Simplex: Manifold Learning via Local Dictionaries
Abiy Tasissa, Pranay Tankala, James Michael Murphy, Demba Ba · IEEE Transactions on Signal Processing · 2023
We propose K-Deep Simplex (KDS)which, given a set of data points, learns a dictionary comprising synthetic landmarks, along with representation coefficients supported on a simplex. KDS employs a local weightedℓ1penalty that encourages each data point to represent itself as a convex combination of nearby landmarks. We solve the proposed optimization program using alternating minimization and design an efficient, interpretable autoencoder using algorithm unrolling. We theoretically analyse the proposed program by relating the weightedℓ1penalty in KDS to a weightedℓ0program. Assuming that the data are generated from a Delaunay triangulation, we prove the equivalence of the weightedℓ1and weightedℓ0programs. We further show the stability of the representation coefficients under mild geometrical assumptions. If the representation coefficients are fixed, we prove that the sub-problem of minimizing over the dictionary yields a unique solution. Further, we show that low-dimensional representations can be efficiently obtained from the covariance of the coefficient matrix. Experiments show that the algorithm is highly efficient and performs competitively on synthetic and real data sets.