Sparse Generative Embeddings of Handwritten Digits
Sergiy O. Gnatyuk, Pylyp Prystavka, Serge Dolgikh, Vasyl Kondratiuk, Oleksandr Kutsenko · 2023
The process of unsupervised generative learning with visual data and the structure of informative low-dimensional sparse generative representations of images of handwritten digits were investigated. Learning models with the architecture of a sparse convolutional autoencoder with constraints to produce low-dimensional representations achieved successful learning demonstrated by training metrics and high accuracy of generation of images of digits. A well-defined, continuous and connected “stacked” structure of low-dimensional slices in the sparse latent space produced by activations of participating latent neurons was observed and described in detail. The conclusion is that structured informative representations obtained with unsupervised generative models can be an effective platform for investigation of the emergence of common types or “concepts” in sensory inputs in artificial and biological learning systems.