Acquisition of Highly Independent Latent Space for Feature-based Data Distribution Control Using DenseNet
Yuki Ikezumi, Masataka Seo · 2023
The Transformer architecture has been applied i n many natural language processing tasks such as BERT(Bidirectional Encoder Representations from Transformers) and GP T(Generative Pretrained Transformer). In image recognition, Vision Transformer has garnered much attention for outperforming convolution-based convolution neural network structures . In the attention mechanism, focus is placed on the important information in the input data and more weight is assigned to such information to improve the output accuracy. In this research, we first acquire various latent variables using DenseNet. Multiple encoders are then used in an encoder-decoder structure to obtain a non-redundant latent space using the attention mechanism. Furthermore, multi-objective learning is employed in multiple stages to achieve intuitive feature map acquisition based on the data characteristics.