Dynamic Relevance-Weighting-Based Width-Adaptive Auto-Encoder
Malak Almejalli, Ouiem Bchir, Mohamed Maher Ben Ismail · Applied Sciences · 2025
This paper proposes a novel adaptive autoencoder model that autonomously determines the optimal latent width during training. Unlike traditional autoencoders with fixed architectures, the proposed method introduces a dynamic relevance weighting mechanism that assigns adaptive importance to each node in the hidden layer. This distinctive feature enables the simultaneous learning of both the model parameters and its structure. A newly formulated cost function governs this dual optimization, allowing the hidden layer to expand or contract based on the complexity of the input data. This adaptability results in a more compact and expressive latent representation, making the model particularly effective in handling diverse and complex recognition tasks. The originality of this work lies in its unsupervised, self-adjusting architecture that eliminates the need for manual design or pruning heuristics. The approach was rigorously evaluated on benchmark datasets (MNIST, CIFAR-10) and real-world datasets (Parkinson, Epilepsy), using classification accuracy and computational cost as key performance metrics. It demonstrates superior performance compared to state-of-the-art models in terms of accuracy and representational efficiency.