Deep Multimodal Classification of Musical Genres
Bassem Jandoubi, Moulay A. Akhloufi · 2025
Music genre classification is a challenging problem due to the subjective and complex nature of music perception. This project explores three complementary approaches to genre classification: audio-based classification using Mel-spectrograms and Convolutional Neural Networks (CNN), lyrics-based classification using RoBERTa for natural language processing, and a multimodal classification approach that integrates both modalities. The audio-based approach achieved remarkable performance improvements through augmentation techniques, while the lyrics-based model demonstrated effective text comprehension using pre-trained transformers. Finally, a multimodal approach that combines Mel-spectrograms and lyrics through concatenation achieved the highest accuracy, underscoring the advantages of integrating diverse data modalities. These results demonstrate the potential of multimodal deep learning for complex classification tasks, opening the way to improved music retrieval systems.