Speech LLMs in Low-Resource Scenarios: Data Volume Requirements and the Impact of Pretraining on High-Resource Languages
Seraphina Fong, Marco Matassoni, Alessio Brutti · 2025
Large language models (LLMs) have demonstrated potential in handling spoken inputs for high-resource languages, reaching state-of-the-art performance in various tasks.However, their applicability is still less explored in low-resource settings.This work investigates the use of Speech LLMs for lowresource Automatic Speech Recognition using the SLAM-ASR framework, where a trainable lightweight projector connects a speech encoder and a LLM.Firstly, we assess training data volume requirements to match Whisper-only performance, reemphasizing the challenges of limited data.Secondly, we show that leveraging mono-or multilingual projectors pretrained on high-resource languages reduces the impact of data scarcity, especially with small training sets.Using multilingual LLMs (EuroLLM, Salamandra) with whisper-large-v3-turbo, we evaluate performance on several public benchmarks, providing insights for future research on optimizing Speech LLMs for lowresource languages and multilinguality.