Advances in Small Language Models: A Comprehensive Survey on Efficient NLP Solutions for Resource-Constrained Environments
Anjali Kshatriya, Komal D Prajapati · 2025
Small Language Models (SLMs) signify a new paradigm in natural language processing. They provide effective alternatives to large-scale transformer models when deployed on restricted resources. Given that SLMs are a recent development, this report examines advance SLMs and recent scholarship, including types of architectures based on transformer, and examples of domain specific applications and deployments from the various domains of healthcare, education and speech processing. We provide a systematic review of seven studies conducted between 2022-2025, including methodologies which utilize knowledge distillation, parameter optimization, and hybrid attention mechanisms. We conclude here with noting significant findings for SLMs that performing similarly to larger models, and at less operational cost, with accuracy performance ranging from 85%-95% over several tasks and parameters less than 10 million. Finally, we identified significant limitations for the above studies including cross domain generalization, model bias, and real-world deployments that are scalable. We hope that this survey represents an excellent basis for researchers pursuing efficient solutions in NLP processing, and we will develop future SLM studies in support of sustainable AI ecosystems.