Exploring the Challenges of Serverless Computing in Training Large Language Models
Kushal Walia · International Journal of Computer Trends and Technology · 2024
This paper delves into the exploration of utilizing serverless computing frameworks for the training of Large Language Models (LLMs), a cornerstone of modern artificial intelligence and machine learning advancements. While serverless computing offers significant benefits, including reduced infrastructure costs and enhanced scalability, its application in the context of LLM training introduces a unique set of challenges and limitations. Through an in-depth analysis, this study identifies key obstacles such as statelessness, execution time limits, cold start latency, resource constraints, data management complexities, dependency management, and cost predictability issues that inherently complicate the deployment of LLM training pipelines in a serverless environment. Despite these hurdles, the potential of serverless computing to revolutionize the scalability and cost-efficiency of LLM training remains undeniable. By presenting a balanced view on the feasibility, challenges, and prospective solutions, this paper aims to provide insights into the current state and future possibilities of serverless computing in the realm of large language model training, marking a critical step towards optimizing computational resources in the advancement of AI technologies.