Fine-tuning of financial Large Language Model and application at edge device
Juntao Zeng, Bo Chen, Yuandan Deng, Weiqin Chen, Y. Z. Mao, Jiawei Li · 2024
Large Language Model (LLM), particularly conversational models, have garnered significant attention in recent years, owing to their novel approaches to human-computer interaction and natural language processing tasks for individuals or organizations. Against the backdrop of private computing becoming a prevailing trend, the training, fine-tuning, and application of private large models have become exceptionally challenging. This study aims to delve into how fine-tuning LLM for vertical domain applications, in conjunction with edge device, facilitates innovative applications of LLMs in private and vertical domains. We fine-tuned the BaiChuan-2-7b-chat model, specifically enhancing its reasoning capabilities in the financial domain. Through comparative analysis of benchmark datasets, the fine-tuned model exhibited a 2% increase in average response accuracy across various financial domains. Furthermore, we implemented the fine-tuned model on both central computing infrastructure and edge devices, undertaking a comprehensive investigation into the model's inference efficiency and nuanced power consumption across diverse hardware platforms. Comparative analyses unveiled that the deployment of personalized Language Model Machines (LLMs) on edge devices yields superior cost-effectiveness.