IT Service Desk Ticket Classification via Large Language Models
Ezgi Paket, Göksu Şenerkek, Fatma Betül Akyol, Furkan Salman · 2024
Service desk systems are utilized in companies to enable employees to forward their issues or requests to IT support operators. These tickets are assigned by support operators to the appropriate categories based on their personal experiences and the manually prepared keyword-matching catalogs. The dependency on individual experiences and the difficulty of maintaining and updating manually prepared catalogs increase the misclassification rate. This study aims to address a challenging multi-class classification problem in the Turkish language, which includes 172 classes, by utilizing large language models (LLMs). For this purpose, next-generation LLMs such as Titan, Llama, and Mistral are compared with BERT-based models using zero-shot, fine-tuning, and RAG methods within this st udy. This research shows that BERT-based classification models outperform the LLMs, which are not specifically trained for classification tasks with several classes.