Fine-Tuning Bert On The Atis Dataset: Data Enhancement To Improve Intent Classification Accuracy
Lina Sun, Konstantin A. Aksyonov · 2024
Intent recognition is a pivotal task within the domain of Natural Language Understanding (NLU), specifically within the context of natural language classification. Accurate intent recognition is crucial for systems to more effectively comprehend user needs and intentions, thereby offering more precise and effective services and solutions. The dataset utilized in this project is the "ATIS Airline Travel Information System". It comprises questions sent by users through an airline service platform, which encapsulate the user's intent along with corresponding text labels for each question. The original document of this dataset contains 5,078 texts and is annotated with 11 types of labels. Upon the evaluation of the data, this project employed data augmentation techniques to fine-tune the BERT-BASE model for the construction of an intent recognition classification model specific to the aviation industry. Subsequently, the performance of the model in intent recognition was assessed using a test dataset. Additionally, a comparative analysis was conducted to benchmark the model's performance against similar research projects.The performance was subsequently compared with that of similar research projects. The project also included the plotting of a confusion matrix to analyze the model's accuracy in predicting intentions for specific intent labels. Additionally, directions for model improvement were identified. The classification model developed using BERT in this project achieved an intent recognition accuracy of 98.2%.