InterpreTutor: Using Large Language Models for Interpreter Assessment
Cihan Ünlü · 2023
The recent development of large language models (LLMs) has shown remarkable natural language processing capabilities and created new possibilities for applications in various fields.With their advanced NLP features, such as text comparison, semantic analysis, text summarization, text classification, and text completion, this study aims to investigate whether these models have the potential to be used as tools for evaluating translation/interpretation output in a textual representation.In this paper, we question the capabilities of an LLM and propose InterpreTutor 1 , an LLM-powered application with simple UI that utilizes Generative Pre-trained Transformer (GPT-3.5 turbo and GPT-4 models) and speech recognition (OpenAI Whisper) to offer detailed feedback on interpreters' performances based on automatic analysis of the transcriptions of their interpreting practice.The tool's primary focus is to act as an easy-to-use selftutoring tool, offering feedback based on four human evaluation criteria.While InterpreTutor may not cover all aspects of interpreting performance, such as prosodic features of the delivery, it still provides insights into aspects that can be assessed through textual representation.In this paper, we will discuss the potential of LLMs, the development of InterpreTutor, its underlying methodology, and provide examples of its application with a small-scale experiment.