BEEXPERT: Voice-based Chatbot with Retrieval Augmented Generation for Personal High School Mathematics Tutor
Angeline Mary Marchella, Naufal Hardiansyah, Nathaniel Candra, Rhio Sutoyo, Ubaidah Ubaidah, Dennis Alexander, Andry Chowanda · 2025
Artificial Intelligence (AI) chatbots are increasingly used as personal tutors, especially in teaching high school mathematics. However, existing tools often lack conversational depth, neglect critical thinking, and risk providing false information. To address these issues, this paper introduces BEEXPERT, a voice-based chatbot designed to assist high school students in learning trigonometry. Powered by a Retrieval-Augmented Generation (RAG) system, it retrieves accurate, contextually relevant information from an OpenWebMath dataset of 515 trigonometry documents, supporting interactions in English and Indonesian. While the RAG system achieved substantial correctness (72.98%), it struggled with faithfulness (31.91%) and relevance (47.87%) due to hallucinations and context fragmentation. Feedback from preliminary testing and the Godspeed Questionnaire Series (GQS) highlights positive user reception, with 88% for likeability, 80% for perceived intelligence, and 75% for anthropomorphism. Expert validation also confirmed alignment with the school curriculum. Suggestions for improvement, such as refined prompts and enhanced features, have been partially implemented, with future updates planned to optimize the system further. In essence, BEEXPERT demonstrates significant potential as a personalized learning tool while highlighting areas for improvement to ensure a more robust and effective educational experience.