Evaluating Artificial Intelligence Bias In Answering Religious Questions
Ghadeer Al-Badani, Akram Alsubari · 2024
Question Answering (QA) is a specialized field in the field of NLP and most studies in this field focus on the English language, while other languages, such as Arabic, are still in their early stages. Recently, research has turned to developing answering systems for Questions for Arabic-Islamic texts, which may impose challenges due to the nature of the Arabic language and due to the lack and scarcity of reference data sets. Research has also tended to develop systems to answer open-ended questions that aim to extract the answer to the user’s question from a specific text or from a specific context in this study. We evaluated artificial intelligence to answer religious questions to facilitate access to religious information. We created a fatwa dataset consisting of questions and answers from approved websites. We used the transformer-based Arabic language generation model AraGPT2, and we reconfigured the answer generation task, as our approach provides answers to questions by generating... The answer is among the answers on which the model was trained, and there is no context available to extract the answer from. This is what is known as the system for answering closed-field questions. Then we evaluated the resulting model using the evaluation methods, which are BERTScore, Levenshtein distance, and BLEU. The proposed model achieved the best evaluation score of 0.73 on the BERTScore scale, which indicates Good performance. In addition, the performance results were analyzed to determine the strengths and weaknesses of the model. The researchers suggest improving the model's performance by expanding the data set, modifying the model structure, and applying human evaluation of the model.