FLUENT: Factoid Retrieval Based Chatbot Using LSTM
Fitra Abdurrachman Bachtiar, Andi Alifsyah Dysham, Alfirsa Damasyifa Fauzulhaq, Ja'far Shidqul Azzam, Himawat Aryadita · 2023
Advancement in technology especially in the Industrial Revolution 4.0 drive various automation in various field including the education field. Automation in the education filed may improve or support the daily business process. However, not all automation is implemented to support the student services. For example, a staff still needs to reply student questions repetitively and try of respond every inquiry shortly. These activities is hardly done by the staff since the number of staff is limited and could only serve few student at a time. One of the example to overcome this problem is the usage of chatbot. This study presents a development of a model that can act as an agent to answer students' questions about the faculty's general services and academic information. This study compares two types of models: LSTM and DNN which is built based on the data from a knowledge base collected in this study. The LSTM model consist of 7 million parameters with the Seq2seq architecture which has an encoder and a decoder. Meanwhile, the DNN model has four layers with 27 million parameters. The evaluation criteria used in this study is the loss metrics during training, the BLEU score and the human evaluation results from three respondents. The results show that LSTM outperforms DNN in all aspects. The LSTM model achieved 0.0024 for loss, much lower than the DNN achieved 1.77. The mean of the BLEU Score of LSTM is also 70% higher than the DNN. The results from the human evaluation shows that 53.33% of responses generated by LSTM are considered Good. In contrast, only 13.33% responses generated by DNN are considered good response. The better performance of LSTM over DNN is due to its ability to capture the question-answer pair like of the data used, which is more suitable than the feedforward structure of DNN, where more appropriate for the data used in class.