Question-Question Similarity in Online Forums
Yllias Chali, Rafat Bin Islam · 2018
In this paper, we applied deep learning framework to tackle the tasks of finding duplicate questions. We implemented some models following the siamese architecture using the popular recurrent network such as Long-Short term memory (LSTM), Bi-direction Long-Short term memory (biLSTM) to find the semantic similarity between questions. We started with a basic model and further extended the basic model into three different models. Our models provide a refined, composite representation of the questions. The addition of Convolutional Neural Network (CNN) with the recurrent networks is a new approach for the sentence representation. We also applied attention mechanism for getting better contextual meaning of the questions. We generated a representation of a question according to the context of another question for solving the task. As neural models are data driven, we trained our models extensively by making pairs, such as question-question over a large-scale real-life dataset. We used a datset consisting of 400K labeled question pairs which are published by a well known question-answer forum Quora. We evaluate our models based on metrics like accuracy, precision, recall, F1 scores. Our methods and experiments demonstrate some significant improvements over the baseline systems and the state-of-the-art systems.