DeepDup: Duplicate Question Detection in Community Question Answering
Mohomed Shazan Mohomed Jabbar, Luke Kumar, Hamman Samuel, Mi-Young Kim, Sankalp Prabharkar, Randy G. Goebel, Osmar R. Zai͏̈ane · 2021
Duplicate question detection is an ongoing challenge in community question answering because semantically equivalent questions can have significantly different words and structures. The identification of duplicate questions can reduce the resources required for retrieval and increase findability of the associated community forums. This ongoing study presents DeepDup, a deep learning model for duplicate question detection. Our research also explores the possibility of domain adaptation with transfer learning to improve the under-performing target domains for the text-pair duplicates classification task, using heterogeneous datasets from the Stack Exchange sub-communities for Ubuntu and English. Ultimately, our study investigates the null hypothesis that there is no significant difference between a base model and a transfer-learned model.