A Study on Paraphrase Corpus Detection Using Various ML Models
Vidhya Shree H, Jayita Saha · 2023
Paraphrase detection, a crucial task in natural language processing, involves determining if two given sentences convey the same meaning. This research paper explores the advancements made by Microsoft Research in paraphrase detection using a range of machine learning models, ensemble techniques, and fine-tuned hyper parameters. The paper then presents the significant contributions of Microsoft Research, which involve employing a diverse set of machine learning models and leveraging ensemble techniques to enhance the accuracy and robustness of paraphrase detection. Additionally, the use of grid search cross-validation for hyper parameter optimization is explored to fine-tune the models' performance. The BERT (Bidirectional Encoder Representations from Transformers) model was also trained for paraphrase detection using the ktrain library. Experimental setups, including datasets, evaluation metrics, and preprocessing steps, are described in detail. Results and analysis showcase the effectiveness of the comparative analysis, demonstrating significant improvements over existing methods. Overall, this research paper provides valuable insights into the advancements achieved by Microsoft Research in paraphrase detection through the utilization of diverse machine learning models, ensemble techniques, and fine-tuned hyper parameters, thus driving the progress in natural language processing tasks.