SimiHawk at SemEval-2016 Task 1: A Deep Ensemble System for Semantic Textual Similarity
Peter Potash, William Boag, Alexey Romanov, Vasili Ramanishka, Anna Rumshisky · 2016
This paper describes the SimiHawk system submission from UMass Lowell for the core Semantic Textual Similarity task at SemEval-2016.We built four systems: a small featurebased system that leverages word alignment and machine translation quality evaluation metrics, two end-to-end LSTM-based systems, and an ensemble system.The LSTMbased systems used either a simple LSTM architecture or a Tree-LSTM structure.We found that of the three base systems, the feature-based model obtained the best results, outperforming each LSTM-based model's correlation by .06.Ultimately, the ensemble system was able to outperform the base systems substantially, obtaining a weighted Pearson correlation of 0.738, and placing 7th out of 115 participating systems.We find that the ensemble system's success comes largely from its ability to form a consensus and eliminate complementary noise from its base systems' predictions.