A method for measuring sentence similarity and its application to conversational agents
Yuhua Li, Zuhair A. Bandar, David McLean, James D. O’Shea · University of Salford Institutional Repository (University of Salford) · 2004
This paper presents a novel algorithm for computing similarity between very short texts of sentence length. It will introduce a method that takes account of not only semantic information but also word order information implied in the sentences. Firstly, semantic similarity between two sentences is derived from information from a structured lexical database and from corpus statistics. Secondly, word order similarity is computed from the position of word appearance in the sentence. Finally, sentence similarity is computed as a combination of semantic similarity and word order similarity. The proposed algorithm is applied to a real world domain of conversational agents. Experimental results demonstrated that the proposed algorithm reduces the scripter's effort to devise rule base for conversational agent.