Efficacy of Deep Neural Embeddings based Semantic Similarity in Automatic Essay Evaluation
Manik Hendre, Prasenjit K. Mukherjee, Raman Preet, Manish Godse · International Journal of Computing and Digital Systems · 2021
Deep neural embeddings are widely used in natural language processing (NLP) applications like question answering, prediction of next word or sentence, translation of language, word sense disambiguation, and many such applications.Recent methods like Google Sentence Encoder (GSE), Embeddings for Language Models (ELMo), and Global Vectors (GloVe) are also engaged in NLP.Traditional methods such as TF-IDF and Jaccard index are also beneficial in NLP.One of the primary steps performed by these methods is to determine semantic similarity, which is at the core of automatic essay evaluation.In this paper, we have proposed to use semantic similarity for an automatic essay evaluation.We have utilized all these text embedding methods to compute semantic similarity on the dataset of essays provided by the Center for Indian Language Technology (CFILT), IIT Bombay.Our experimental analysis of semantic similarity score distributions shows that the GSE outperforms other methods by accurately distinguishing essays from the same or different sets.Semantic similarity calculated using the GSE method is further used for finding the correlation with human-rated essay scores.Correlation of semantic similarity scores with different essay-specific traits given in the ASAP++ dataset is also performed.