The Application of Markov Chains to Linguistic Predictions by Utilising its Inherent Information Entropy
Pradeep Jha, Pankaj Jain, Amit Kumar, Sangeeta Soni, Yash Sharma, Pulkit Agarwal · 2025
Linguistic prediction is a process in psycholinguistics arising if information about a word or other linguistic device is triggered before that unit is eventually encountered. Information Entropy (IE) on the other hand is a concept of Information Theory that deals with the “uncertainty” inherent in the variable's possible outcomes. For our purposes, consecutive English words are considered to be uncertain variables. This project aims to predict the most probable next-word during dynamic typing by employing the Markov Chain Model to sequences of words. A transition matrix is generated using large datasets, and predictions are made accordingly. This model performs better compared to deep learning models in some instances owing to their hardwiring based on the concept of IE.