Optimizer Comparison with Dropout for Neural Sequence Labeling in Myanmar Stemmer
Yadanar Oo, Khin Mar Soe · 2019
In Myanmar language, texts typically contain many different forms of a basic word. Morphological variants are generally the most common problem in mis-spellings, wrong translation and irrelevant retrieval query. The effectiveness of searching is obviously related to the stemming process. Moreover, there is no space separation in Myanmar language. Therefore, the tasks of segmenting the initial texts to words sequence is fully related to the stemming process. In present-day, deep learning approaches have become very good performance in variety of tasks, such as natural language processing, speech recognition, image recognizing. Among different types of neural networks, CNN networks have been most extensively used in text processing to extract morphological information (prefix and suffix of a word). This paper proposes the optimization process in Neural Architecture, how loss functions fit into the equation and finding the best optimizer. This paper also classified the efficiency of dropout under each optimizer to improve CNN-based model which jointly learns stemming and segmentation boundaries in parallel. It has obtained significant improvements on model performance after using dropout and the highest F-score is dropout probability 0.2. According to the experimental results, the SGD and Adam optimizer have a vast effect on the performance. And then, RMSProp optimizer performs better than other optimizers even though there is less dropout nodes.