Comparative Study of Seq2Seq and Transformer Model for Chat Bot
Nipun Bansal, Abhishek Kumar Singh, Abhishek Kumar, Abhishek Tyagi · 2023
Today, every company relies on technology for the effective delivery of services and the optimal use of technical resources. With the increasing adoption of AI based technologies in company operations worldwide, including India, the global Chatbot market will grow rapidly in the coming years. Chatbot technology has gained widespread acceptance in recent years, especially in the food delivery, finance, and e-commerce industries. Chatbots are computer programs that can answer questions posed by humans, and they are accessible anytime, providing quick and efficient communication with thousands of people simultaneously. Chatbots are becoming increasingly popular in human-machine interactions due to their ability to provide information without requiring time-consuming searches. In this paper, we will compare the methodologies, underlying algorithms, accuracy, and constraints of the various models for chat bots. Models compared in this study are Seq2seq Model with Attention Mechanism and Transformer. The result shows that the transformer model requires less number of iterations (epochs) to train than seq2seq model. Compared to LSTM networks, training transformer networks is easier due to the smaller number of parameters involved and Transformer network is faster than RNN based models.