PeQA: A Massive Persian Question-Answering and Chatbot Dataset
Fatemeh Zahra Arshia, Mohammad Ali Keyvanrad, Saeedeh Sadat Sadidpour, Sayed Mohammad Reza Mohammadi · 2022
TA question-answering (QA) system is an application able to communicate with humans using natural language processing. Modelling a dialogue between humans and machines is considered one of the most important tasks of Artificial Intelligence (AI). Creating a Chatbot with a good performance in modelling human-machine conversations is still one of the unsolved challenges in this field. Although Chatbots have many applications, in general, they should understand users' meaning through their words and provide them with relevant answers. In the past, Chatbot architectures mainly relied on rules or statistical methods. With the advent of deep learning methods, trainable neural networks soon replaced the traditional models. These sorts of deep models are highly affected by the dataset that would be fed into them, and there is no big enough one available in the Persian language! We present a huge dataset of 14 million Persian tweets from tweeter that is meticulously processed to create a rich collection of 420,000 pairs of question-answer data. We also present modelling results on Transformers, including Sensibleness and Specificity Average (SSA) and the BLEU metric. We will release our dataset, modelling code, and models publicly1.1https://github.com/mut-deep/PEQA