Fake News Detection for COVID-19 Using PySpark Framework MLlib
Lalith Abhiram Dasari, Jetty Sowmith, Naveen Krishna Mamidi, Manju Venugopalan · 2024
Besides causing a very severe health crisis for the whole world, the COVID-19 epidemic has speeded the spread of fake news and false information. Solving false news problems is important in these time-sensitive conditions to provide correct information and guarantee people's safety. This work aims to use PySpark, a distributed data processing platform, to create a prediction model for identifying COVID-19 fake news. The experiments are conducted on a dataset sourced from Kaggle and a prediction model is created using PySpark's machine learning libraries, such as MLlib and SparkML, to classify news articles as real or false. Different classification methods, such as logistic regression, decision trees, and random forests, are researched and improved for the best performance. The accuracy, precision, recall, and F1 score of the model are assessed using relevant measures, confirming its efficacy in detecting bogus news. The best results were achieved by the logistic regression classifier which achieved accuracy and F-measure of 0.9.