Twitter Sentiment Analysis using Novelty Detection
Mahalakshmi Shanmugam, Aayushi Agawane, Anchal Tiwari, Rugved Vivek Deolekar · 2020 Third International Conference on Smart Systems and Inventive Technology (ICSSIT) · 2020
This paper focuses on devising an approach, where a huge amount of data is collected from social media websites such as Twitter and further they are categorized based on their sentiment as positive, negative and neutral. This paper focuses on creating a model that would only consider tweets relevant to the topic in consideration by eliminating unwanted and irrelevant tweets from the dataset and then performing sentiment analysis on them for predicting the overall sentiment of the topic. The concept of novelty detection is used for the same. Novelty detection helps in removing the irrelevant tweets and considers only relevant tweets from a huge dataset and sentiment analysis is performed on these tweets. Novelty detection detects all the outliers present in the dataset using the k-means clustering algorithm. After detecting all the outliers, the model will eliminate those outliers and perform sentiment analysis only on novel data. Naive Bayes Algorithm is used for the classification of the novel data. The project aims to compare the accuracy of the process of sentiment analysis on Twitter data of correctly classifying the data as positive, neutral and negative before and after applying Novelty Detection on it thereby analyzing whether considering only novel data from the huge dataset will have a positive effect on the precision and accuracy of the model. Accuracy, Recall, F-score measures are used to calculate the performance of this analysis.