Beyond Sentiment: A Multifaceted Review Scoring System for Enhanced Customer Feedback Analysis
Bhavesh Kukreja, Aritra Ghosh Dastidar, Radhika Mundra, Kartikey Singh, Javaid Nabi · 2024
Effective customer review analysis is crucial for driving product development and improving customer satisfaction. Traditional review analysis predominantly relies on sentiment scores to gauge customer feedback. However, sentiment scores alone often fail to capture the full spectrum of review significance, leading to misinterpretations and hindering informed product development decisions. For instance, a product may receive an overall positive sentiment score, yet closer examination reveals numerous complaints about specific features. This paper introduces a novel multi-faceted review scoring system (MRS) designed to address this limitation by integrating additional parameters such as product features, sub-features, review recency, review frequency, and product priority alongside a modified sentiment analysis. This modified sentiment analysis assigns a score from 0 to 1 for both positive and negative sentiment phrases within a review, allowing for a more nuanced understanding of sentiment expression. By adopting this comprehensive approach, we aim to enhance the accuracy and relevance of review prioritization. Our methodology employs large language models (LLMs) for review processing and score generation for some dimensions, while statistical methods are used for others. These scores are then combined to produce an overall review score. The proposed system, validated using an in-house review dataset, demonstrates superior performance over traditional sentiment-based methods by revealing deeper insights into customer feedback. Our system effectively prioritizes critical reviews (achieving a precision of 0.95, recall of 0.92, and F1-score of 0.93), identifies key features driving customer dissatisfaction, and aligns strongly with human judgment (correlation coefficient of 0.9). This multifaceted review scoring approach empowers businesses to make more informed decisions and prioritize critical reviews effectively.