GRAINY-XAI: A Domain-wise Granular Analysis of Sarcastic Text Using Explainable-AI Techniques
International journal of intelligent engineering and systems · 2024
Automatic sarcasm detection is essential as sarcastically written text produces noisy hidden information, which is detrimental to data-dependent systems.According to the literature, research focuses on single domain datasets.However, cross-domain or mix-domain sarcasm detection is necessary to understand the limitations and strengths of algorithms and also to accelerate the development of new solutions.In addition, understanding the underlying features of the model's outcome is significant to understand the reasons behind model's behaviour.Nevertheless, providing such explanations for models output across domains is less explored area in the literature.Thus, to fill these gaps, authors propose a model named GRAINY-XAI.The proposed model works in two phases.In phase-1, ensemble learning based models are developed across-domains on baseline datasets related to sarcasm detection namely Sem-Eval-2022 and MUStARD dataset and their performance is evaluated.In phase-2, authors check the robustness of these models using Explainable AI (XAI) techniques with granular explanations having several test cases.Further, the explanations provided by post-hoc XAI techniques, Local Interpretable Model-agnostic Explanations (LIME) for local explanations and Shapley additive explanations (SHAP) for local and global explanations were tested for quality using stability, fidelity, and coverage for LIME results and additivity and consistency for SHAP results.The results obtained through cross-domain experimentation (phase-1) provide the benchmark for further analysis as such type of experimentation is carried out for the first time in the literature.In addition, model's showed improved performance of 4.88% and 8.74%, for cross-domain and mix-domain analysis.Further, models' explanations are provided based on essential features identified using LIME and SHAP.Also, it was observed that SVC shows consistently poor performance across domains, while LGBM, CatBoost, and XGB have performed better across domains compared to other classifiers.The authors' validation test confirmed that the LIME and SHAP model's performance matches with the model's outcome.