Diversity-Aware Graphormer: Insights from Hate Speech Detection on 4Chan
Allana Tavares Bastos, Martı́n Gómez Ravetti · 2025
Transformers have reshaped the AI landscape, powering important breakthroughs like BERT. Graphormer, a Graph Neural Network that leverages Transformer-based attention mechanisms, has found success in capturing complex relationships in structured data like molecular structures and recommendation systems. This success can be attributed to Graphormer's capabilities in leveraging graph's topological information. Building upon this foundation, we propose a novel Graphormer-based approach incorporating diversity to improve the node relationship's representation. We also propose a novel Graphormer framework designed to tackle hate speech detection on the fringe platform 4chan. To enhance the model's ability to discern harmful intent from humorous or ironic expression, we introduce sarcasm detection as an auxiliary task. We evaluate our model on a 4chan dataset, demonstrating the model's performance over baseline methods through quantitative and qualitative analysis. Our research highlights the potential of diversity-aware Graphormers for addressing challenging graph-based tasks in online content moderation and other applications.