Sociotechnical Perspectives on Balancing Human Judgment and AI in Content Moderation
Budi Hartono, Iman Saufik Suasana, Moh Muthohir, Ramsundar Suresh, Setiyo Prihatmoko · 2025
The rapid advancement of Artificial Intelligence (AI) has revolutionized content moderation across digital platforms, but it has also introduced complex sociotechnical challenges. This study aims to explore the interaction between human moderators and AI systems in social media content moderation, with a focus on understanding the balance between automation and human judgment. A sociotechnical case study approach is employed, incorporating semi structured interviews with content moderators and AI engineers from major social media platforms. Thematic analysis identifies five key challenges is emotional distress among moderators, unclear role distribution between humans and AI, overreliance on AI decisions, algorithmic bias, and a lack of transparency. The findings highlight that while AI enhances efficiency, it struggles with contextual and cultural nuances, leading to misclassification. Additionally, human moderators face psychological strain due to exposure to disturbing content. The study concludes that hybrid moderation systems, while promising, remain imbalanced without clear ethical guidelines, psychosocial support, and robust feedback loops. Recommendations are provided for more defined human AI role definitions, the design of inclusive AI systems, and the implementation of organizational policies that support moderator well being and promote digital justice. This research contributes to advancing the ethical deployment of AI in content moderation, addressing both technological and human factors for more effective, fair, and sustainable moderation practices.