Exploring the Suitability of the Cerebras Wafer Scale Engine for the Fast Prototyping of a Multilingual Hate Speech Detection System
Michael Hoffmann, Jophin John, Nicolay Hammer · 2024
The era of digital communication has brought about a concerning rise of online hate speech. In response, researchers have focused on developing automated systems to detect and monitor such harmful content. While much attention has been given to monolingual detection systems, recent years have seen the emergence of new approaches for multilingual hate speech detection. However, there remains a limited understanding of how the underlying computational infrastructure impacts the training and development times of such systems. This study presents an innovative experimental design aimed at investigating the relationship between accelerator infrastructure and multilingual hate speech detection. It begins by constructing a prototype system of different classification algorithms for detecting hate speech in English, German, Italian and Spanish text-based social media content. The study then evaluates the fine-tuning times of these classifiers using both conventional GPU-based accelerators and cutting-edge AI-accelerator hardware, such as the Cerebras CS-2 system. The latter claims to speed up the development and fine-tuning of large language models significantly. Furthermore, the study compares the fine-tuning times of the same classifiers on two separate AI Accelerator machines. The study shows that the Cerebras AI Accelerator quickens training times by factor 4 compared to traditional setups, with little variation across high-performance computing infrastructures. However, the technology is in its early stages, with drawbacks including substantial upstart and compilation times, a limited number of models portable to the CS-2, and a need for refinement to improve accessibility for non-experts. Nonetheless, the Cerebras CS-2 technology heralds a new era for the rapid development of effective hate speech models.