Topic detection on dark web
Joelle Ningxi Chew · DR-NTU (Nanyang Technological University) · 2026
The dark web serves as a hidden and dangerous hub for threat actors to coordinate illegal activities, contributing to a global surge in cybercrime. The rise in cyber attacks emphasises the need to monitor these platforms closely and gain threat intelligence. This project proposes a semantically-aware, hybrid machine learning pipeline designed to extract, classify, and cluster noisy dark web forum data into threat intelligence. K-Nearest Neighbours (KNN), a supervised machine learning algorithm, is used to categorise data into predefined threat categories. Subsequently, an unsupervised machine learning algorithm, Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN), is applied to cluster the data into density-based groups. Thereafter, Keyword BERT (KeyBERT) and a Large Language Model (LLM) are utilised to securely extract semantic keywords and generate topic labels. Ultimately, this pipeline transforms noisy dark web data into structured threat intelligence, empowering cybersecurity professionals to efficiently detect, investigate, and mitigate emerging cyber threats.