Enhanced Voice Phishing Detection Using an LLM-Based Framework for Data Augmentation and Classification
H. PARK, Jiwon Lee, Sanghyun Han, Hae-Won Byun · IEEE Access · 2025
Existing voice phishing detection models based on call transcripts suffer from limited generalizability due to insufficient scenario diversity and the lack of ambiguous samples in data. To address these challenges, we propose an integrated framework in which large language models (LLMs) are used both to generate realistic call transcripts based on actual fraud cases and to build an expert-guided phishing detection model. Based on case reports from the Financial Supervisory Service (FSS) and transcripts from the KorCCVi dataset, we generate phishing call transcripts that capture previously underrepresented fraud tactics. Additionally, we generate non-phishing call transcripts that retain phishing-like linguistic patterns by removing or attenuating core fraudulent cues, enabling training on ambiguous cases. The generated data are quantitatively evaluated based on linguistic naturalness, scenario diversity, and detection difficulty. To further assess the complexity of sample-level detection in semantic space, we introduce a metric called the Class Centroid Distance Variability (CCDV). Furthermore, we propose the Domain Expert LLM, a prompt-engineered detection model that incorporates six analytical criteria validated by a domain expert in phishing detection. The model not only improves the detection performance but also generates structured analytical reports to enhance the interpretability of its outputs. Experimental results show that the Domain Expert LLM achieves an F1 score of 0.9686 on previously unseen and ambiguous transcripts, significantly outperforming conventional models such as RandomForest and KoBERT, which yield an average F1 score of approximately 0.70.