Evaluating Phishing Email Efficacy
Carlos Olea, Alexander P. Christensen, Lisa K. Fazio, Laurie E. Cutting, Maxwell Lieb, Jessica Phelan, Alyssa Friend Wise, Holly Tucker · 2025
Phishing remains one of the most prevalent attack vectors in the modern cybersecurity landscape.Though filters and other providerside methods are useful in blocking certain categories of phishing emails, it remains evident that the weakest link in email security is the user.With the advent of LLMs as highly accessible and easy-touse tools for text generation, the employment of these tools in the creation of phishing emails is almost certainly a current reality.In this study, we evaluate the likelihood of over 160 undergraduate students correctly labeling both human and LLM emails as phishing (also referred to as malign in this paper) or real (also referred to as benign in this paper) emails in a semi-controlled environment both with and without time pressures.We find that LLM generated emails are a possible security weakness for this user group and deserve increased attention.We also find that certain personality traits measured by the Need for Cognition (a trait described as a need to structure relevant situations in meaningful, integrated ways and a need to understand and make reasonable the experiential world [7]) scale and the Big Five personality Inventory may correlate with a higher likelihood of susceptibility to being phished depending on the email source.We present possible explanations for these findings and propose future work in training and research on this growing concern.