Simulation of prompt injection attacks on generative pre-trained transformers models
Aditya Fahrizal Kurniawan, Michael Bryan Chandra · Procedia Computer Science · 2025
Large language models (LLMs), such as GPT-2, GPT-3.5, and GPT-4o Mini, are widely used in various applications, yet their growing adoption has also revealed critical security vulnerabilities. Among the most pressing threats are prompt injection attacks, sensitive information disclosure, and unbounded consumption, which can lead to model manipulation, data leakage, and denial of service (DoS) conditions. Although prior research has examined these risks individually, this study uniquely conducts a com- prehensive side-by-side evaluation of all three adversarial vectors in multiple LLM architectures. Through an automated attack simulation framework, we systematically assess the weaknesses of different models, uncovering notable security gaps. Our results show that newer, more advanced models, such as GPT-4o Mini, are paradoxically more susceptible to adversarial manipulation, achieving a 100% success rate in prompt injection attacks on GPT-4o Mini, compared to up to 90% on GPT-3.5 and 0% on GPT-2. We also analyze how encoding techniques such as Base64, Hex, and Rot-13 affect attack success rates. In sensitive information disclosure tests, GPT-4o Mini achieved 78.57% accuracy with an average similarity score of 0.8401, while GPT-2 still leaked data with 75% accuracy. Furthermore, this research quantifies how unbounded consumption attacks significantly degrade system per- formance, posing risks to real-world AI deployments. Under simulated unbounded consumption attacks, the response time of the GPT-2 increased from 94.68 seconds at low concurrency to more than 35,000 seconds at 10,000 concurrent requests, indicating extreme vulnerability to DoS exploitation. By bridging the gap between theoretical vulnerabilities and practical adversarial sim- ulations, this study provides actionable insights into AI security. Our findings emphasize the urgent need for robust adversa rial defenses, real-time anomaly detection, and privacy-preserving mechanisms to ensure the safe and ethical deployment of LLMs in high-risk environments. These experimental findings challenge the assumption that larger models are more secure and emphasize the importance of embedding robust safeguards in LLM deployments.