Reinforcement Learning-Based Adaptive Honeypot Systems for Mitigating Cyber Threats in Cloud-Based Enterprise Environments
Alex, Jisha · TRAP@NCI (National College of Ireland) · 2025
This research presents a reinforcement learning-based adaptive honeypot framework designed to proactively detect and respond to modern cyber threats such as brute-force attacks, port scans, ransomware, and Structured Query Language (SQL) injections. Unlike traditional static honeypots, this system incorporates a dynamic decision-making model capable of evolving based on attacker behaviour. The implementation leverages a tri-Virtual Machine (VM)setup comprising a Cowrie honeypot, a synthetic attacker environment, and a Reinforcement Learning (RL) agent trained using Q-Learning and Deep Q-Network (DQN). The system was evaluated using synthetic logs and the TON_IoT dataset to simulate real-world attack scenarios. The RL model achieved an overall accuracy of 83%, with perfect detection of brute-force and port scan attempts and weighted average of 92% for F1 Score on synthetic cowrie-based honeypot data. Deployment on Amazon Web Services (AWS) infrastructure confirmed the system’s scalability and operational readiness. This research contributes to the advancement of intelligent, real-time cybersecurity defence systems and provides a deployable blueprint for protecting enterprise and IoT-based networks against evolving threats through behavioural analytics and reinforcement learning.