[Short Paper] Forensic Analysis of Indirect Prompt Injection Attacks on LLM Agents
Maxim Chernyshev, Zubair Ahmed Baig, Robin Doss · 2024
Large language model (LLM) agents are vulnerable to a range of evolving attacks including Indirect Prompt Injection (IPI). Digital investigations involving IPI attacks on LLM agents are challenging due to growing data volumes and agent flow complexity. We introduce a novel approach for forensic analysis of LLM agent execution logs to identify IPI attacks. We implement a custom digital forensic analysis framework featuring a realistic agentic task benchmark to generate agent logs, for subsequent analysis using 12 state-of-the-art LLMs. Our preliminary results show promising potential for malicious trail detection, with varying performance across domain-specific test suites. We discuss key findings, limitations, and future work directions to support the empirical evaluation of LLMs for digital forensic applications.