Detecting Prompt Injection Attacks on Purpose-Specific LLM Agents

Large language models (LLMs) are increasingly used as purpose-specific agents for tasks like customer service and code generation. These agents must adhere to both general safety guidelines and specific restrictions tailored to their roles. However, these additional restrictions expand the attack surface, particularly for prompt injection (PI) attacks. Current detection methods focus on input-output patterns but are limited in effectiveness. This study analyzes the hidden activation space of LLMs, revealing that they retain latent policy-violation (PV) concepts when faced with requests outside their intended purpose. We propose PVDetector, a training-free framework that detects PI attacks during LLM inference by measuring hidden-state alignment with PV concepts derived from policy-violating and compliant prompts. Experiments show PVDetector achieves a <1% false negative rate with minimal overhead, outperforming existing methods.