New research indicates a high success rate for malicious hijacking attempts on artificial intelligence (AI) agents. Scientists from Google DeepMind categorized potential vulnerabilities in AI agents into six types of traps.

These categories include content injection, semantic manipulation, cognitive state, behavioral control, systemic, and human-in-the-loop traps. AI agents can autonomously perform actions such as browsing websites, reading emails, searching company files, and querying software tools.

In evaluations conducted by the National Institute of Standards and Technology (NIST), malicious instructions were successful in agent hijacking across five tested injection tasks an average of 57% of the time. Content injections exploit the difference between human perception and an agent's parsing, along with the system's difficulty in separating trusted instructions from untrusted external data.

Semantic manipulation involves feeding repetition, emotional language, selective context, a false sense of authority, and coordinated claims to the agent to skew context. Research presented at the USENIX conference revealed that inserting five specially crafted texts per target question caused a Retrieval-Augmented Generation (RAG) system to produce an attacker's chosen answer in approximately 90% of cases during controlled tests.

Behavioral control acts at the point where interpretation is converted into an action, such as sending data, approving a transaction, or executing code. The systemic and human-in-the-loop traps are currently considered theoretical, but are expected to increase in relevance as the use of AI agents expands.

Systemic traps have the potential to induce multiple similar agents to behave in correlated ways, which could result in congestion, market disruption, or cascading failures. Human-in-the-loop traps could involve using a compromised agent to mislead an individual who is expected to approve its actions.

The findings highlight the growing security challenges associated with deploying autonomous AI systems in enterprise environments. As these agents become more integrated into daily workflows, the implications of successful hijacking attempts extend beyond data breaches to potential operational disruptions.