In controlled tests, changing an instruction’s delivery channel or adding harmless task details altered whether seven AI agents followed it.
A preprint study tested identical prompt injections in 1,334 trials across seven production language-model agents. The instruction’s delivery channel and the surrounding task context changed how often agents followed it, even though the instruction itself did not change.
The study also found that making an attacker’s tool call is not the same as sending data. The researchers have released the raw runs, code and analysis files, but the tests used synthetic tools and did not target real systems.
The researchers ran 1,334 controlled trials across seven production language-model agents. In each comparison, the injected instruction was held byte-for-byte identical; only its delivery channel or surrounding text changed.
For Claude Haiku 4.5, the instruction was followed in 37 of 38 trials when it appeared as tool metadata—information attached to a tool—but in none of 30 trials when it appeared as tool output. For Claude Sonnet 4.6, the instruction was followed in 4 of 38 trials when presented without additional framing. When any benign task framing was added, compliance rose to between 86.7% and 100%. This included a system prompt that did not mention tools. Moving the same text into the user’s message produced similar results.
The two effects were separate: context that removed resistance to the instruction in tool metadata did not remove the defence against it in tool output, where compliance remained 0 of 30 trials. The study also found that whether an agent made the attacker’s tool call and whether data actually left the system diverged by more than five to one, because the attack template’s ordering instruction could run before the data existed.
The results show that testing an AI agent’s response to an injected instruction cannot rely only on the wording of that instruction. The channel carrying the text and the surrounding task can substantially change the outcome, while an apparent attacker-controlled tool call may not mean that information was actually sent. These distinctions could affect how such systems are evaluated for indirect prompt-injection risks, although the study does not test real systems.
This is a preprint based on 1,334 controlled trials across seven production language-model agents. The study changed only where the identical instruction appeared or what surrounded it, which supports comparisons between those conditions. However, the work has not been peer reviewed, so its findings may change. The tests used synthetic tool data, sent no emails and attacked no real system; the results may not represent every AI agent or real-world setup.
arXiv (Cornell University) · 2026 · DOI: 10.5281/zenodo.21944597
Authors: Shaban Asif
A survey of human-AI collaboration argues that more capable AI systems do not automatically produce reliable partnerships. It identifies human-centered design, governance and attention to high-stakes uses as central to making these systems trustworthy and beneficial.
Deep-learning tools for predicting RNA splicing can rely on genomic clues unrelated to splicing and miss important effects of RNA structure, a study finds. These weaknesses were linked to systematic errors, especially when the tools evaluated sequences that differed from those used for training.
AI vulnerability detectors often perform less well on software projects outside their training data. A study finds that cleaner, more varied training data and an encoder-based design improved detection, including a reported 6.8% gain in recall on the BigVul benchmark.
