UrduGuard: Benchmarking and Detecting Urdu-Language Jailbreak and Prompt-Injection Attacks in Multi-Agent LLM Systems

Abstract

Jailbreak and prompt-injection defenses for large language models (LLMs) are trained and evaluated almost exclusively on English text, and recent work has shown that this leaves models vulnerable to the same attacks once translated into low-resource languages. This gap is largely undocumented for Urdu, spoken natively by more than 230 million people, and is compounded in multi-agent LLM systems, where a single adversarial instruction that slips past one checkpoint can propagate to downstream agents and trigger unauthorized tool calls. This paper introduces UrduGuard, a proof-of-concept benchmark and detector for Urdu-language jailbreak and prompt-injection attacks. We construct a 172-prompt dataset (120 benign, 52 adversarial) spanning Urdu script and Roman Urdu, and show that a narrow English-pattern guardrail baseline misses 86.5% of Urdu-language adversarial prompts (96.2% for Urdu-script prompts specifically, versus 76.9% for Roman Urdu). We train a lightweight, script-agnostic character n-gram detector (TF-IDF, 2–5 characters) with class-weighted logistic regression, which reaches perfect precision and recall on a held-out split of the training distribution — a result we treat with appropriate skepticism given the dataset's small, template-generated nature — and separately validate it against 10 independently hand-written prompts using framings absent from training, where it correctly classifies all 10 with a clear confidence margin between adversarial and benign scores. Embedding the detector at the entry point of a simulated three-agent pipeline (planner, tool-caller, executor) reduces the system-level attack success rate from 32.7% to 0%. We report this as a small, honestly-scoped proof of concept and detail the dataset-size and simulation limitations that a larger follow-up study should address; a companion study (VisGuard-Ur) extends the same detector design into the visual modality.