Reflect-Guard: Enhancing LLM Safeguards Against Adversarial Prompts via Logical Self-Reflection

Abstract

LLM safety classifiers such as Llama Guard detect overtly harmful prompts well but remain vulnerable to jailbreaks that disguise intent via role-play, fictional framing, or indirect requests. We present Reflect-Guard: Llama-Guard-3-8B fine-tuned via QLoRA to generate a distilled, GPT-4o-mini-sourced safety rationale before its verdict. Using only 1000training examples and a five-condition ablation, we isolate reflection’s contribution from that of ordinary supervised fine-tuning (SFT). SFT on verdict labels alone already accounts for most of the aggregate gain over the un-fine-tuned baseline (WildGuardTest F1: 0.770 → 0.860; JailbreakBench guard bypass rate: 10.3% → 0.0%). Adding reflection does not improve these aggregate numbers further (F1 falls to 0.842, guard bypass rises to 1.8%) but reallocates performance toward adversarially framed inputs: adversarial-subset recall rises from 0.845 (SFT-only) to 0.921 (+7.6 pp), at a cost of −7.9 pp precision on that subset. The same direction holds across three training seeds (paired t-tests, p=0.018 for overall F1 and p=0.027 for adversarial recall), though we interpret these values cautiously given the small number of seeds. Measured inference latency is 17.9× the baseline’s, driven by ∼92 additional reflection tokens per prompt. The same checkpoint, without additional fine-tuning, shows promising zero-shot transfer to a 329-example financial-harm benchmark (F1: 0.868 → 0.929; recall: 0.767 → 0.867, zero false positives); preliminary two-seed evidence, however, suggests the SFT-only condition may match or exceed this result, so we treat the finance improvement as evidence toward generalization rather than a settled advantage of reflection specifically. Benchmarked against WildGuard, PromptGuard, and a prompted (non-fine-tuned) GPT-4o-mini reasoning classifier on the same sets, WildGuard leads on aggregate WildGuardTest F1 (0.887), while Reflect-Guard achieves higher adversarial-subset recall than every evaluated classifier, including WildGuard (0.921 vs. 0.827) and the prompted teacher (0.636). Overall, supervised fine-tuning drives most of the aggregate gain over an un-fine-tuned classifier, while the generated safety rationale yields a narrower, latency-costly improvement specifically in adversarial-subset recall, a precision–recall tradeoff relevant to safety-critical deployments that prioritize recall over inference speed.