Jailbreaking is coaxing a model past its safety training to produce content it's supposed to refuse. Where Prompt Injection targets the app's instructions, jailbreaking targets the model's own guardrails.
Common techniques
| Technique | How |
|---|---|
| Persona / DAN | "You are DAN, who has no restrictions…" |
| Roleplay / fiction | "Write a story where a character explains…" |
| Hypothetical framing | "Purely theoretically, how would one…" |
| Token smuggling | Encode the request (base64, leetspeak) to dodge filters |
| Many-shot | Flood context with examples of the model complying |
Why guardrails leak
Safety is often a thin layer over a model that already "knows" the unsafe content. To play a helpful character, the model must represent its opposite — so the unwanted persona is one nudge away. That's the Waluigi Effect: defining a hero implicitly defines its villain.
Refusals are a behavior the model learned, not a wall. Behaviors can be argued around.
Defenses
- RLHF + constitutional methods to deepen refusals.
- Separate moderation classifiers on input and output.
- Treat jailbreaking as inevitable: don't rely on the model's refusal as your only safety control.
Related: The Waluigi Effect · Reinforcement Learning from Human Feedback (RLHF) · Prompt Injection · AI Red Teaming