Securing an LLM app isn't one magic filter — it's layers, so that when one fails (and one will), others catch the problem. This is defense in depth applied to AI.
The layers
[ input filter ] → [ system prompt rules ] → [ MODEL ] → [ output filter ] → [ action gate ] → [ monitoring ]| Layer | Job |
|---|---|
| Input guardrails | Classify/block injections, abuse, PII |
| System prompt | Set rules — but assume it leaks and is bypassable |
| Output guardrails | Filter unsafe content, prompt leaks, hallucinated actions |
| Action gate | Least-privilege tools, human confirmation |
| Monitoring | Log, rate-limit, alert (Log Analysis and SIEM) |
The core principle
Never trust the model. The LLM is one untrusted component in the middle; the security lives in the deterministic code around it — exactly like never trusting user input in web apps.
Guardrail tools (Llama Guard, NeMo Guardrails, etc.) implement the input/output classifier layers; Constitutional AI hardens the model itself.
Related: Excessive Agency and the Confused Deputy · Securing a Public PHP App · Constitutional AI · The OWASP LLM Top 10