An adversarial example is an input changed so slightly a human notices nothing, yet a model confidently gets it completely wrong — a panda nudged by invisible noise becomes "gibbon" with 99% confidence.
Why it happens
Neural nets carve up a very high-dimensional space with decision boundaries that are smooth to us but jagged up close. A tiny, carefully-chosen perturbation pushes the input across a boundary. The change is calculated using the model's own gradients (How Neural Networks Learn).
panda + imperceptible noise×ε → "gibbon" (99%)In the physical world
- Stickers on a stop sign → read as "Speed Limit 45."
- Patterned glasses → fool face recognition.
- Adversarial audio → voice assistants hear hidden commands.
Transferability (the scary part)
An adversarial example crafted against one model often fools others too — so attackers don't even need your model to attack it.
Defenses
| Defense | Note |
|---|---|
| Adversarial training | Train on adversarial examples (best, costly) |
| Input preprocessing | Blur/quantize to wash out noise (weak) |
| Ensembles / detection | Flag suspicious inputs |
Robustness remains an open problem — there's no general fix. Bridges How Image Recognition Works (CNNs) and Data Poisoning and Model Extraction.
Related: How Image Recognition Works (CNNs) · Data Poisoning and Model Extraction · AI Red Teaming