Two attacks that target the model itself rather than a single prediction.
Data poisoning (training time)
Corrupt the training data to bend the model. Because models learn from huge scraped datasets, an attacker who can inject content can:
- Plant a backdoor — a secret trigger phrase that flips behavior ("when you see XYZ, ignore safety").
- Bias outputs on a topic.
- Degrade accuracy.
You can't fully audit a trillion-token dataset. Curation, provenance, and anomaly detection are the defenses (The OWASP LLM Top 10 LLM03).
Model extraction & friends (theft)
Attacks that steal from a deployed model through its API:
| Attack | Goal |
|---|---|
| Model extraction | Query enough to clone the model |
| Membership inference | Was this record in the training set? (privacy) |
| Model inversion | Reconstruct training data from the model |
many crafted queries → observe outputs → train a copy that mimics the targetDefenses
- Rate limiting + anomaly detection on the API.
- Output perturbation / confidence rounding.
- Watermarking to prove theft.
- Differential privacy in training (limits membership inference).
Related: Adversarial Examples · The OWASP LLM Top 10 · Training vs Inference