A neural network "learns" by adjusting its weights to make fewer mistakes. The whole process is a feedback loop run millions of times.
The loop
1. Forward pass — feed input through the network, get a prediction
2. Loss — measure how wrong it was (loss function)
3. Backprop — compute how each weight contributed to the error
4. Update — nudge each weight a little to reduce the error
repeat ...Gradient descent, intuitively
Imagine the loss as a hilly landscape; the network is trying to walk downhill to the lowest valley (least error). The gradient is the slope — which way is down. Each step, every weight moves a bit downhill:
weight = weight − learning_rate × gradient- Learning rate too big → bounce around, never settle. Too small → crawl forever.
- Backpropagation is just the chain rule from calculus, applied efficiently to share the "blame" for the error across millions of weights.
Why it works at scale
No one programs the rules. You define the architecture (How Transformers Work) and a loss, then gradient descent discovers the weights that minimize error. Scale the data and the network, and surprisingly capable behavior emerges.
This is the engine behind training; it's also what Adversarial Examples exploit (running the gradient backward to fool the model).
Related: Training vs Inference · What Is Machine Learning · How Transformers Work