AI feels weightless, but it runs on enormous physical infrastructure — data centers that draw serious electricity and water.
Where the cost goes
| Phase | Cost |
|---|---|
| Training | One frontier run = thousands of GPUs for weeks; large carbon footprint |
| Inference | Smaller per query, but billions of queries — the bigger total over time |
| Cooling | Data centers evaporate large volumes of water to stay cool |
| Hardware | Mining and manufacturing GPUs has its own footprint |
The scale problem
A single LLM query uses far more energy than a web search, and demand is exploding — data-center power draw is straining grids and reviving fossil and nuclear plants.
The "cloud" is a building full of hot silicon drinking water and electricity. The convenience is real; so is the bill the planet pays.
What reduces it
- Efficiency — smaller models, MoE, quantization, speculative decoding.
- Local/edge inference on efficient hardware.
- Renewable-powered data centers and better cooling.
- Right-sizing — don't use a frontier model for a task a small one handles (Choosing a Local LLM).
Related: Training vs Inference · Mixture of Experts (MoE) · Choosing a Local LLM