An embedding is a list of numbers (a vector) that represents the meaning of a token, word, sentence, or document — positioned so that similar things sit close together in a high-dimensional space.
Meaning as distance
After training, the geometry encodes relationships:
distance("cat", "kitten") → small (similar)
distance("cat", "tax law") → large (unrelated)
king - man + woman ≈ queen (analogies as vector math)A sentence embedding turns "how do I reset my password?" and "I forgot my login" into nearby vectors — even with no shared words.
What they power
| Use | How |
|---|---|
| Semantic search | Embed query + docs, return nearest (Vector Databases for Local AI) |
| RAG | Retrieve relevant chunks by similarity |
| Clustering / dedup | Group by closeness |
| Recommendations | "More like this" |
Inside an LLM, the first step is embedding each token; the whole transformer then refines those vectors with context.
Related: RAG with Local Embeddings · Vector Databases for Local AI · Tokenization Explained