What Are Embeddings? Vector Embeddings and Vector Databases Explained

The intuition: meaning as coordinates

Imagine placing every word on a map where distance means difference in meaning. “Cat” lands near “kitten” and “dog,” far from “mortgage.” Now give that map not 2 dimensions but 768 or 1,536 — enough room to encode topic, tone, formality, domain and countless subtler features. That’s an embedding space.

The famous illustration from early word embeddings: vector("king") − vector("man") + vector("woman") lands close to vector("queen"). Directions in the space can correspond to concepts. Modern sentence embeddings are richer still: they represent whole passages, so “How do I reset my password?” sits right next to “I forgot my login credentials.”

Measuring similarity: cosine similarity (live demo)

Cosine similarity measures the angle between two vectors, ignoring their length:

cosine(A, B) = (A · B) / (‖A‖ × ‖B‖)

 1.0  → same direction  (same meaning)
 0.0  → perpendicular   (unrelated)
-1.0  → opposite

Other metrics you’ll see are dot product (equivalent to cosine when vectors are normalised to length 1, which many models do) and Euclidean distance. Use whichever your embedding model’s documentation recommends.

How embeddings are created

Embedding models are neural networks — typically transformer encoders — trained so that related pairs end up close and unrelated pairs far apart. A common recipe is contrastive learning: show the model a question and its correct answer (a positive pair) alongside unrelated passages (negatives), and adjust it until positives score higher. After training on millions of such pairs, the model generalises to text it has never seen.

Every input produces a vector of the same fixed length, whether it’s three words or three paragraphs — though very long inputs get truncated at the model’s token limit, which is one reason documents are chunked first.

Embeddings in Python

An open-source example using the sentence-transformers library, which runs locally:

# pip install sentence-transformers
from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("all-MiniLM-L6-v2")   # small, fast, 384 dimensions

docs = ["Your payment is late. Please settle the balance.",
        "Our office is closed on public holidays.",
        "How to reset your account password"]
query = "invoice overdue"

doc_vecs = model.encode(docs, normalize_embeddings=True)
q_vec    = model.encode(query, normalize_embeddings=True)

scores = util.cos_sim(q_vec, doc_vecs)[0]
for doc, s in sorted(zip(docs, scores), key=lambda x: -x[1]):
    print(f"{s:.3f}  {doc}")
# The payment sentence ranks first despite sharing no words with the query.

Hosted APIs from model providers work the same way: send text, receive a vector, store it.

What is a vector database?

Comparing a query against every stored vector (brute force) is fine for thousands of items but too slow for millions. Vector databases solve this with approximate nearest-neighbour (ANN) indexes that trade a tiny amount of accuracy for huge speed-ups:

• HNSW (Hierarchical Navigable Small World) — a layered graph you “hop” through toward the nearest vectors. Fast and accurate; the most common default.
• IVF (inverted file) — clusters vectors and searches only the closest clusters.
• Quantisation (e.g., product quantisation) — compresses vectors to cut memory.

They also handle the boring-but-essential parts: metadata filtering (“only docs from 2026, region = EU”), updates and deletes, persistence and scaling.

Choosing an embedding model

• Benchmark on your data. Public leaderboards such as MTEB are a starting point, but 50 real queries with known correct documents tell you more.
• Language and domain. Need multilingual or code search? Pick a model trained for it.
• Dimensions vs cost. Larger vectors cost more storage and slower search; some models support shortened vectors.
• Max input length. Must fit your chunk size.
• Hosting and privacy. API models are easy; local open-source models keep data in-house.

Use cases beyond RAG

Pitfalls to avoid

• Exact-match blind spots: embeddings can blur IDs, SKUs and error codes — add keyword search (hybrid).
• Negation: “refund allowed” and “refund not allowed” can look very similar. Don’t rely on similarity alone for decisions.
• Poor chunks in, poor vectors out: a chunk mixing three topics produces a muddy vector.
• Similarity thresholds don’t transfer: a “good” score differs by model — calibrate on your data.