Go to the help center of an online shop and type how do I get my money back. A plain keyword search looks for pages containing those words, and the page you need contains none of them. It’s called “Refund policy”, and it says “Refunds are available within 60 days of purchase.” Not one word in common. Ask the same question at a shop counter and you’re pointed to the right desk in a second. The person behind the counter isn’t matching words. They know what you mean.
You can give a computer that ability with nothing but arithmetic, by turning meaning into a place: give every text a set of coordinates, arranged so that texts with similar meanings land near each other. Then “find the pages about my question” becomes “find the points nearest to my question’s point”, which is geometry, and computers are very good at geometry.
A map where nearby means similar
You already use a system like this. Latitude and longitude turn “is Boston near New York?” into arithmetic. We can do the same for things, as long as we decide what the axes mean.
Pick two properties and score each from −1 to 1: how round something is, and how fruit-like. An apple is quite round and very fruity: (0.8, 0.9). An orange is a bit rounder and a bit fruitier: (0.9, 0.95). A car is neither: (−0.6, −0.9). Each thing is now a list of two numbers, which you can draw as a point, or as an arrow from the center of the map to that point. A list of numbers like this is a vectorVectorAn ordered list of numbers. Here, an embedding., and a vector that stands for the meaning of something is called an embeddingEmbeddingA vector of numbers representing the meaning of a token or a piece of text..
Now “how similar are an apple and an orange?” has an answer we can compute. Two answers, in fact.
The first ruler is the straight-line distance between the tips, the Euclidean distance, which is just Pythagoras. Apple and orange differ by 0.1 in roundness and 0.05 in fruitiness, so their distance is √(0.1² + 0.05²) ≈ 0.11: very close. Apple and car differ by 1.4 and 1.8, giving √(1.4² + 1.8²) ≈ 2.28: far apart.
The second ruler ignores how far out the tips are and asks only which way the arrows point. That’s cosine similarityCosine similarity(a·b)/(‖a‖‖b‖). Measures how closely two vectors point in the same direction, from −1 to 1., the cosine of the angle θ between them, and the coordinates give it to you without measuring any angle:
For apple and orange, the dot product is 0.8 × 0.9 + 0.9 × 0.95 = 1.575. The lengths come from Pythagoras: ‖apple‖ = √(0.8² + 0.9²) ≈ 1.204 and ‖orange‖ = √(0.9² + 0.95²) ≈ 1.309. So cos θ ≈ 1.575 / (1.204 × 1.309) ≈ 0.9995, an angle of under two degrees. Apple and car come out at about −0.99, almost exactly opposite. A cosine runs from 1 (same direction) through 0 (right angles) to −1 (opposite).
That answers the prediction. The switch slides every tip along its own arrow onto the circle: the apple–orange distance drops from 0.11 to 0.03, but the cosine stays at 0.9995, because changing an arrow’s length doesn’t turn it.
Many embedding models output length-1 vectors anyway, since for them what matters is the direction. Then the rulers agree. The dot product equals the cosine, because the bottom of the fraction is 1 × 1. And the distance follows from the cosine: the squared distance is (a − b) · (a − b) = ‖a‖² + ‖b‖² − 2 a · b = 1 + 1 − 2 cos θ, so the distance is √(2 − 2 cos θ), and a bigger cosine always means a smaller distance. All three sort any list of candidates in the same order, so for normalized vectors the cheapest, the dot product, is a common choice.
Vector databases still let you pick among three “metrics” (cosine, dot product, and Euclidean, also called L2), because for vectors that aren’t normalized the three can disagree: the dot product grows with length. Against a query (1, 0), the candidate (0.9, 0.1) points almost the same way (cosine 0.99, dot product 0.9), while (2, 1.5) is 37° off (cosine 0.8) but long, with a dot product of 2. Rank by dot product and the long, off-target vector wins. Use the metric your embedding model’s documentation recommends.
From fruit to sentences
Real embeddings work the same way, with two big changes. First, nobody chooses the axes. An embedding model is a neural network that reads a text, anything from a sentence to a whole passage, and returns one vector for all of it. Its designers fix how many dimensions it has, typically somewhere between 384 and 3,072; training decides what each one encodes, and none ends up with a readable name like “roundness”: meaning is spread across all of them at once.
Second, the model has to learn to put similar texts close together. One common recipe trains it on huge numbers of pairs that belong together, such as a question and its answer, pulling partners together and pushing random pairs apart, which is called contrastive training. Afterwards, “how do I get my money back” and “Refunds are available within 60 days” point in similar directions, much closer to each other than to unrelated text, without sharing a word, because questions like the first are so often answered by text like the second.
A database that searches by meaning
A vector databaseVector databaseDatabase that stores embeddings with their source text and finds nearest neighbours by similarity. stores these vectors and makes them searchable. Strip away the features and it does three things:
- When you add a document, it stores the document’s embedding next to the original text.
- When a question arrives, the question is embedded with the same embedding model (by the database itself if it has a model built in, more often by your application before it asks).
- It returns the stored texts whose vectors are nearest to the question’s vector.
That’s the whole idea; everything else is engineering to make step 3 fast. Here it is in miniature, with eight help articles and a toy model whose six dimensions I’ve named so you can read them.
“Refund policy” wins with a cosine of 0.95, and the line under the list confirms that it shares no words with the question. The named dimensions show why: the question lights up money and undo, and so does “Refund policy”, because the toy model knows a refund is money coming back. “Stop paying every month” finds “Cancel a subscription” (0.97) the same way.
Then the switch. Model B is no worse a model: it knows the same words and ideas, but keeps them in different slots. That alone wrecks everything. The question lights up the wrong columns, “Refund policy” drops to 0, and the top hit becomes “Reset your password”. Nothing complains: a cosine happily compares any two lists of numbers of the same length, meaningful or not.
Why “unrelated” scores near 0, not −1
Look back at Fig. 2. For the money-back question, the articles that have nothing to do with refunds scored between 0.05 and 0.18, not −1. Yet in Fig. 1 the car scored −0.99 against the apple. Which picture tells the truth about unrelated things?
The toy can’t settle it, since its numbers are never negative. The answer comes from how much room there is in hundreds of dimensions. Pick two directions at random and measure the cosine between them, two thousand times over. In two dimensions, where do most of the cosines land? And in 1,536?
In two dimensions, random directions are often nearly aligned or nearly opposite: two pairs in three have a cosine beyond ±0.5. Add dimensions and the histogram collapses into a spike at zero: at 384 dimensions, 95% of pairs land within about ±0.10, and at 1,536 within ±0.05. Random directions in a high-dimensional space are almost always nearly perpendicular.
Why? The cosine of two length-1 vectors is their dot product, a sum of d tiny products, one per coordinate, each as likely to be positive as negative. Each coordinate is about 1/√d in size (the squares add up to 1), so each product is about 1/d, and d of them with random signs mostly cancel, like a random walk that ends only about √d steps from home: the total is typically √d × 1/d = 1/√d. At 384 dimensions that’s about 0.05, and 95% of pairs fall within two of those, the ±0.10 you saw; quadruple the dimensions and the spread halves. Pointing the same way or opposite ways would need hundreds of coordinates to line up at once, and chance essentially never arranges that.
Meaning doesn’t arrange it either. No text is the opposite of a refund policy, and even words we call opposites sit close together: “hot” and “cold” turn up in the same kinds of sentences, so their embeddings point in similar directions. To an embedding model, unrelated means having nothing in common: roughly perpendicular, a score near 0. The −0.99 for apple and car in Fig. 1 came from my scoring, which made “not a fruit” negative; a trained model has no reason to put cars opposite apples.
Finding the nearest of ten million
Back to step 3. The obvious way is to compare the question with every stored vector and keep the best. That’s exact, and fine for tens of thousands of vectors, often more. At scale, count the work: ten million 1,536-dimensional embeddings means ten million dot products of 1,536 multiply-adds each, about 15 billion operations per search. Worse, every stored vector must be read to do them: 10,000,000 × 1,536 numbers × 4 bytes ≈ 61 GB streamed through the processor per question, over half a second even at 100 GB/s.
So large vector databases cut a corner, carefully. An approximate nearest neighbor (ANN) indexANN indexApproximate-nearest-neighbour index (e.g. HNSW, IVF) for fast vector search. finds almost the nearest vectors while measuring only a small fraction of them. Now and then it returns the second- or fifth-closest vector instead of the closest; the share of true nearest neighbors it does return is its recall.
The most widely used ANN index is HNSW (Hierarchical Navigable Small World; “small world” as in six degrees of separation), and the idea is a road trip. To reach a street in another city you don’t inspect every street in the country: you take the highway to the right region, main roads to the right district, then local streets to the door. HNSW builds that out of links between vectors. The bottom layer holds every vector, each linked to a few near neighbors. Each layer above holds a random, much smaller subset, with longer links. A search starts at a fixed entry point on the top layer and hops greedily to whichever linked neighbor is closest to the query, until none is closer; then it drops a layer and carries on. On the bottom layer it keeps a shortlist of the best candidates found so far instead of just one. The shortlist’s length is called ef.
With 300 points and ef = 1, a search measures about 21 points instead of 300 and finds the true nearest neighbor for 95% of the test queries. When it misses, the magnifier in the corner shows why. The search stopped at the red diamond because every one of its links (the red lines) leads farther from the query. The true nearest point, the green circle, is inside the dashed circle, so it really is closer, but reaching it takes a detour that a greedy search never makes. Raise ef to 2 and recall climbs to about 99%; at 4 it’s 100%, for a few more distance computations. That knob, more work for more recall, exists in every real HNSW deployment.
Now set ef back to 1 and switch to 3,000 points. Brute force gets ten times more expensive, while the average HNSW search goes from about 21 distances to about 29. That slow growth, roughly logarithmic in the size of the collection, is why HNSW is the default index in so many vector databases.
Don’t carry the 21 over to production, though. In two dimensions four links per point cover every direction; in hundreds of dimensions real indexes keep something like 16 to 64 and a much longer shortlist, so a typical query measures hundreds to a few thousand vectors. Out of ten million, still a sliver. HNSW’s costs are memory for all those links and slower inserts, since each new vector runs a search to find its neighbors.
HNSW links vectors; the other big family groups them. IVF (inverted file index) clusters the vectors once, with an algorithm such as k-means, and files each one under its nearest cluster center: one list per cluster, like the page lists in a book’s index. A query is compared with the centers, then only with the vectors filed under the few nearest, so most of the collection is never touched. It misses when the true neighbor sits just across a cluster border; probing more clusters buys that recall back.
Product quantization (PQ) goes after memory instead. It chops each vector into short pieces and replaces every piece with the number of its closest entry in a small learned codebook, a cousin of the quantization in Part 10. Cut a 1,536-dimensional vector into 96 pieces of 16 numbers, give each piece a codebook of 256 entries, and each piece fits in one byte: 96 bytes instead of 6,144 (1,536 numbers at four bytes each), 64 times smaller, with distances computed approximately from the codes. The two are often combined as IVF-PQ. All of these trade a little recall for a lot of speed or memory.
You rarely build any of this yourself. Dedicated vector databases such as Pinecone, Weaviate, Qdrant, Milvus and Chroma do it as their main job, and existing databases have added it: the pgvector extension for PostgreSQL, and vector search in Elasticsearch, OpenSearch and Redis. Most offer HNSW, several IVF too. The math is the same everywhere, so the choice is mostly about operations: where your data already lives, how big it will get, how you need to filter it.
Where meaning alone falls short
Embeddings capture what a text is about. They’re poor at exact strings that don’t mean anything: error codes, product SKUs, version numbers, names. A real embedding model does see the characters, but E-4012 and E-4013 mean almost the same thing (some error code), so they land nearly on top of each other, and in a whole sentence one changed digit barely moves the vector. Someone who pastes “getting E-4012 at checkout” wants the one article that mentions E-4012, and a pure vector search may rank a general article about declined payments above it.
Keyword search has the opposite strengths. Its standard scoring rule, BM25, rewards a document for containing the query’s words, with three refinements: rarer words count for more (E-4012 appears in one article, so it’s worth a lot), repeats help with diminishing returns, and long documents are discounted so they can’t win just by containing more words. If an article contains the code exactly as typed, BM25 all but guarantees it a place near the top (typed as E4012, though, it’s a different word to BM25). It also has no idea that “money back” means “refund”.
Hybrid search runs both and merges the two ranked lists. Adding the raw scores doesn’t work: a BM25 score of 2.78 and a cosine of 0.95 aren’t on the same scale. You could rescale both first, but a simpler, very popular fix uses only the positions, with reciprocal rank fusion (RRF):
A document near the top of either list scores well; one near the top of both scores better. With k = 60, first place is worth 1/61 ≈ 0.0164 and fifth place still 1/65 ≈ 0.0154, so agreement between the lists counts for more than any one rank. I’ve added three articles to the toy help center: “Why was my payment declined?”, “Error E-4012” and “Error E-4013”. For each question below, guess first, then flip between the methods: does keyword search, meaning search, or both put the right article in the top three?
Neither search wins all three. Keywords nail the error code, where meaning buries the right article at #5, a hair ahead of the article about E-4013 (to my toy model, the two codes are the same word). Meaning nails “how do I get my money back”, where keyword search returns nothing at all: once common words like “how” and “my” are dropped, no article contains “get”, “money” or “back”. The parcel question is the easy case: both lists put “Track your order” first. Hybrid puts the right article first every time. The E-4012 article’s fused score is 1/61 + 1/65 ≈ 0.0318, nearly double the 1/61 ≈ 0.0164 of the meaning search’s favorite, “Why was my payment declined?”, which only one list returned.
Hybrid isn’t magic, though: RRF is a compromise, and with other documents a keyword fluke can outrank the best semantic match. That’s one reason many pipelines add a re-ranker, a slower model that reads the question and each candidate together (Part 12).
One last trap concerns what goes into the database. An embedding is one vector per piece of text, however long the text. Embed a forty-page manual as a single vector and two things go wrong. Most embedding models read only a few hundred to several thousand tokens and drop the rest, often without an error, so the vector may describe just the opening pages. And a model that reads it all gives you something like the average of forty pages of meaning: vaguely about everything, a strong match for nothing. Embed single sentences and each vector is sharp but may lose what it refers to: “It must be requested within 60 days.” What must? So documents are split into passages before embedding, a step called chunkingChunkingSplitting documents into passages before embedding them for RAG. (Part 12 puts it to work), and how you cut them often matters as much as which model you pick.
We now have a machine that finds the passages relevant to a question, even when they share no words with it. Next, hand those passages to an LLM along with the question, and it can answer from documents it never saw in training.