How It All Connects: The Whole Machine on One Page

There’s an old interview question for web developers: what happens when you type a URL into a browser and press Enter? A good answer takes ten minutes and passes through DNS, TCP, TLS, HTTP, a server and a renderer. Nobody expects you to know every layer in depth. The point is that you can follow the request all the way down without hitting a box labeled magic.

Here’s the same question for this series. A customer types Can I still get a refund for my lamp? into a support chat and presses Enter. Two seconds later a confident answer appears, with a source. It’s even mostly right. What happened in between?

Fifteen parts ago, the honest answer had a magic box in the middle. Now every stage has a name and a part of its own, so this last part puts them all on one page and then follows that one question through them.

Three stories, and how many ways in?

The series told three stories. How text becomes a prediction (Parts 1–4): a tokenizer turns text into IDs, each ID becomes a vector, a stack of transformer layers mixes those vectors, and out comes a score for every possible next token. How the weights get their values (Parts 5–10): from random numbers, through pre-training, fine-tuning and reinforcement learning, to a quantized file small enough to run. And how the model reaches beyond its training (Parts 11–15): retrieval, tools, a shared protocol for tools, and a loop that ties them together.

Two. Training reaches the model through its weightsWeightsThe learned numbers that scale inputs inside the network. Set during training, frozen at inference., and only before it runs: once training is over (and the weights perhaps quantized), they’re frozen, and nothing you send changes them (Part 3). Everything else, every fact about this customer, every retrieved paragraph, every tool result, every “memory”, arrives through the prompt, fresh on every call, because the model is statelessStatelessKeeps no memory between calls. The full context must be sent every time. (Part 1). (Strictly, the app’s sampler settings, like temperature or a forced JSON format, also shape what comes out. But they act on the model’s scores after it runs, not on what it knows or reads.)

Read the third story with that in mind and it stops looking like a grab bag. Embeddings, vector databases, RAGRAGRetrieval-Augmented Generation. Retrieve relevant text and add it to the prompt before generating., tool calling and MCP are all ways of getting the right text into the prompt. Tool calling also opens the only way out: the model writes a request, and your code decides whether to act on it. An agentAgentAn LLM running in a loop that calls tools, reads results, and decides next steps until a task is done. is what you get when the model helps make those choices, call after call, in a loop.

Press Enter

Here is that refund question, followed all the way through. Acme is the lamp shop from Part 12. Its support app offers the model two tools: an order lookup that lives on Acme’s MCP server, and a help-center search built on retrieval.

Three calls, 62 runs, no weights. The calls are the agent loop from Part 15: the model asked for the order, then for the policy, then answered, and the loop stopped because the last reply asked for no tool. The runs are the next-token loop from Part 1, one per token written. The first run of each call reads that call’s whole input in one parallel pass, which is the prefill from Part 3; every later token costs one more pass. And the input grew each time, 572 tokens, then 619, then 733, because the model keeps nothing between calls: the app re-sends everything. (Providers can cache the work on a long repeated prefix and bill it at a discount, as Part 1 showed. That makes the re-reading cheaper, but the model still receives every token, every call.)

Now look at where the answer came from. “45 days ago” came from the order system, through a tool call and MCP (Parts 13–14). “60 days” and “original packaging” came from a retrieved paragraph (Part 11, Part 12). The weights supplied everything else: reading the request, deciding to look things up, writing valid JSON, rewriting the search query (which put the 60-day window first, at a cost you’ll see in a moment), and phrasing the answer, short as the saved note asked, with a citation. That’s the division of labor the map predicted.

Where the eight billion numbers came from

The trace treated the eight billion weights as given. Someone made them, one stage at a time, and each stage learned from something different.

To watch that whole pipeline at once, I need a model small enough to see. The one below is the last step of an LLM at toy size. A vector standing for the prompt (6 numbers instead of 4,096) is multiplied by one row of weights per possible output, each row gives a score, and softmax turns the scores into odds (that’s exactly how the transformer in Part 4 ends). Its whole vocabulary is five replies to What is the capital of France? That’s thirty weights, and every stage trains them for real.

Pre-training lands exactly on the web’s odds: another quiz question 50%, “Paris.” 12%. That’s the result from Part 5: cross-entropy rewards betting the true odds, so the loss bottoms out at 1.255, the unpredictability of the data itself, and no lower. The model “knows” Paris and still prefers to continue the quiz, which is Part 7’s base model in miniature. Backpropagation (Part 6) supplies every nudge, and with one layer the gradient fits on a line: for each weight it’s (p − q) × h, the reply’s predicted minus true probability, times that weight’s input. At the start, “Another question” has p = 0.31 against the web’s q = 0.50, so for its first weight (h = 0.9) the gradient is (0.31 − 0.50) × 0.9 ≈ −0.17. At learning rate 0.5 that weight rises by about 0.09, from 0.30 to 0.39. Do that for all 30 weights, 60 times, and you have Fig. 3’s pre-training.

Fine-tuning moves all 30 weights, 33 steps at a smaller learning rate, and “Paris.” jumps to 62%. LoRA gets past the same mark without touching any of the 30. It trains a 5-number column and a 6-number row whose product is added to every weight (Part 9), and because that column starts at zero, its first steps barely move. Don’t be too impressed, though. With only one prompt, the change full fine-tuning makes is itself a column times a row (every step adds a multiple of h to each row), so an 11-number adapter can always keep up here. The surprise in Part 9 is that a low-rank change often does nearly as well in a real model, where millions of different prompts pull in different directions.

RLHF never shows the model an answer, only scores. It ends at 91.8% “Paris.”, exactly where Part 8’s formula, π* ∝ π_ref · e^r/β, says it should: gradient steps on scores found the same place the algebra did (Part 8). Quantization then rounds every weight to a 4-bit code, and no weight moves by more than about 0.12. A score can move further than that, because it adds up six products, each weight times its number in h. If all six errors lined up, a score would move by 0.12 × 3.4 ≈ 0.4 (3.4 is h’s six numbers added up, ignoring signs). Here the errors have mixed signs and partly cancel: the biggest shift is 0.07, and the odds barely change (91.8% → 91.5%). Even in the worst case, the leader loses 0.4 and a rival gains 0.4, so only replies within 0.8 of each other could swap places, and “Paris.” leads by more than 3 (Part 10). In a real model each of these sums has thousands of terms, and the errors flow through dozens of layers, so whether they cancel or pile up is something you find out by running it. Then the numbers freeze, and you can ask a thousand times without changing one.

Five ideas that kept coming back

The triangle. You watched it in Fig. 2: to write 62 tokens, the model read 572 + 619 + 733 = 1,924, because every call re-reads everything before it. One call’s bill grows in step with its length. But when every step works over everything before it, the total grows like 1 + 2 + … + n, so doubling the length roughly quadruples it. That’s the long chat in Part 1, attention’s comparisons inside one call in Part 4, and an agent’s context in Part 15.

Bytes, not arithmetic. Writing each token means streaming every weight from memory (Part 3): 16 GB for an 8B model at 16 bits. At the roughly 3 TB a second of an H100-class GPU, that’s about 5.4 ms per token, so one conversation tops out near 185 tokens a second however fast the arithmetic is, and at 4 bits (about 5 GB) the ceiling roughly triples. That’s part of why output tokens cost more (Part 2), why fewer bits per weight speed up generation (Part 10), and why providers batch requests: one read of the weights serves many users.

The model only proposes. In Fig. 2 the customer’s identity came from the signed-in session, never from the model’s JSON, and the app, not the model, sent the MCP request. The model writes text; code does things. A sampler picks the token, the app runs the tool, the MCP client sends the request. So the safety controls live in code: validation, least privilege, confirmation (Parts 13–15).

Untrusted text travels. The three pasted chunks and the order record sat in the same context as Acme’s instructions, with only formatting to tell them apart. Retrieved chunks, tool results, tool descriptions and web pages all become tokens like that, so prompt injectionPrompt injectionMalicious instructions hidden in content the model reads (web pages, documents, tool results). follows the text wherever it goes (Parts 12–15).

You get what you measure. Fig. 3’s base model learned exactly what web text rewarded: another quiz question at 50%, the answer at 12%. Overfitting (Part 6), flawed examples turning into habits (Part 7), reward hacking and sycophancy (Part 8) are the same story: training delivers exactly what the loss or the score rewards, flaws included.

Every term in one place

My notes end with a glossary, and so does the series. Here are all of its terms, each linked to the part that explains it. Try quiz mode: if you can define a term before you reveal it, that part did its job.

That’s the whole machine, and no box on it says magic. If one still feels a little like it, its part is one click away on the map, and the card below goes back to where it all started: a model that guesses the next token and remembers nothing.