The Harness, As Feynman Might Have Taught It

A lecture hall. He walks in without notes, picks up a piece of chalk, and draws a single box on the board.

All right. Today I’m going to show you how to build an agent. And I’m going to tell you the secret right at the start, so you don’t think I’m hiding anything: the smart part, you don’t build. Somebody else built it, it cost a fortune, and you rent it by the word. What you build is the dumbest program you’ve ever written. And without it, the smart part can’t do a thing.

That dumb program is called the harness. Let me show you why you need one.

The box that only talks

He taps the box.

This is the model. Now, I want you to forget everything you’ve read about it. Forget “intelligence”, forget “reasoning”. I want to know only one thing: what goes in, and what comes out?

What goes in is text. What comes out is text. That’s it! That’s the whole interface. You hand it a page, it hands you back a page. It has no hands. It can’t open a file. It can’t look up an invoice. It can’t press a button. It doesn’t even have a clock.

So here’s a puzzle. You ask it, “What’s the total on invoice 42, in euros?” It doesn’t know. The invoice is in a database somewhere, and the box can’t reach it. What does it do?

It does the only thing it can do. It writes. It writes a little note that says, more or less: “Please run get_invoice with id = 42 and tell me what you get.” People call this a tool call, and it sounds very grand, but it’s a note. A request. Text.

And now, you see, somebody has to read that note and actually do it.

Let’s do it by hand first

He puts the chalk down.

When I was at Los Alamos during the war, we had to do enormous calculations, and we’d ordered some IBM punch-card machines to do them. Before the machines arrived, we did a funny thing. We sat people at desks, and each person did exactly one step: this one multiplied, that one added, the next one took a cube. They passed cards from desk to desk. Nobody at any desk understood the whole problem. And it worked! The people didn’t need to understand anything. They needed to follow a few rules exactly.

So we’re going to do the same thing. You are going to be the harness. I’ll be the model. Actually, I’ll play the model with a script, because I don’t want to be accused of cheating. Your job is to decide what happens around the model, one move at a time.

Now look at what you did. You called the model. It asked for something; you ran it. You wrote the answer down. You called the model again. Over and over: call, run, write down. Call, run, write down. And once, at the end: return.

And notice what you didn’t do. You never needed to know anything about invoices. You never needed to know the exchange rate. When the conversion service broke, you didn’t fix it; you wrote “it broke” in the list and let the model decide. Every decision that needed brains, the model made. Everything else was bookkeeping. And bookkeeping, as I learned at Los Alamos, is exactly the thing you can hand to a machine.

Now write down what you did

He picks the chalk back up and writes, saying each line out loud.

If you can do something by following rules, you can write the rules down. Here is everything you did, in the order you did it:

def harness(task):
    messages = [user(task)]              # the list starts with the question
    for step in range(MAX_STEPS):        # around and around...
        reply = model(messages, tools=TOOLS)   # send the whole list
        messages.append(reply)           # write its reply down
        if reply.stop_reason != "tool_use":    # no request? it's finished:
            return reply.text            #   give the text to the user
        results = []
        for call in reply.tool_calls:    # it asked for something:
            try:
                out = TOOLS[call.name](**call.input)   # run it
            except Exception as e:
                out = error(e)           # an error is just another result
            results.append(tool_result(call.id, out))
        messages.append(results)         # write the results down
    raise RuntimeError("step limit")     # ...but not forever

That’s a harness. Sixteen lines. I’m not hiding anything else; this is the whole idea.

There’s one line in there you never did by hand: the step limit. You didn’t need it, because you’re a person, and if the model had asked you for the same invoice fifty times in a row, you’d have gotten bored and stopped. The program doesn’t get bored. It will happily run all night, and every trip to the model costs money. So we have to give it the boredom you got for free.

Now, I don’t trust a description of a machine until I’ve seen what happens when you break it. If a line really matters, taking it out should break something, and you should be able to say what before you try it. So let’s take it apart.

Let’s go through what you found.

Take out the loop, and you’ve built a vending machine that takes your money and then asks you a question. The model asked for the invoice, you fetched it, and the program ended. One trip is never enough, because the model can only learn things by asking, and it can only ask one round at a time.

Take out the check for “finished?” and something almost funny happens. The model does produce the right answer: “€110.40”. It’s right there. But nobody looks, so the harness goes around again and sends the model a conversation that makes no sense, and the API throws it back. The answer was in the list the whole time and never reached the user. This is the important point: the model decides when it’s done, and the harness only notices. If the harness doesn’t notice, the decision is wasted.

Take out catching errors, and the first hiccup kills everything. The conversion service timed out once; one retry would have fixed it. But an uncaught exception doesn’t ask anybody’s opinion. Notice the fix isn’t clever. We don’t retry in the harness, we don’t guess what went wrong. We write the error down like any other result, and let the part of the system that can think, think.

Take out writing down the reply or writing down the results, one at a time, and the API refuses the next call. A result with no request, or a request with no result: either way the story doesn’t add up, and the API checks the story. Take them both out and the API is perfectly happy, because now the list just says “user asked a question”, over and over. The model can’t see that it already asked for the invoice, so it asks again. And again. With the step limit in, it stops after ten tries. Take that out too, and it’s the most expensive bug in this lecture: nothing crashes, nothing complains, and the meter keeps running.

So who’s actually smart here?

He draws a line down the middle of the board. “Model” on one side, “Harness” on the other.

People get confused about this, so let’s sort it out properly. I’ll give you some jobs; you tell me whose they are.

Did you notice the pattern? Every job the model gets is a decision: which tool, whether to retry, when it’s done, what to say. Every job the harness gets is doing or remembering or refusing. That’s not an accident. It’s the design. You put the judgment in the part that has judgment, and you put everything you need to rely on, like memory, limits and checks, in plain code that does exactly what it says.

The real thing

He wipes the board, and writes the program again. Longer this time.

Now let’s be honest and do it for real, with a real model, so you can run it tonight. The pseudocode above was the idea; here’s the same idea with the actual API’s spelling. Two tools: one lists files, one reads a file. Ask it what’s in a folder, and it’ll go and look.

import pathlib
import anthropic

client = anthropic.Anthropic()
MAX_STEPS = 20

# The hands: ordinary Python functions.
def list_files(path="."):
    return "\n".join(sorted(p.name for p in pathlib.Path(path).iterdir()))

def read_file(path):
    return pathlib.Path(path).read_text()[:20_000]

TOOLS = {"list_files": list_files, "read_file": read_file}

# The menu: what we tell the model it may ask for.
SCHEMAS = [
    {
        "name": "list_files",
        "description": "List the files in a directory.",
        "input_schema": {
            "type": "object",
            "properties": {"path": {"type": "string"}},
        },
    },
    {
        "name": "read_file",
        "description": "Read a text file and return its contents.",
        "input_schema": {
            "type": "object",
            "properties": {"path": {"type": "string"}},
            "required": ["path"],
        },
    },
]

def harness(task):
    messages = [{"role": "user", "content": task}]
    for step in range(MAX_STEPS):
        reply = client.messages.create(
            model="claude-opus-5", max_tokens=16000,
            tools=SCHEMAS, messages=messages,
        )
        messages.append({"role": "assistant", "content": reply.content})
        if reply.stop_reason != "tool_use":
            return "".join(b.text for b in reply.content if b.type == "text")
        results = []
        for call in (b for b in reply.content if b.type == "tool_use"):
            try:
                out, failed = TOOLS[call.name](**call.input), False
            except Exception as e:
                out, failed = f"{type(e).__name__}: {e}", True
            results.append({"type": "tool_result", "tool_use_id": call.id,
                            "content": out, "is_error": failed})
        messages.append({"role": "user", "content": results})
    raise RuntimeError("step limit reached")

if __name__ == "__main__":
    print(harness("What is this project? Look around, then say it in two sentences."))

pip install anthropic, set ANTHROPIC_API_KEY, and run it in any folder. Line for line, it’s what you did by hand. A few things in the real spelling are worth pointing at:

• There are two lists of tools. TOOLS is the hands: real functions. SCHEMAS is the menu: names, descriptions and argument shapes, which is all the model ever sees. The model can only order from the menu, and the harness decides what’s on it. That’s the first and cheapest safety measure you have.
• “Finished?” is a field. The API tells you why the model stopped. stop_reason == "tool_use" means “I’ve written a request, go do it”. Anything else means it’s done talking.
• Each result carries the id of its request. tool_use_id=call.id is the API’s way of checking the story hangs together: every result must answer a question that’s actually in the list. That’s the 400 you saw in Fig. 2.
• Errors are results with a flag. is_error: True tells the model “this one failed”, in a way it was trained to read. An unknown tool name ends up here too: TOOLS[call.name] raises KeyError, and the model reads that it asked for something that doesn’t exist.

What I cannot create

He writes one more line, in the corner of the board, and underlines it.

There was a sentence on my blackboard: “What I cannot create, I do not understand.” I like it because it’s a test you can’t fool yourself on. You can read about agents all day and nod along. But if you can’t write the loop, you don’t really know what’s going on.

Now you can. You were the loop, then you wrote it down, then you broke it on purpose and watched each piece fail, and then you made a real one. And the surprising thing, I hope, is how little of the cleverness is in the part you built. The harness is just bookkeeping: write it down, pass it on, don’t lose anything, know when to stop. It’s dumb, and it should be. You want the smart part and the reliable part to be different parts.