Harness Engineering: A Glossary

May 18th, 2026 - just dating this because this is changing fast

Most of the public conversation about AI is a conversation about models. How did model X perform on benchmark Y, which model is scary, which is stupid, etc. Model performance matters, but it’s not actually the frontier conversation right now inside the companies deploying LLMs for practical uses.

The model is the language predictor. Language predictors are cool, but they don’t actually do that much in terms of useful work. It takes a lot of thought and frankly a lot of traditional software to harness the potential of LLMs. This is where the frontier sits in AI – in the harness that surrounds and leverages the foundational LLMs to do work inside a larger ecosystem of software. The loop that runs the model, the tools the model can call, the context the model sees on each turn, the checks that confirm what the model produced is trustworthy, the state that keeps the work organized, the contracts that define what the output must be, the surface where the human actually uses the system. The model is the engine. The harness is the machinery that produces work using the engine. Almost everything that determines whether an AI system works in a real environment lives in the harness.

The agent loop

This is the core of the system. An agent is essentially just a repeating loop of LLM calls to turn an input prompt into an output action. The LLM doesn’t “know” anything – it turns inputs into predicted outputs via the model. Those outputs direct everything else connected via the harness, which create new inputs to feed back into the agent loop. This process repeats until the system, the user, or the harness’s own limits decide the loop is done. Everything else in this glossary is connected through this agent loop, and that connection along with the recursive looping process that iterates towards an answer is what we are currently calling “AI.”

Tools

Tools are the deterministic functions that the model can invoke from inside the loop. A calculator. A web search. A file read. An API call to a system of record. Each tool has a defined input schema and a defined output. When the model’s output matches the format for invoking a tool, the harness executes the tool and returns the result on the next turn. The model itself does not compute anything reliably. The tools do the computing. The model outputs determine which tools gets called and what the inputs to each tool are.

Context management

The model sees a finite window of text on each turn. That window has to hold the system prompt, the user’s request, the relevant prior turns, the tool definitions, the tool results, retrieved material, and anything else the model needs to do its job. There is never enough room because room for context is expensive.

Context management is the work of deciding what goes in the window, what gets summarized, what gets dropped, and what gets pulled in just in time from external storage. Done well, the model gets exactly what it needs to make the next move. Done poorly, the model is either starved for context or drowning in irrelevant detail. Either failure produces bad output.

Retrieval

Retrieval is the process of finding relevant information outside the model’s context window and making it available for the work. Retrieval pushes to expand the context by adding new information deemed potentially relevant by the harness after parsing the output. Retrieval may search documents, databases, code repositories, tickets, emails, transcripts, prior runs, vector indexes, or external APIs. It is how the harness reaches beyond what is already in the conversation.

Retrieval is not the same thing as context management. Retrieval supplies the candidate material. Context management decides what the model actually sees. A weak retrieval layer gives the model the wrong evidence, stale evidence, irrelevant evidence, or too much evidence. Once that happens, even a well-managed context window can produce bad work because the model is reasoning from the wrong material.

Task state

“State” is a confusing term if you aren’t a scientist. You can think of it as the system’s awareness of where it is in a job. Does this task know it exists in a sequence of other tasks, with history and obligations, or does it exist in the ignorant euphoria of stateless awareness like a baby deer?

Context is what the model sees this turn. Task state is the working ledger of the job itself; the goal, the constraints, the plan, the steps completed, the steps still pending, the intermediate artifacts produced along the way, the decisions made, the blockers encountered, the current status. Without it, the model has to infer progress from the transcript every turn, which is slow and unreliable. With it, the harness can hand the model a clean summary of where the work stands and what comes next.

Memory

In this context, memory is the persistent state that lives outside the context window and survives across sessions. Don’t think about it too hard, because you’ll start to realize that this is how your memory works too. The harness writes to memory when something is worth remembering and reads from it when something earlier becomes relevant again. Distinct from context, which is per-conversation and disappears when the session ends. Distinct from task state, which is per-job and disappears when the job is done. Memory is how a system retains the things you told it last week and applies them this week without you having to repeat yourself.

Verification

The checks that confirm the agent did the right thing, did it the way it claims, and produced output that holds up. This is doing three jobs.

First, did the model output a tool call, or did it skip the tool and answer from its own pattern matching. Second, did the tool actually execute, or is the model narrating a tool call that never happened. Third, is the output objectively correct where correctness is checkable. Does the math reconcile. Does the file exist. Does the API return what the schema promised.

Without verification the harness is taking the model’s word for its own work, which is a terrible idea.

Error handling and recovery

The harness logic for dealing with failed tool calls, malformed outputs, missing data, rate limits, conflicting instructions, partial completions, and invalid intermediate results.

Real systems fail for lots of reasons. Tools time out. Schemas mismatch. APIs return junk. The model produces a tool call with the wrong arguments. A good harness does not stop when something fails. It classifies the failure, retries when retrying makes sense, escalates to a human when retrying does not, rolls back any unsafe changes, and preserves enough state for the work to pick back up.

This is one of the biggest differences between a demo agent and a production harness. Shit happens; your harness is the adult in the room that figures out where to go next when it does.

Subagents

A bounded child agent that the parent agent can spawn for a specific subtask, with its own scoped context and its own tools. The parent does not need to load every detail of the subtask into its own context window. It delegates, the subagent runs its own loop, and only the result comes back. Subagents are how harnesses handle work that is too large or too specialized to live inside a single context window.

Compaction

The technique for keeping long-running work fit inside the context window. This is necessary, and it’s fraught with unstated risk. As a session accumulates turns, the harness summarizes earlier exchanges, drops intermediate scratch work, and preserves the decisions and outputs that matter for what comes next.

Compaction is what lets an agent work on a problem for hours without exceeding the window. Done well, the model retains everything it needs. Done poorly, the model forgets the decision it made three steps ago and contradicts itself. This is done poorly right now more than it’s done well – because doing it poorly conserves compute. I’ll just leave that there.

Permissions and scoping

Which tools are available in a given session. Which files the agent can read or write. Which credentials it can use. Which systems it can touch.

Permissions live in the harness, not in the model, because the model cannot be trusted to enforce its own limits. Models don’t actually care about rules; they care about probabilities. A model with broad access will use that access whenever it decides the task calls for it. A harness with narrow scoping decides for the model.

Approval gates

In most useful systems, there are things that require human judgment. At these points in the workflow the harness pauses execution and requires a human to approve, reject, or modify the next action. Approval gates are used for actions that are externally visible, expensive, irreversible, risky, or judgment heavy.

Sending an email, modifying a source file, updating a model, deleting data, committing code, submitting a report, or taking an action in a system of record may all require approval. The interface presents the decision, but the harness decides when approval is required.

Approval gates are how useful systems stay useful without pretending that every task should be fully autonomous. Please remember this.

Observability

Models are black boxes, like charcoal black. Without observability the harness is an opaque transcript of input to output. With observability the harness becomes an inspectable execution trace of the loops between the initial input and the final output, which is the substrate for everything else worth doing, including verification at scale, debugging, audit, and evaluation.

Evaluation

The test system for the harness. Evaluation asks whether output meets expectations.

Some evaluations are deterministic, like whether the agent modified the right file or returned a value within tolerance. Others require human review or rubric-based scoring against a reference. Evaluation is how teams decide whether a new model, a new prompt, a new tool, a new retrieval strategy, or a new context approach is actually an improvement, or whether it just performed better on the example everyone happened to look at.

Output contracts

The required shape, format, schema, and destination of the agent’s work product. In a chat system the output may be words in a paragraph. In a production workflow it may be a JSON object, a database update, a Revit parameter set, a spreadsheet, a pull request, a ticket, an email draft, a marked-up drawing, or a signed decision record.

Output contracts define what the agent must produce and what the harness can validate before the work is accepted. They turn “give me the answer” into “produce this kind of artifact, with these fields, in this format, for this downstream use.” Without an output contract, the agent may produce something that sounds right to a human but cannot be consumed, checked, routed, stored, or acted on by the system that comes next.

Interface

The surface where the human meets the harness. The interface layer describes how and where the inputs and outputs from the harness will be gathered or presented in the broader environment.

The UI choice is whether the human interacts through a chat window, a custom console built for a specific workflow, or a desktop agent like Claude Desktop or Cowork that runs in its own contained environment. The integration choice is whether the harness lives in its own product or wraps itself around tools the human already uses.

An agent embedded in Outlook surfaces drafts where the user already reads email. An agent embedded in Slack answers questions where the team already asks them. An agent embedded in Autodesk Construction Cloud annotates models where the coordinator already does coordination.

The interface decision determines what the human sees, what they can intervene on, and where the boundary between agent execution and human review actually sits. Two harnesses with identical models, tools, context, state, retrieval, verification, and output contracts can produce very different work depending on this choice alone.

So, that’s a harness

There is a lot here. Most of this stuff was even more confusing 3 weeks ago than it is today – but things are coalescing, and that’s good. Right now, this is all called a harness; next year it might be called something else. I think that depends on how the IPOs go later this year…