Why Agents Fail Even With a Smart Model
Many teams start an AI agent project with the wrong question: which model is the smartest? Once the best model is wired in, the agent still stalls mid-task, declares victory on work that is not finished, or calls tools with nonsense arguments. The problem is rarely the model's reasoning. It is the software layer around the model that turns reasoning into real action. That layer is the harness.
In practice, most production agent failures come from the harness rather than the model. That is why harness engineering emerged in early 2026 as a discipline in its own right, sitting alongside prompt engineering and context engineering.
Definition: The Harness Is the Infrastructure Around the Model
An agent harness — also called agent scaffolding — is the software infrastructure surrounding a large language model that lets it operate as an agent. It manages tool use, memory, state persistence, execution environments and feedback loops, as opposed to the model's internal reasoning.
Why does that matter? An LLM is stateless, unaided, and produces only text. Without a harness, a model can answer questions but cannot reliably run code, call APIs, read files, remember prior work, or complete multi-step workflows. A harness also lets record-keeping be offloaded into a structured software environment instead of forcing the model to re-read an ever-growing transcript inside its context window.
For a single prompt-and-response exchange, a harness is unnecessary. It becomes decisive as soon as tasks turn multi-step, tool-oriented, or long-running.
The Agent = Model + Harness Formula
The UK's AI Security Institute already described an AI agent as the model plus the scaffolding back in 2023. The relationship is now compressed into one line:
agent = model + harness
The easiest analogy: the model is the brain, the harness is the body and workspace around it, and the agent is the complete worker that can both think and act.
| Component | What it does | Analogy |
|---|---|---|
| Model | Reasons, predicts, generates output | The brain |
| Harness | Executes actions, manages memory, runs tools, enforces rules | The body and workspace |
| Agent | The full working system combining both | A worker who thinks and acts |
At the core of most agents sits a repeating cycle: reason → act → observe → repeat. The model reads context and picks the next action; the harness carries it out via a tool, sandbox, API call or write to storage; the harness captures the result and feeds it back as new context; the loop continues until the task is done. This is the ReAct loop, introduced in the ReAct paper in 2022.
Eight Building Blocks of a Production Harness
Most serious harnesses are assembled from the same components, each one patching a specific limitation of the raw model.
| Component | Purpose |
|---|---|
| System prompt | Standing instructions: who the agent is, its goal, its rules |
| Tools and tool execution | Callable functions; the harness actually runs them |
| Sandbox | Isolated workspace to run code without touching real systems |
| Filesystem and durable storage | Where code, notes and intermediate work persist across sessions |
| Memory and context management | Compaction, summarisation, retrieval of history across sessions |
| Feedback loops and self-verification | Running tests or inspecting results before the agent moves on |
| Guardrails and human-in-the-loop | Permissions, policies and approval checkpoints for risky actions |
| Observability and logging | Logs, traces and audit trails for debugging and compliance |
Why Reliability Comes From the Harness, Not the Model
As raw model capability converges, the harness increasingly determines performance. On public benchmarks, the exact same model can place significantly higher or lower depending entirely on how the harness is built. For workflow-heavy tasks, a strong harness around a mid-tier model routinely outperforms a weak harness around a stronger one.
The impact is measurable. When Databricks paired GPT-5.5 with the OfficeQA Pro Agent Harness, built for complex multi-part enterprise document tasks, the score rose to 52.63 percent from 36.10 percent with GPT-5.4 — cutting errors nearly in half. The model improved, but the harness is what turned that improvement into reliable production performance.
The most common production failure modes are harness failures too: context rot as conversation history grows, tool overload from offering too many tools at once, brittle tool wiring that fails silently, latency from long chains of tool calls, irrelevant retrieval that produces confident wrong answers, weak verification that lets agents stop too early, and missing guardrails around irreversible actions.
One feature sets a harness apart from ordinary software scaffolding: the component being wrapped is non-deterministic. A harness must therefore be designed to recover gracefully when the model fabricates an action or reports a task as finished when it is not.
Inner Harness vs Outer Harness
Birgitta Böckeler of Thoughtworks draws a distinction worth internalising before deciding what to build yourself.
The inner harness ships from the model's builder: an official agent SDK, or a coding tool such as Cursor or Codex. You choose it rather than build it.
The outer harness is what you assemble on top: instruction files in your repository, Model Context Protocol (MCP) servers connecting the agent to internal systems, and custom skills for specific workflows. This is where most engineering effort actually lives, and where competitive advantage accumulates.
The split is also useful for debugging. If the agent misuses a built-in tool, that is an inner-harness issue usually fixed by changing versions or configuration. If the agent does not know your project's code conventions, that is purely an outer-harness failure.
Guides vs Sensors: Steering Before, Verifying After
Böckeler further separates two distinct roles inside a harness:
Guides steer the agent before it acts.
Sensors observe the result after the action and let the agent self-correct.
Each can be computational (deterministic checks such as linters or tests) or inferential (semantic checks such as an LLM as judge).
| Computational | Inferential | |
|---|---|---|
| Guides (before acting) | Project templates, tool schemas, type definitions | Instruction files, tool descriptions, worked examples |
| Sensors (after acting) | Linters, unit tests, type checkers, HTTP smoke tests | LLM as judge, semantic review of a diff |
A common beginner mistake is to pile on guides only — writing thousand-line instruction files — with no sensors at all. The agent still reports success confidently. Combining both beats scaling either one alone.
Lessons From a Long-Running Agent Harness
Anthropic documented a concrete example worth copying. The core challenge with long-running agents is that they work in discrete sessions, and each new session begins with no memory of what came before — like a software project staffed by engineers working in shifts where every new arrival knows nothing about the previous shift. Context compaction alone turned out not to be enough.
Two failure patterns appeared. First, the agent tried to do too much at once, ran out of context mid-implementation, and left a feature half-built and undocumented. Second, a later session would look around, see progress had been made, and declare the job done.
The fix came in two parts. An initializer agent sets up the environment on the very first run: an init.sh script to launch the dev server, a claude-progress.txt log, an initial git commit, and a structured feature list in JSON — over 200 features in their example, all marked passes: false at the start. JSON was chosen deliberately because the model is less likely to overwrite or edit it inappropriately than a Markdown file.
A coding agent then runs every subsequent session: read pwd, read the git log, the progress file and the feature list; pick the single highest-priority unfinished feature; implement it; verify it end-to-end through browser automation such as the Puppeteer MCP server; then close with a descriptive git commit and a progress update. Agents are explicitly told that removing or editing tests is unacceptable, since it can hide broken functionality.
Notice that nearly every fix above is environmental rather than a prompt tweak. That is the essence of harness engineering.
What We Build at the Harness Layer for Clients
At katili.dev, when a client asks for an internal agent to handle website operations or a business workflow, the model is not the part we spend the most time on. The work concentrates in the outer harness:
Per-repository instruction files. Naming conventions, Laravel folder structure, database migration rules and off-limits directories are written out explicitly as guides.
Internal MCP servers. The agent gets controlled access to staging logs, hosting service status and read-only database queries — never raw credentials.
Custom skills for repeat workflows. Multilingual blog import, WordPress-to-Laravel migration checklists, domain and SSL setup procedures.
Layered sensors. Every code change passes a linter and static analysis, then unit tests, then an HTTP smoke test against staging after deploy. The agent may not call a task done until those sensors are green.
Guardrails and human-in-the-loop. Production database migrations, DNS changes and file deletions require human approval.
Progress files and commit conventions. The same initializer-plus-coding-agent pattern, so the next session never has to guess.
Observability. Every tool call is logged so behaviour can be audited and debugged when results drift.
The practical payoff: when a client wants to swap models — for cost, latency or regional availability — every layer above is reused as-is. That is the real return on treating the harness as an asset rather than as leftover code around a prompt.
Conclusion
Harness engineering is a broader layer than prompt engineering, which optimises a single interaction, and broader than context engineering, which governs what the model sees at a given moment. The harness designs the whole operational environment and contains both of them as parts.
Models will keep improving, and some harness responsibilities will likely move into the model itself. But execution environments, tool orchestration, guardrails, observability and feedback loops still decide whether a model can operate reliably inside real systems. Strong harnesses make average models useful; weak harnesses waste the best models available.
If your team is starting to build AI agents, measure maturity not by which model you picked but by three questions: what are your guides, what are your sensors, and what happens when the agent gets it wrong?
