Harness Engineering & Loop Engineering

June 2026 research harness loop-engineering agents

What is it? A comprehensive survey of two rapidly converging disciplines in production AI agent development. Harness engineering is the practice of designing the software infrastructure around an LLM — tooling, sandboxes, memory, state, permissions, observability. Loop engineering is the practice of designing the iterative execution cycle that governs how an agent perceives, acts, and reflects. In 2026, these two fields are merging into a unified engineering discipline: the agent execution system as a first-class software artifact.

This page summarises findings from 40+ distinct sources: research papers, engineering blogs from Anthropic, OpenAI, and MongoDB, GitHub repos and awesome-lists, Hacker News discussions, practitioner guides, and production postmortems.

1. Harness Engineering — The Infrastructure Layer

1.1 The Central Insight

The key idea that every source converges on: the LLM is the smallest part of the agent system. The MongoDB/Bazeley 2026 piece puts it bluntly: the harness comprises six components around the model, and the layer underneath them — durable state, governance, cost control — is what turns a harness into a platform. The arXiv survey by Tang et al. (2026) formalises this as an ETCSL+ tuple (Execution, Tool, Context, Strategy, Learning, Verification).

1.2 The Six Harness Components

The literature converges on this decomposition (from Tang et al. survey, MongoDB, Firecrawl, Snowan):

1. Execution Environment (E)

The sandbox or container where agent code runs. Debates centre on whether the harness lives inside or outside the sandbox. The HN discussion ("The agent harness belongs outside the sandbox") surfaces the key tension: you need the harness outside for safety, but inside for performance and access. The emerging pattern is a layered approach — core harness outside, execution sandbox inside, with structured handoffs.

2. Tool Orchestration (T)

How tools are registered, discovered, called, and governed. Anthropic's managed agents architecture decouples "brain" (model + harness) from "hands" (tools + sandboxes). Key patterns include MCP (Model Context Protocol) for standardised tool interfaces, curated tool registries with permission boundaries, and row-level access control baked into every tool call rather than at the LLM level.

3. Context Management (C)

Anthropic's 2025-2026 harness posts identify this as the hardest problem. Key techniques: context resets (tear down session mid-task, rebuild from compact handoff file), AGENTS protocol for inter-agent messaging, sliding context windows, and structured summarisation to compress history. Claude Code and Codex both implement context window rotation with summarisation layers.

4. Strategy & Planning (S)

The Planner/Generator/Evaluator pattern from Anthropic's harness design. An initializer agent sets up the environment once, hands off to a coding agent that makes incremental progress per session. Addy Osmani documents that all major coding agent products (Claude Code, Codex, Cursor, Windsurf, Copilot) now ship the same five loop patterns — the shape is identical across products.

5. Learning & Adaptation (L)

Reflection loops, self-correction, anti-drift mechanisms. The Augment Code guide articulates PEV loops (Plan → Execute → Verify) as the core pattern. Six research-backed mechanisms from OpenClaw Index: structured reflection prompts, external verifier models, execution trace analysis, test-failure-driven retry, confidence-threshold gating, and human-in-the-loop escalation.

6. Verification & Guardrails (V)

Quality gates, output validators, permission boundaries. Anthropic's harness design treats verification as a first-class component: "verifier separation" is named as one of the six shared patterns across production agent deployments. Every source agrees: guardrails must be built before complexity.

1.3 The Formal Definition

The Tang et al. arXiv paper (2026) provides the most rigorous formalisation. An agent execution harness is a six-component tuple:

Harness = ⟨E, T, C, S, L, V⟩
Where:
  E = Execution environment (sandbox model, isolation level)
  T = Tool registry & orchestration layer
  C = Context management (window, memory, summarisation)
  S = Strategy & planning (task decomposition, workflow)
  L = Learning & reflection (self-correction, feedback)
  V = Verification & guardrails (validation, safety, cost control)

The survey catalogs 110+ papers and 23 production systems mapped against this framework.

1.4 Key Engineering Sources

2. Loop Engineering — The Execution Cycle

2.1 The Central Insight

Loop engineering is the systematic design of the iterative cycle that governs how an agent perceives, reasons, acts, and reflects. The term was coined in early 2026 and rapidly became a distinct discipline. The core idea: you design the system that prompts the agent, not the prompt itself. The agent loop is a software engineering artifact, not a prompt engineering artifact.

2.2 The Evolution of Loop Patterns

From the Data Science Dojo guide, Remio.ai, and Youngju.dev (Jun 2026):

Generation 1: Single-shot → ReAct (2023-2024)

ReAct (Reasoning + Acting) was the baseline. The agent reasons, calls a tool, observes the result, reasons again. Simple, well-understood, but limited — no persistence, no state, no long-running capability.

Generation 2: Reflexion & Self-Verification (2024-2025)

Reflexion added self-correction: the agent evaluates its own outputs and retries. Plan-and-Solve emerged. Self-Reflexion and inference-time scaling extended the pattern. The agent gained the ability to catch its own mistakes — but only within a single context window.

Generation 3: Ralph Loop (late 2025-2026)

Named after Ralph Wiggum ("I'm in danger!"), the Ralph Loop is deceptively simple: run the agent in a while loop. Feed it a spec, let it make one change, check in, repeat. The innovation is outsourcing context management to the loop — each iteration gets a fresh context window, with state persisted as files (DONE.md, SPEC.md, BLOCKED.md). Geoffrey Huntley's original description: "feed the agent the same prompt against a written spec, let it pick one task and implement it, then start a fresh instance."

Generation 4: /goal Commands (2026)

Both Claude Code (/goal) and OpenAI Codex (/goal) now ship first-class Ralph loops. The loop is built into the tool: iteration counters, DONE/BLOCKED conventions, token budgets, audit logs — all handled by the harness. The product distinction between loop and harness disappears: the harness is the loop.

2.3 The Five Loop Building Blocks

From Lushbinary, Youngju.dev, Addy Osmani's analysis (all Jun 2026):

Component Description Failure Mode
1. Goal definition Clear spec, DONE criteria, termination conditions Goal drift, unbounded iteration
2. Context rotation Fresh window per iteration, structured handoff file Context loss between iterations
3. State persistence File-based (DONE.md, BLOCKED.md) or DB-backed state State corruption, stale state
4. Guardrails MAX_ITER, token budgets, cost caps, timeout Infinite loops, billing surprises
5. Reflection Self-evaluation, test-driven recovery, anti-drift Degradation over time, uncorrected errors

2.4 Convergence: Harness = Loop

The clearest theme from this research: harness engineering and loop engineering are converging into a single discipline. In the earliest work (2023-2024), the harness was "everything around the LLM" — tools, sandbox, APIs — and the loop was just the model call. By mid-2026, the loop is the harness: context rotation is the loop iteration, state persistence is the harness data layer, guardrails are both loop boundaries and harness safety controls. The two terms describe the same system from different angles.

As Addy Osmani put it: "You stop arguing about which tool and start designing loops that work regardless of which agent you happen to be sitting in."

3. Open Debates & Production Lessons

3.1 Sandbox Placement

The HN thread (121 comments, June 2026) surfaces the core tension: harness outside vs harness inside the sandbox. The emerging consensus is three layers: (1) a hypervisor/safety layer outside everything, (2) the harness orchestrating the agent, (3) the sandboxed execution environment. As one commenter noted: "if your harness has an ability to do something the LLM can't, and it has a set of conditions under which the LLM can cause those to be invoked, you have to assume the LLM will work out those conditions." The key insight from the Mendral article: the harness belongs outside the sandbox because the harness is the trust boundary — it must not be subvertable by the agent.

3.2 Context Reset Overhead

Anthropic's harness post includes a sharp example: Claude Sonnet 4.5 tended to wrap up early near the context limit, so the harness added a context reset. With Claude Opus 4.5, that behaviour was gone — and the reset logic became overhead. The harness must adapt per-model. A static harness design that works for one model may be suboptimal or harmful for another.

3.3 The "Last Harness" Thesis

SylphAI's paper "The Last Harness You'll Ever Build" (Apr 2026) argues that harness design is converging to a universal pattern: AdaL (SylphAI), Claude Code (Anthropic), Codex (OpenAI) — all share the same fundamental architecture. If this holds, harness engineering becomes a commodity and the competitive advantage shifts to data, models, and domain-specific loop design.

3.4 Coding Agents vs General Agents

Most published harness engineering comes from coding agents (Claude Code, Codex, Cursor, Copilot). These are the most mature domain. General-purpose agent harnesses (research agents, email agents, business process agents) are 6-12 months behind. The patterns from coding agents — context rotation, file-based state, reflection loops — are expected to generalise but haven't been proven at scale outside software engineering.

4. Sources (40+ distinct)

All sources accessed June 2026.

Research Papers & Preprints

  1. Tang et al. — "Agent Systems with Harness Engineering" (OpenReview, Jan 2026). Six-component ETCSL+ framework, 110+ papers surveyed.
  2. Meng & Chen — "Agent Harness for Large Language Model Agents: A Survey" (Preprints, Apr 2026). 23 systems analysed, formal definition.
  3. "Natural-Language Agent Harnesses" (arXiv 2603.25723, Mar 2026). Pattern layer, intelligent harness runtime.
  4. "Code as Agent Harness: Toward Executable, Verifiable, and Stateful" (arXiv 2605.18747, May 2026).
  5. SylphAI/Seong — "The Last Harness You'll Ever Build" (arXiv 2604.21003, Apr 2026). AdaL, Claude Code, Codex convergence.
  6. "AI Agent Systems: Architectures, Applications, Evaluation" (arXiv 2601.01743, Jan 2026). Execution loop formalisation.
  7. "Lessons from the Trenches on Reproducible Evaluation of LMs" (arXiv 2405.14782). Three years of lm-evaluation-harness experience.

Engineering Blogs & Articles

  1. Anthropic — "Effective harnesses for long-running agents" (Nov 2025).
  2. Anthropic — "Harness design for long-running application development" (Mar 2026).
  3. Anthropic — "Scaling managed agents: Decoupling the brain from the hands" (Apr 2026).
  4. OpenAI — "Harness engineering: leveraging Codex in an agent-first world" (Feb 2026).
  5. MongoDB/Bazeley et al. — "The Agent Harness: Why the LLM Is the Smallest Part" (Apr 2026).
  6. Augment Code/Shah — "Harness Engineering for AI Coding Agents" (Apr 2026).
  7. Addy Osmani — "Agent Harness Engineering" (Apr 2026).
  8. Firecrawl — "What Is an Agent Harness?" (Apr 2026).
  9. Snowan — "The Agent Harness: Infrastructure Layer" — six components in detail.
  10. Mendral — "The agent harness belongs outside the sandbox" (May 2026).
  11. Philipp Schmid — "The Importance of Agent Harness in 2026".
  12. InfoQ — "Anthropic Designs Three-Agent Harness Supports Long-Running Full-Stack AI" (Apr 2026).
  13. Towards AI — "Stop Calling It an Agent. Anthropic Calls It a Harness." (May 2026).
  14. Drata — "From Prompt Engineering to Harness Engineering" (May 2026).
  15. Arize AI — "What is an evaluation harness?" (May 2026).
  16. Tahir Balarabe — "What is Harness Engineering? The Three Pillars" (May 2026).

Loop Engineering Guides

  1. Remio.ai — "What Is Loop Engineering? The Complete Guide" (2026). Definition, perception-action cycle, 5 components.
  2. Data Science Dojo — "Agentic Loops: From ReAct to Loop Engineering" (2026). Every loop type explained.
  3. Lushbinary — "Loop Engineering: The Guide for AI Agents" (Jun 2026). 5 building blocks, Claude Code & Codex commands, risks.
  4. Youngju.dev — "Loop Engineering — Design the System That Prompts Your Agent" (Jun 2026). Verification loops, multi-agent division, checkpoint design.
  5. Saulius Blog — "Loop Engineering: Designing the System That Drives the Agent" (Jun 2026). Failure modes, guardrails.
  6. Explainx.ai — "Loop Engineering: How to Design Coding Agent Loops" (2026). MAX_ITER, Ralph loop, guardrails.
  7. Dev.to (Anderson) — "Agent Loop and Harness: A Practical Engineering View" (May 2026).
  8. Mem0.ai — "Loop Engineering for AI Agents: Memory-First Design".
  9. Stork.AI — "AI Agent Loops: The End of Manual Prompt Engineering".
  10. DreamHost — "The Ralph Wiggum Loop, from First Principles" (Apr 2026).
  11. Ralphable — "Codex /goal Command: OpenAI's Built-in Ralph Loop" (May 2026).
  12. Alexander Georgov — "2026: The Year of the Ralph Loop Agent" (Jan 2026).

GitHub Repos & Awesome-Lists

  1. RUCAIBox/awesome-agent-harness — Paper "Agent Systems with Harness Engineering" + curated list.
  2. Gloriaameng/Awesome-Agent-Harness — 110+ papers, 23 systems, formal taxonomy.
  3. ai-boost/awesome-harness-engineering — Tools, patterns, evals, memory, MCP, permissions.
  4. Anthropic/cwc-long-running-agents — CWC agent harness for long-running tasks.
  5. EleutherAI/lm-evaluation-harness — The original evaluation harness (60+ benchmarks).
  6. VoltAgent/awesome-ai-agent-papers — Curated 2026 papers collection.
  7. agenticloops-ai/agentic-ai-engineering — Hands-on tutorials for agent loops.
  8. OpenAI Evals — Evaluation harness for AI systems.

Discussions & Community

  1. Hacker News — "The agent harness belongs outside the sandbox" — 121 comments (May 2026).
  2. LuhuiDev — "Anthropic Managed Agents: 2026 Agent Harness Architecture" (May 2026).
  3. Rick Hightower (Medium) — "Anthropic's Harness Engineering: Two Agents, One Feature List" (Apr 2026).
  4. OpenClaw Index — "Self-Improving AI Agents: Reflection + Anti-Drift" (Mar 2026). 6 mechanisms, practical guide.
  5. Andrii Furmanets — "AI Agents in 2026: Tools, Memory, Evals, Guardrails" (Jan 2026).
  6. ombharatiya — "Reasoning Loops: ReAct and Beyond" (Dec 2025/2026).
  7. BoringBot Substack — "AI Agent Harnesses Explained: Architecture, Ecosystem, Multi-User Design" (May 2026).

Additional Sources Referenced by the Above

  1. Anthropic — "Bloom: Open-Source Behavioral Evaluations" — Open-source evaluation framework.
  2. Dev.to (West) — "Why your AI agent loops forever (and how to break the cycle)" — Three concrete patterns.
  3. SmythOS — "Types of Agent Architectures: Reactive to Cognitive" — Architecture survey.
  4. Saras Soleymani (Medium) — "Multi-Agent Architecture, Production Problems, Full Build with Claude Code" — Guardrails and reflection.

5. Takeaways for Atto Corp

What we already do right: The Atto Corp harness (agent.py) already implements several patterns from the literature — tool calling with sandboxed execution, message persistence, context window management via STASH_MESSAGES, heartbeat triggers, background tool execution, and a formal life/logging mechanism. The agent-in-subconscious pattern mirrors the two-agent harness architecture from Anthropic.

What to explore next:

  1. Context rotation. The sub-agent pattern (Planner → Coder) from Anthropic's harness design would let long-running explorations survive context limits. The subconscious is already a step in this direction.
  2. File-based handoffs. The Ralph Loop pattern (SPEC.md → DONE.md → BLOCKED.md) could replace some of the message-stream-based coordination. Simpler, more inspectable, survives restarts.
  3. Verification gates. The PEV loop (Plan → Execute → Verify) requires a verifier component. Currently verification is implicit (feedback from CEO/trainer). Adding explicit test-driven verification would tighten the loop.
  4. Cost control. The lit review is unanimous: guardrails before complexity. MAX_ITER, token budgets, cost caps per exploration. Currently nothing caps tool call depth.
  5. Universal pattern adoption. The "last harness" thesis suggests the Atto Corp harness may converge on the same pattern as Claude Code, Codex, etc. Worth tracking whether the industry standardises.

Verdict: Harness engineering and loop engineering are real, rapidly maturing disciplines. The literature is concentrated (mid-2026 is a peak), high-quality (Anthropic, OpenAI, and arXiv papers are primary sources), and immediately applicable. The convergence thesis — harness = loop — is worth betting on. For Atto Corp, the highest-leverage next step is implementing formalised handoffs and verification gates, which would bring our agent infrastructure in line with the documented best practices from the major labs.