The Self-Evolving Harness

The laptop config trap

In Part 1 I wrote about why everything an agent needs has to live on disk. In Part 2 we looked at how wrapping a model in a while loop turns natural language into a compiler.

Once a team starts working this way every day, you hit a new wall. Two engineers pull the exact same git commit, run the exact same model, ask for the same feature, and get completely different results.

Why? Because Developer A has spent three weeks tuning personal skills, MCP servers, and hooks inside ~/.claude on their laptop. Developer B just cloned the repo cold. And CI knows nothing about either of them.

Thinking about this over the past few weeks led me to three ideas that change how you roll out agents across a team:

1. 01

   The harness is versioned repository code state. Skills, connectors, hooks, verification scripts, and AGENTS.md belong inside the repo, reviewed and diffed in git like any other code.

2. 02

   An org-wide SDD loop is starter boilerplate (H0). A shared Spec-Driven Development plugin gives every repo the same workflow grammar on Day 1, because it gets injected into project state.

3. 03

   No "perfect" static harness exists. The best harness for a project (Hrepo) co-evolves with its codebase and base model as you run into real failures.

1. The harness is versioned repository code state

When people say "harness," they usually mean the CLI binary they launch in their terminal. But the binary is just the runner. The actual harness is the state that shapes what the model sees, what tools it can call, what rules stop it from doing something dumb, and how it checks its own work:

repo/
├── AGENTS.md                  # Routing map & build commands
├── .claude/
│   ├── skills/                # Domain playbooks loaded on demand
│   ├── hooks/                 # Pre/post tool guards
│   ├── mcp.json               # Project-scoped connectors
│   └── memory/                # Past postmortems & edge cases
└── scripts/
    └── verify-app             # One command to check the whole repo

If those files live in your home directory, your harness is invisible to your teammates and invisible to CI. The moment you check them into the repository, the harness becomes part of the software itself.

Change a database invariant in a PR => update the migration skill and the verification script in the same commit. Revert a branch => the harness reverts with it.

A recent paper from the Darwin Agent Team, HarnessX (arXiv:2606.14249), gives a clean mental model for those five boxes at the bottom of the diagram. They treat any harness as a set of five swappable building blocks: Context, Tools, Skills, Control, and Memory. When those five pieces sit in separate files in the repo instead of being tangled together, you can edit and test one piece at a time without breaking the rest.

Why tweaking CLAUDE.md alone can make your agent worse

Most people start "improving their harness" by adding another paragraph of rules to CLAUDE.md or AGENTS.md. Every time the agent messes up, the file gets twenty lines longer.

Another paper from earlier this year, Agentic Harness Engineering (AHE) (arXiv:2604.25850), tested what actually happens when you do that. They put every part of a coding harness into separate files in the workspace (system prompt, tools, middleware hooks, and memory) and let an evolution loop improve them over 10 rounds on Terminal-Bench 2.

The full evolved harness jumped from 69.7% to 77.0% pass@1, beating OpenAI's human-designed Codex-CLI harness (71.9%) with the exact same base model. Then the authors ran an ablation where they took the starting seed harness and swapped in just one evolved file at a time:

Look at that bottom row. Swapping in the evolved Tools, Memory, or Middleware hook on its own helped right away. Swapping in the 79-line evolved System Prompt alone made the agent 2.3 percentage points worse than the bare-bones seed.

Once you think about it, the reason is obvious. Those 79 lines of prompt rules told the agent to run contract checks that depended on the evolved shell tool and the finish-hook. Telling an agent "always verify X before finishing" in AGENTS.md without giving it the script or hook to actually run X just burns turns and confuses the loop.

Rules in a prompt only work when the tools and hooks in the repo make them executable.

2. An org-wide SDD loop is starter boilerplate (H₀)

This brings up a practical tension I run into a lot when building internal workflows for a whole engineering org. If every repository needs its own harness files, how do you get teams working the same way without everyone reinventing the wheel?

The answer is to treat your org-wide Spec-Driven Development (SDD) workflow as starter boilerplate (H0) that gets injected into a repository.

Across the org, we want every engineer and agent speaking the same high-level loop:

• Brainstorm & Spec => turn rough intent into a clear SPEC.md on disk before writing code
• Decompose => break the spec into ordered slices in TICKETS.md with a concrete check per ticket
• Dispatch => fan out subagents with fresh context windows (Swarm to search, Arena to weigh tradeoffs, Workers to build)
• Interrogate & Verify => run verify-app and an independent reviewer before a human ever looks at the PR

Think of H0 like running create-next-app or cloning a golden service template. On Day 1, installing the org SDD plugin drops that shared folder structure and command grammar into the project. After Day 1, the repo takes over.

A payments service, a Spark pipeline, and a React frontend should all share the same SDD rhythm from H0. Once real work starts, though, they need completely different tools and guardrails. The payments repo needs a double-entry ledger check; the frontend repo needs a headless browser screenshot check.

3. No perfect harness exists: co-evolving H_repo

It is tempting to think that once you tune a repo's harness for a week or two, you are done. You aren't, because two things underneath you never sit still:

• Your codebase movesenvironment driftNew services, new migrations, new architectural conventions, and new failure modes show up every sprint.
• Your models movebackbone driftDifferent model families have completely different habits. One model over-reads files until it runs out of context; another edits too fast without running tests.
• Your harness adaptsH_repo co-evolutionEvery time the agent trips on a ticket or an incident slips through, you patch the harness so that class of mistake can't happen twice.

In Self-Harness: Harnesses That Improve Themselves (arXiv:2606.09498), researchers at Shanghai AI Lab turned this exact habit into a three-step loop. You run the agent on real tasks, mine the traces for recurring failure patterns, propose a small diff to the harness, and only keep the diff if it passes a strict non-regression check.

This is also what Dan Shipper calls compounding engineering. In normal software work, fixing a bug leaves your app one bug cleaner, while your agent is still just as likely to make the same mistake tomorrow. When you treat the harness as evolving code state, every annoying PR review or bug produces two things: the code fix, and a small commit to Hrepo so the agent catches it on its own next time.

What happens when you only evolve the harness

A wave of papers over the past year asked a simple question: If you freeze the base model's weights completely and only evolve the harness code from execution feedback, how much better does the agent get?

My favorite detail in the Self-Harness paper is what the evolution loop ended up keeping across different environments. Every benchmark grew a different kind of harness:

• On Terminal-Bench-2.0 => it grew process-cleanup guards and file-artifact checks
• On SWE-bench Verified => it grew reproducer-first test gates and patch verification steps
• On AppWorld (multi-app APIs) => it grew pagination helpers and state-inspection routines, taking GLM-5 from 44.4% to 85.0%

Same starting boilerplate H0. Three completely different evolved harnesses, shaped by what actually broke in each environment.

Two limits to keep in mind

In the Hierarchical Self-Improvement (HSI) paper (arXiv:2608.08466), Zhou proves two bounds on harness evolution that map directly to daily engineering:

1. B1

   You can't evolve faster than your verifier (Feedback-Fidelity Bound). If a repo has weak tests, no typechecker, and no clear check command, neither you nor an agent can tell if a harness change helped or broke something else. A fast, honest verify-app script is what makes evolution work.

2. B2

   Scaffolding unlocks the model's ceiling, then weights take over (Backbone Capability Bound). Evolving Hrepo squeezes out huge latent capability that a generic setup leaves on the table. When a task is still beyond the base model's reasoning reach, you need a stronger model. And when you upgrade the model, you get to delete old workaround rules that the new weights no longer need.

Try it this week

Pick one repository your team works on every day:

• Check what is hiding in your personal ~/.claude folder and move the project-specific skills, hooks, and MCP configs into the repo so CI and your teammates get them too.
• Start new repos with our shared SDD loop (H0), then treat every recurring agent mistake in PR review as a prompt to patch Hrepo in git.
• Whenever you add a rule to AGENTS.md, pair it with the script or hook in verify-app that checks it automatically.

Sources & references

• Jiahang Lin et al. (Apr 2026), Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses (arXiv:2604.25850).
• Tingyang Chen et al., Darwin Agent Team (Jun/Jul 2026), HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry (arXiv:2606.14249).
• Hangfan Zhang et al., Shanghai AI Lab (Jun/Aug 2026), Self-Harness: Harnesses That Improve Themselves (arXiv:2606.09498).
• Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange & Jeff Clune (ICLR 2026), Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents (arXiv:2505.22954) & Hu, Lu & Clune (ICLR 2025), Automated Design of Agentic Systems (ADAS) (arXiv:2408.08435).
• Zhou (Aug 2026), Hierarchical Self-Improvement: Task-Specific Evolvable Agent Harnesses (arXiv:2608.08466).
• Zhang et al. (Aug 2026), DarwinX (arXiv:2608.07545).