Building a Personal AI Coding Harness: 14 Traps Between "It Runs" and "It Works"
This picks up the last layer from Experience Sediment 2.0 — L3, harness orchestration. In that post I said a harness is "the code that encodes your team's workflow as scheduling logic," and then didn't unpack it. Unpacking it now.
For the past year I've run a multi-coding-agent harness on my team's actual work. It started as a 200-line Python script running tasks serially; today it schedules 4-6 agents in parallel with automated review and cross-model routing. The number of traps in between was well past what I expected.
Here they are as 14 items in 4 groups. Every group is a threshold a harness has to cross to get from "demos fine" to "actually usable."
1. Economics: survive first, then talk about quality (3 traps)
1. Multi-model routing is mandatory, not optional
On a finite budget, sending every task to Opus is suicide. The baseline split:
- Haiku / small models: intent classification, JSON parsing, categorization, tag extraction
- Sonnet / workhorse: main code generation, single-step refactors
- Opus / big model: cross-file review, complex spec decomposition, hard debugging
One rule of thumb: 80% of your tokens should go to 20% of the nodes. Three months in, we moved 70% of calls to Haiku / Sonnet and kept Opus for the reviewer and the critical planner only. The monthly bill went from $1,800 to $400 — and output quality went up, because we didn't cheap out where it mattered.
2. There's a hard token ceiling per task
Too big a task and the agent freelances; the odds of drifting spike. Too small and coordination overhead exceeds execution cost.
The sweet spot I've measured is around 3,000 input tokens per task. Past that, have the planner split it first, keeping every subtask under the threshold. This one item basically determines your output stability — decompose right and the output holds; let a task get large and no amount of downstream review or retry saves it.
3. Three tiers of hard budget caps
Per task, per session, per day. Hard cap on each.
I put this last in the economics group because you won't believe it matters until it bites you once — and then one night an agent enters an infinite retry loop, burns the month's budget by morning, and it becomes priority one.
My current settings: $0.5 per task, $5 per session, $30 per day. Over the line is a hard kill. Losing a few low-priority tasks is a lot cheaper than losing the budget.
2. Robustness: don't crash, don't cause an incident (4 traps)
4. Context management and handoff
Long sessions always rot. After about 30 minutes the output quality falls off a cliff — the context is packed with the agent's own earlier garbage and attention is scattered.
The fix isn't a bigger context window (that's treating the symptom), it's auto compaction plus structured handoff:
- At the end of each stage, the planner compresses state into a structured brief
- The next agent gets the brief plus its current task, and never sees the raw history
- Briefs are schema'd (JSON), not natural language
Skip this and the harness starts hallucinating after half a day. Do it and multi-day tasks become theoretically possible.
5. Retry needs three tiers
Don't throw an agent error straight back at a human:
- Same-model retry: network blips, transient rate limits — most failures die right here
- Different-model retry: the current model is stuck on a prompt injection or an overloaded context; a different model usually walks straight through it
- Escalate to a human: two consecutive failures with the same error signal — stop and wait
Most "the agent is broken" reports are actually an overstuffed context or too high a temperature, and swapping models fixes it. A harness with no three-tier chain dumps a pile of auto-recoverable failures on a human and grinds down whoever is championing the thing.
6. Sandboxing and permission tiers
An agent can run commands but not rm -rf /; can edit files but not push to main; can read the DB but not drop a table.
Assigning execution permissions by task type is a hard requirement of harness design, not a nice-to-have. Any agent that gets root by default will eventually cause an incident. My tiers:
- Read files / run tests: on by default
- Write files / run builds: only inside whitelisted directories
- Hit the database / call external APIs: sandbox or mock environment
- Touch a git remote / deploy to production: human confirmation required
7. Task idempotency
An agent retry must not produce side effects.
Otherwise: an email agent fails and retries three times, and the user gets three identical emails. A payment agent times out at the network layer and retries, and the account is charged twice.
Implementation: every task carries an idempotency key, and every side-effecting call dedupes on it. This sounds like payments-system trivia, but in the agent era every harness needs it — because retry is the default path, not the exception path.
3. Quality: getting output you can actually ship (3 traps)
8. Complex tasks must self-review
Simple tasks optimize for speed and can skip review. Complex ones — cross-file, cross-service, new modules — need a reviewer agent underneath them.
The reviewer isn't just "ask again." It carries the checklist from your L2 sediment: edge conditions, N+1, error handling, naming, test coverage. It emits a structured verdict (pass / fix / reject) and the harness decides from that whether the work flows back to a coder.
This item sets the floor on your output. With no reviewer, a harness is permanently stuck at "everything gets audited by a human." With a reviewer and a non-empty checklist, 80% of PRs go out unaudited.
9. Agents talk to each other in structured output, never prose
planner → coder → reviewer → merger. Every hop is a JSON schema and the next agent parses fields directly.
Letting agents hand off in natural language is the laziest early mistake. It looks "flexible," but every hop carries a 5-10% parse failure rate, and stack four of them and end-to-end success collapses below 60%.
The other benefit of schemas: output becomes persistable, diffable, replayable. When you want to know why the reviewer rejected that task last week, you pull one JSON record instead of replaying the whole session.
10. Human-in-the-loop nodes are designed up front, not bolted on later
Which nodes block for human confirmation, and which push asynchronously to an inbox? Decide this during harness design. You cannot wait until something breaks and then patch it in.
My hard-block nodes:
- Production deploys / traffic shifts
- Database schema changes
- Deleting data / deleting users
- Cross-service breaking changes
Soft-push nodes (async to an inbox, agent keeps moving):
- Naming and UX suggestions on new features
- Review of non-core modules
- Documentation quality scores
Get this right and total human intervention time drops by 60%+ — you spend attention only where it genuinely can't go wrong.
4. Extensibility: a harness evolves, it isn't a one-shot artifact (4 traps)
11. Follow the standard protocol, don't roll your own
The ~/.agents convention has been adopted by everything except Claude Code at this point, and cc will likely follow. For agent role definitions, tool invocation, context passing — use the standard wherever a standard exists.
Our harness currently runs dual-track: one canonical agents/ directory and a .claude/ mirror. We'll merge them once cc catches up. A bit of over-engineering short term buys you the freedom to swap agents later without rewriting the harness.
12. The task graph is a DAG, not a linear pipeline
A linear pipeline is the beginner's version. It runs, and it wastes half your wall-clock time. Real tasks have plenty of parallelizable segments:
- Frontend and backend changes can run in parallel, but both wait on the spec agent
- The test agent and the review agent can run at the same time (review output doesn't depend on test results)
- Coders on independent modules are fully parallel
Encode the work as a DAG with explicit dependency edges and let the scheduler parallelize. A 4-hour serial task typically compresses to 1-1.5 hours.
13. Observability decides whether a harness can improve at all
Every agent task writes a structured log: duration, tokens, model, output quality score, human interventions, retry count, final verdict.
Without that data you don't know which agent or which rule is dragging you down, and the harness stays at "it runs" forever instead of reaching "it runs well."
My current floor: one JSON log per task plus a weekly dashboard with 7 metrics (success rate, mean duration, mean tokens, retry rate, human-intervention rate, cost per task, reviewer reject rate). When a metric looks wrong, drill into the sample tasks behind it.
14. cursorrules / specs / agent definitions all live in git
Every piece of harness config — agent role definitions, cursorrules, spec templates, reviewer checklists — belongs in git.
Because one day you'll tweak a rule, and three weeks later some class of task will suddenly fall apart, and you need to be able to git bisect your way back. An unversioned harness is an unmaintained harness, and it turns into an unreadable mess eventually.
Bonus: rules can go through PR review, so several people can work on them without overwriting each other, and rollback is cheap enough that people make bold changes.
Priority among the 14
If you're just starting, you don't need all of them. By dependency order, 4 of these are foundations — get them wrong and everything after gets redone:
- #1 multi-model routing — without it the bill talks you out of the harness before month two
- #9 structured output — without it, every agent you add pushes the parse failure rate higher
- #14 versioning — without it, the faster you iterate the faster it breaks
- #13 observability — without it you think you're optimizing and you're guessing
The other 10 you can patch after you hit them. These 4 are already too late by the time you hit them.
Closing
Harness engineering is fundamentally freezing a team's workflow into code. And the code isn't just execution logic — the important part is the structural constraints: scheduling, fallback, observation, budget. It's ten years of your judgment as a team lead turned into a program that runs.
The moat here isn't clever prompts and it isn't model selection — anyone can copy both. What can't be copied is the judgment inside your harness about why a human blocks here, why those two can run in parallel, why this retry stops after three attempts. Every one of those came from a real incident.
If you're a TL: building a harness is the single best use of your 2026. Not to save headcount — saving headcount is a side effect. The point is moving ten years of your judgment out of your head into something that executes 24/7. The day you leave the team, change roles, or step away, the harness is still running. That's what "preserving experience" actually means.
These 14 aren't the finish line, they're the entry ticket. A harness that clears all 14 has only just become usable. Above that there's agent self-evolution, cross-project reuse, multi-team federation — that's another post.