The Harness Is the Thing

Reading this really makes me wish that I had a slightly better workflow. I'm really soley dependent of Fable to the point that I don't use other models, and I've already sort of hit a point where I'm running into usage limits every week. I am really living on borrowed time — when Anthropic finally collapses their 50% usage increase at the end of August, I'll definitely be forced to switch my workflow. When that happens, I have a hard time imagining that I'll be sticking with a single model on a single provider.

What I've learned in last week is that a harness is basically a while loop.

In each iteration you make an LLM call, perform some work (e.g. tool call), augment the prompt (append or compact etc.)- not necessarily in that other- and continue.

Until an end condition is satisfied. Then you break out.

i doubt it. someday harnesses get complex enough that you start asking models to interpret and modify them.

the pattern of layering of deterministic, probabilistic, deterministic, probabilistic. it's a strange pattern but it seems somehow natural.

This guy knows harnesses!! I'm curious to know what your thoughts are on dsh(deepseek harness), where everything is a plugin. I read an article where the author described it as a motherboar in the Mobo combo purchases we use to do to save money, with the processor(deepseek), being replaceable by other llms but shipped with it by default. Did you evaluate it as a possible replacement? If not, why not?? In my nondev brain, it looks revolutionary, the equivalent of the x86, to the SoTAs attempt at keeping us in a walled garden similar to apple consumers.

More agentic than this. The harness is the thing / Wherein I'll grant Claude access to Bing.

Mine is more granular. I start with a scoping brief before planning. But yeah, I'm all-in in this same thesis.

Heh. Interesting article.

> Single developer projects can build to the caliber and consistency of large development teams.

This has always been true. Good developers, like truly good devs, could run rings around a team of mediocre devs. It’s a multiplier, a team of 10 1x devs will get dominated by a single 10x dev no matter how much AI they use. Nothing has changed here, if anything it benefits the good developers.

> At the moment my rig is supported by two subscriptions (Cursor, Claude) that I can augment with Pi as needed.

This conflicts subscriptions with an actual harness, doesn’t bode well for the rest of the article…

> Recently I learned about prewalk, Can Bölük's technique that uses frontier for the planning phase and first task, then hands off once the pattern is set.

This pattern has been known for years and is not attributable to a single person.

> Exploration leads to a plan formalized into an explicit DAG (directed acyclic graph) task list. Then a worker takes over, focusing on implementing the DAG one node at a time. Once complete, I bring in the critic to simplify and question what was implemented. Often this phase will push back enough that the worker phase is revisited. But once satisfied, the critic gives way to a promoter, which is my reminder that a job is not complete until you've properly communicated it to others.

I’ve tried all these complicated workflows. In the end the best way to use LLMs is to give it some instructions, take a look at the code, and then ask it for changes. At the end, ask it (in a fresh session) to review the changes for bugs or incorrect assumptions and architecture. Rinse and repeat. Anything more complex is over engineering.

None of the rest of the article seems particularly interesting. Just more busywork.

> HUMANIST SOFTWARE DEVELOPER

Uh huh?

[dead]