alt Hacker NewsThis is great in principle but in practice I don't have enough resources to maintain a stable normalization layer across 3+ inference providers, each with wildly different and evolving API/product roadmaps.
I could make it work if I didn't care about access to latest reasoning model capabilities, but then my customers would no longer be interested in any of this.
I tried the DIY provider agnostic harness thing and it performs like shit compared to what OAI, Anthropic and Google's engineers have created. I don't have a trillion dollar AI budget. I feel like these comments are sometimes written with the assumption that the reader does.
Replies
I assume I'm missing some context because you sound like you have a lot more experience here, but what kinds of capabilities are there that you can't do something like `interface LlmProvider { }` with a few basic methods for attachments and chat and then just implement from there? I'd even say you could not keep the ones you're not using (say, GPT is your go-to for a year) and update the other ones when you want to use them and swap out the `LlmProvider` implementation with a DI'd different implementation at your application's entrypoint.
> I tried the DIY provider agnostic harness thing and it performs like shit
Just ask AI to make it perform not like shit. /s
It is fine to move up a level, and use models with their providers' harnesses, as long as you stay away from extras like their own memory and infrastructure. There is not much investment required, and you will end up with a more robust solution. A simple setup that works:
- Agents live in their own directories, for example ~/Agents/[project]/[instance]/
- In a tmux session run each agent in its own window and directory. Pi, OpenCode, Claude Code, Codex, a combination, whatever works best at any given time.
- Tell them that they are working with other agents, and that knowledge is shared and stored in an agreed format. They are very good at using OKF[1], for example. In my actual setup I use a harvester agent whose only job is to ensure the quality of the knowledge.
- They can communicate with each other simply by sending messages to each other's tmux windows with tmux send-keys. If the team grows, or you work with other people, you can use a general messaging system (I wrote and maintain https://github.com/awebai/aweb but I am sure there are others).
1: https://okf.md