It feels more like a slot machine than a tool to me. A tool has a clear, repeatable cause & effect, and I can use that to figure out how precisely to get from one condition to another iteratively. With a slot machine, sometimes I get lucky. If I pull the crank a lot, maybe I’ll get the outcome I want. Maybe not.
The first few times I used a rotary sander were similar to a slot machine too -- scary times for a bit there! Fortunately, and I think it applies to LLMs too, you can get more reliable results with experience.
>With a slot machine, sometimes I get lucky. If I pull the crank a lot, maybe I’ll get the outcome I want. Maybe not.
Look at how effective this is though. You just have a computer pull the slot machine in a loop until you get what you want. And what you were unable to do as quickly, if at all. This describes the process for the math, crypto, security, etc. breakthroughs. They just try a ton of shit until they gain traction and meet the goal.
This also describes what a mathematician does, it's just over the course of years not hours.
That’s why you run parallel subagents on the same task and pick a winner. It’s free spins on a subscription.
Very few tools are deterministic in the way you describe. A hand saw without a guide is very non-deterministic. A drill not in a drill press is not deterministic. A hand shovel is chaotic. You never know what you are getting from using hand tools.