alt Hacker NewsThe craziest for me is companies that sticking stochastic agents into automated business processes and expecting stable/reliable outcomes. Businesses want deterministic processes in the vast majority of cases.
Replies
I'm struggling with the assertion that these models cannot provide reasonably deterministic guarantees.
I am using gpt to populate JSON objects conforming to a list of natural language constraints for purposes of generating fake customers. I am finding that gpt5+ never fucks up. Not even a little bit. I've ran this test hundreds of times with 20+ constraints and it's been perfect every time.
Stable information yields stable control flow. Humans are much more likely to forget one of the many constraints during testing. This happy mistake may incidentally cover an edge but it also means we lose coverage elsewhere.
I think whether or not the LLM should be allowed to directly author deterministic control flow (code) is mostly the same thing. If you have a lot of constraints you want to satisfy all at the same time, this can give you a hit very close to the ideal target very quickly. Not knowing exactly what you want is when the LLM takes you for a ride.
People really need to read Dijkstras Go to statement considered harmful letter [1]. If the obscurity of go to for static analysis of the code was too much, of course bringing in a literal ai black box is harmful for stable processes.
[1] https://homepages.cwi.nl/~storm/teaching/reader/Dijkstra68.p...
I can argue that by applying multiple stochastic processes, with a human in the loop, that you will (may) converge on something that is deterministic. You use tests/test vectors to prove this.
We're no different to AI. The code we write to solve a particular problem can (and probably does) change from day to day, depending on your "mood", what you had for breakfast, if you've been fighting with your significant other, other problems/human emotions.
Yeah because "works many times in a row" = "deterministic" to many people.
"Business processes" can also mean "building power point decks" and other things.
But your point stands: for critical business processes that need predictability, we indeed need determinism.
People are stochastic. You build reliable processes out of unreliable parts with feedback and self-correcting mechanisms. AI is not actually magically special in this regard. It has higher variance and we're still figuring out how to get all the tradeoffs right.