alt Hacker NewsBut does this mean that a 99.99% reliable LLM would turn us back into the mode of us building it again? I would not say so.
I think for tasks that are about decisions, having the LLM make decisions is what makes it feel like the LLM did something for me.
Consider mowing the lawn. Imagine I had a lawnmower robot that does the mowing all on its own. Despite perfect accuracy, I didn't mow the lawn; it did. If I sit on that lawnmower the whole time and start driving it instead of letting it go on its own, then I mowed the lawn. Even if I stand there and control it with a joystick, I still mowed the lawn. Ownership comes from the decisions about where to mow.
If the outcome is equivalent regardless which choices the LLM made, then these LLM decisions weren’t material for the outcome, and it would be fine for me. If the decisions don’t materially change the result, then it means that I can still reason about the output that will be produced. If, on the other hand, the decisions materially change the output, then this also breaks the ability for the prompter to predict what will be produced.
This is analogous to the compiler example: It generally doesn’t matter which exact variation of machine instructions the compiler chooses to generate from the source code. It doesn’t significantly impact the ability to predict the resulting program’s behavior in detail based on the source code.
This predictability relation is what allows the creator to reason about the creation, to reason about which exact change to apply to the input to effect a precise modification in the output.
In that sense, the robot lawnmower is like a compiler, if it yields the state of lawn that I specified. I largely don’t care if it first moved left or right, the same as I largely don’t care if a compiler translates a multiplication by a power of two with a shift instruction or a multiplication instruction.
However, we use LLMs precisely for their heuristics, for their property to actually make material choices. Therefore LLMs inherently conflict with the ability to reason about which exact input will yield which desired output.