alt Hacker NewsFrom your comment, I noticed a sort of pattern that is often seen when discussing LLMs. That they are these amazing things that can run so fast they trip themselves in their attempts at achieving a task. So we resort to refining the models, creating guardrails, orchestrating harness, so as to alleviate the 'hallucination' problem.
In contrast to human intelligence, there is an underlying mechanism that propels intelligent behaviour. A person is no less intelligent just because they lose sight, sound or inner voice.
“trip themselves in their attempts at achieving a task”
I see this a lot in Claude Code. I assume it has to do with the training structure.
Example is “fallbacks”. Claude constantly sprinkles “fallbacks” in the code, even when I ask it not to. That is, write multiple candidate implementations into the same code with some kind of switch.
This is a problem because you only need one, and it would seem to have inflated the code for no reason. (The madness accelerates with code volume, so you must push back.) anyway, few people would do it this way.
But I thought, what could be the benefit?
If you’re being conditioned to pass evals one-shot with code that will be discarded and never read, it’s a great strategy. If you have more than one way to solve it, you can just put both. The behavior would easily be reinforced, if trained that way.
But in any case, again, a certain nature and certain conditioning.
I think we’ll learn to accept it as AGI but also that no intelligence is fully divorced from context, limits and conditioning.