alt Hacker News> If the thing can solve complex math problems and at the same time be so dumb as to fall for "social engineering", then that means that it is not "smartness" or "reasoning" that is helping it to solve those problems. Just some form of advanced, but yet dumb, search algorithm.
I'm not just trying to be snarky, but I have no idea how to read this without taking the implication that humans are advanced, yet dumb, search algorithms.
A human being who states X (implying they know it to be true) will behave in a way that is consistent with X being true.
An LLM will happily say X and behaves in contradiction to X. Because it does not reason. Its behavior is not derived from things that it claim (or appears) to know.