> Adding reasoning agents, agent swarms, and so on to enhance the results from what is essentially just predicting the next token still means this technology at its core only predicts the next token. It is not AGI and unless something fundamental changes, it never will be.

A lot of people have dismissed LLMs four years ago for this reason. "It cannot possibly do coding since it doesn't think, it just predicts the next token." On the other hand, Scott Alexander did say in 2019 that next token prediction is a brilliant idea and might cover a lot, including math proofs [1]. So I wouldn't say it is impossible for either 1) LLMs to be all the way to AGI or 2) LLMs to become creative enough to tell us what the AGI architecture is.

[1] https://slatestarcodex.com/2019/02/19/gpt-2-as-step-toward-g...