alt Hacker NewsCurrent AI is at least a few orders of magnitude less efficient than it could be (as evident from biological spiking networks, e.g. human brain). At some point the labs put too much work and money into transformers and nearly abandoned fundamental research, in both ML and hardware. There are tons of low hanging fruits in efficiency but you'll have to redo everything from scratch so nobody bothers. Which is also pretty convenient and lets people like Dario Amodei speak about "natural concentrations of power".
The Chinese labs are picking up on the low hanging fruits on efficiency, and no, you do not need to abandon transformers, you just need to push them closer to the more computationally efficient architectures of the past. OpenAI and Google seem to be trying a few things too.
Anthropic clearly are not though, and to call their operations wasteful is an understatement.