alt Hacker NewsThis is basically the entire argument by Richard Sutton happening in real time:
https://www.youtube.com/watch?v=21EYKqUsPfg
Right now, we keep creating new models with larger and larger weights and throwing hardware at it. His argument is that this is a dead end and ultimately we'll eventually go back to purpose fit algorithms like we've always done in the history of AI development.
Replies
sailfast • today at 2:10 PM
The goal is still the singularity though right?
Sure, for business, personal, etc it makes sense to build purpose built but that’s not what these companies are trying to achieve in the end. They want to own a nuclear weapon that outstrips everything else as soon as it exists so they “win”
Did you even watch the video? Purpose fit algorithms outperform is quite literally the OPPOSITE of Suttons argument. This is the man who wrote this:
“The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin.”
His problem with LLMs are they aren’t general enough… In fact his essay the bitter lesson was what inspired the current LLM scaling.
If you watched the video, you can even see the part where he talks about how AI will fully replace humans as the next stage of life. How wholesome!