alt Hacker NewsInteresting article to read but by defining memory as a prediction about future needs they are side stepping the real problem that these models cannot learn continuously i.e. update their weights. Opus 5.5s real memory , experience and knowledge is set in stone no amount of markdown tricks will change that.
This is a better research direction I feel
https://research.google/blog/the-sleep-framework-a-new-way-f...
Replies
I think the end state of any productive LLM includes both: - a real-time knowledge/context graph that keeps track of state - continuously updating weights
Long term I think it's probably more interesting to explore continuously updating weights. But (and this may be yet another opinion that falls victim to the bitter lesson) even if it's _theoretically_ possible to encode everything in weights alone, I'm not sure it's not the most optimal. Today it's too expensive & finnicky to get better on your own tasks without overfitting (ie [0])
Practically (like next few months) getting the right instructions & context would be a huge improvement in capabilities.
[0] https://thinkingmachines.ai/blog/on-policy-distillation/#tra...
Continuously updating weights will never be viable for a company like anthropic that's providing cloud inference.
So if you're hoping for that you should probably just look at the models that can be ran locally... And then you prolly realize that that's still several years into the future.