alt Hacker NewsI’m just a layman but this feels like a big discovery on par with attention is all you need. For an LLM to have such incredible intuition of weights feels very emergent and unanticipated?
Replies
This does feel big, but the trillion-dollar question: is this trained by OpenAI and can other LLMs do it.
It’s no secret that these big labs see “recursive self improvement” as a primary goal. It seems natural that the models would be explicitly trained to excel at all sorts of ML related tasks as a result. Is this one of those tasks, or is this emergent?
This is just the model editing weights in the same way humans do when we jailbreak/ablate models. The method isn't novel, the interesting bit is that Astra will do it at all (I'd expect some safeguards in this area around jailbreaks)
Weights are just data like any other data, and LLMs are really good at dealing with data. I'm really surprised anyone is surprised by this.
Was it "intuited" or empirical?
How is it even possible though I can’t think of any reason why they would have the intuition.
I’m very much surprised but also not. I recently gave Astra a 3D model file and asked Astra to build a viewer for it, and it figured out the data structures in no time. What I mean to say is that AI is insanely good at things that we humans would never do since the work would be too monotonous.
I wouldn’t mind if AI progress would stop right here and now, but I fear it will continue, and increase inequality in society which is contrary to what the CEOs say (as usual).