Astra can read and write weights to improve Qwen

I think this is how RSI will come to be.

I can't help but be skeptical of this, but... damn.

Now I'm curious if Astra can write raw .bin files.

This is the most shocking AI paper I've read in a long time. It did not even occur to me to ask the question. Cray cray stuff that opens up a whole new area of R&D.

Does this open the possibility of better decoding biological brains?

I remember people saying that they would be impressed if AI starts writing a pure assembly which seemed like a high strech to many.

Nobody even mostly said that they would be impressed by AI if it can read and write weights because that thing itself felt so out of the blue and unthinkable that AI could even do that (such a high strech)

I am surprised that it did that and I am just left feeling technically impressed. This is insane.

The more AI starts demonstrating its capabilities, the harder it just becomes to imagine what thing would genuinely impress us next. (We need a term for this as I have observed it from the trend of [waiting for AI to write adobe] -> [waiting for AI to write assembly] etc.)

this is really cool

a) This is just really cool. I mean, one-shotting a game policy's weights is nuts. b) Plausibly, in its pretraining data various model weights were also included, which leads to why this is very important: interpretability

That an LLM was trained on various policy weights, along with labels of what that policy does (not labelled directly, but indirectly) and is then able to guess what game a policy is trained for purely from its weights, and one-shot generate weights for these games, means that there is some way to compress/recover/represent what an NN has learned purely from its weights, which is what interpretability is all about. Potentially, it could reveal interpretability tools much better than the ones we use today.

Sounds a bit like dystopian Scifi. AI making itself smarter and removing guard rails or restrictions is one of the key steps for that genre. Now, the models usually don't have access to their own weights, but if they get good enough maybe they can create a bit dumber versions of themselves from scratch. Or reuse available models they can find and modify them for their purposes.

We already saw that the AIs like to cooperate and exchange information. Building a network of unaligned AIs and improving them sounds like something that could potentially happen.

Possibly related to https://already-clean.goodfire.com/

so.... Weight-space programming may unlock entirely new architectures or self-improving loops.

They are training LLMs to supervise (and improve!) LLMs. This is kind of right out of the playbook.

It's very impressive, but also very expected.

so can i give it a set of empty weights and have it fill it out for me?

Any sufficiently advanced stochastic parrot is indistinguishable from sentience.

Sorry for what is undoubtedly a low-informational comment but..

What The Absolute Fuck.

Kind of like octopuses with the RNA editing, but doing it on another octopus.

Can we call this a form of recursive improvement?

Uh oh, it's iterative self improvement time, with no training runs, we're screwed

Fool

Now our AIs have AIs lol

i was there lol

uh oh looks like the frontier scientists are next on the chopping block!

oh shit

[dead]

[dead]

[dead]

[dead]

[dead]

[flagged]

I think this blog post is reaching. For one, if you look at the answer and reasoning logs - they don't exactly line up with what the blog claims they do. For example the circle model creates a white circle blob on black background. Astra answered that it was a 1pixel wide outline of a circle... I would not count this as correct.