alt Hacker NewsI do wonder if the labs have already started including weight updates in their training sets as a means of teaching RSI via RL and this is basically just discovering that. Regardless if this ability is emergent or achieved through finetuning though, it's pretty incredible.
I wish these companies were more transparent so every third comment in this thread wouldn’t have to guess whether such shocking behavior is emergent or stems from a part of training.
BTW making DL models that generate weights for other models has been explored before in “hypernetworks”, though these have been far more simplistic and constrained.