logoalt Hacker News

krackers • yesterday at 7:21 PM • 1 reply • view on HN

I guess the connection is that if you assume neural networks are in fact filled with meaningful features instead of superposition blackbox stuff, then what the LLM is doing here isn't quite magic but rather methodical interpretability. And if you can isolate the neuron that is e.g. responsible for steering the paddle then you can probably tweak it to steer more/less aggressively.

It is still very surprising though, considering it apparently displays an "intuitive" understanding of the weights, that make whatever Goodfire did look like primitive monkeys poking at it by comparison.


Replies

supern0va • yesterday at 8:11 PM

> It is still very surprising though, considering it apparently displays an "intuitive" understanding of the weights, that make whatever Goodfire did look like primitive monkeys poking at it by comparison.

FYI, Nick/Goodfire used the models extensively in their work, so it's very possible that there were some similarities in their work. I'm really curious to see his response to this.

➕ show 1 reply