logoalt Hacker News

kridsdale1 • today at 3:35 AM • 1 reply • view on HN

Well I think the first big publication on this front was Anthropic’s Golden Gate Bridge neuron.


Replies

frotaur • today at 3:55 AM

This was not a neuron. This was from a trained sparse auto encoder.

In a few words, they found a way to map residual stream activations (so, a vector) into a few interpretable sparse concepts. They also had the converse maps, that translate concepts into activations.

Golden Gate claude was obtained by artificially increasing the activations in the direction of the 'Golden Gate' concept. It was not neuron surgery.

➕ show 1 reply