alt Hacker NewsThere are good theoretical reasons to think superposition exists in neural networks, particularly in the residual stream rather than specifically in MLP neuron activations. It wasn't just a single example that got people thinking about superposition.
Yeah that makes sense too. Again, downsides of having zero fundamental frameworks. I'm not equipped to really take it apart.