alt Hacker News> Because this is very much a part of what ML engineers do.
Do you realize most people aren't ML engineers?
I even built tiny neural nets myself and never played with the weights directly post-training. The original post was eye-opening for that reason. I knew it was possible to visualize weights, but as far as I understood, it was not really something I could edit with any success. Now, it seems possible without a phd just by asking another LLM. This is fun!
I think you misunderstand. Yet the model is doing something impressive still by writing code to fix model weights on problems. But that's a whole different ballgame than just reading weights and knowing what to change. That would mean that they have an internal representation of how the model weights map to information or a specific task, which if true, throws all the interpretability research out of the window