alt Hacker NewsIt's easy to trace that a bit sets a character in a file with absolute certainty; weights are in a highly dimensional space however, so even if ⍺v is good for one test it could easily regress for another.
I suppose the argument is that "editing weights" seems to imply a complex awareness of what each weight means, which is material for AGI hype. Really it's just doing very broad statistics, which is of course interesting but hardly unexpected.
Replies
Yes, the arguments in this thread seem to be around interpreting the word "editing". Is something consciously "editing" by tweaking the correct neurons/weight. Or is it just some linear algebra.
Though, even on re-read, it sure seems like Astra was making some decisions, otherwise what is this saying, that Astra knew how to download some linear algebra tools?
I look at it at a slightly different angle. AGI implies broad capabilities including self-improvement without using another AGI. Being able to perform a “surgical procedure” on weights is a clear step in that direction. Direct weight interpretation may work if small model weights are included as training tokens but i doubt it’s useful for self-editing. On the other hand, using simple tools to interpret and edit it’s own weights might work.