alt Hacker NewsI'm still impressed by the "Weight-Space Programming" idea. Perhaps its because I didn't know that LLMs could understand or successfully modify weights at all previous to reading it. However, I think the examples provided in the original piece were eye-opening. Yes, humans can build tooling to modify weights, but the author here agrees that it's tough. The examples where Astra was interacting with weights are still novel, and Astra seemed to handle them with surprising ease.
Here's the work that shows how rule-based logic can be compiled into transformer weights:
* Thinking Like Transformers” by Weiss, Goldberg & Yahav (ICML 2021).: https://arxiv.org/abs/2106.06981
* Tracr: Compiled Transformers as a Laboratory for Interpretability https://arxiv.org/abs/2301.05062 (code at https://github.com/deepmind/tracr)
So it's not really an "LLMs can modify weights" thing. Rather it's "LLMs can write programs that can write or modify weights". Indeed LLMs can write many programs, it's impressive.