alt Hacker NewsIf open models on local hardware were more cost effective and competitive, it would be obvious that this is such a superficial approach. (I mean, it still is obvious but what are ya gunna do?)
We would be doing the same general loop, but fine tuning the model overnight.
I still think the current LLM architecture(s) is a very useful local maximum, but ultimately a dead end for AI.
Time will tell. I find the transparency of explicit memory systems paired with the perfect-forgetfulness of LLMs a very pleasurable toolkit. I guess a black-box memory could work too, but at the very least I'd want to be able to rewind and branch. My memory systems are currently git controlled, so it's fairly straightforward.