alt Hacker NewsI think LLMs will start being etched into custom chips. There’s already some companies experimenting with it. The benefits are enormous - you get better performance, lower memory usage and no need to load a model in ram. Companies would be able to sell local LLMs while keeping their models proprietary.
If that happens, hopefully the ram price comes back to earth.
I’d love to buy a tiny swappable coprocessor that slots in next to my cpu and can take this stuff on. Swap it out every 8 months as you can benefit from something better. Hell, sign me up for a subscription!