alt Hacker NewsIt is my understanding that just baking the model itself into silicon only gives moderate gains because memory bandwidth remains a bottleneck.
Replies
chorizo • today at 3:07 AM
The big benefit is ROM cells require fewer components than DRAM. So the chips would be tiny, dense, cheap and consume far less power.
➕ show 1 reply
Did you try using the the talaas chat? Something stupid like 18k tokens/second.
Think it's called Askjimmy or similar.