alt Hacker NewsCerebras provides high-speed inference at high cost. It's never going to be the cheapest and thus it will probably remain niche.
Cerebras provides high-speed inference at high cost. It's never going to be the cheapest and thus it will probably remain niche.
But that's supply and demand, not technology. Right now a lot more people want their inference than they can supply. as supply catches up in the next 5-10 years, the underlying tech at scale is probably cheaper than GPUs per token produced.