alt Hacker News A Fable 5 model running at 9,000 tokens/s on an ASIC rather than 150 tokens/s on electricity chugging Nvidia GPUs, or even giant SRAM Cerebras or Groq chips could be good enough to meet the majority of demand.
Replies
dymk • yesterday at 10:49 PM
I am really confused about the point you're trying to make. 150tok/s is slow, but so is 9000tok/s? Or they're both fast? Or 150tok/s should be enough?
➕ show 2 replies
foobit-dev • yesterday at 9:34 PM
> 640K ought to be enough for anybody.
I get this reference!
Agreed 100%. This guy thinks there's a limit on the demand for intelligence. You think that Fable 7 which can run a billion dollar corporation on its own has no consumer demand just because we have fable 5 at 9k tok/s? Who do you think will be the biggest customer of such a model? Fable 7, obviously.