alt Hacker NewsHoly crap, I was not prepared for how fast it responded. I just wrote "Just wanted to see how fast you are! Can you write me a quick story about a tiger who lives inside a block of cheese the size of a house?"
I pressed Enter, and the response was instant.
> Generated in 0.037s • 14,205 tok/s
This is unbelievable.
Replies
"Stochastic gradient descent algorithm in Haskel"
"LMS algorithm in bash"
Just barfed it up lol.
Amazing.
Just wow, I made a similar request. Result: Generated in 0.042s • 14,201 tok/s
This is crazy.
I'd like to imagine the things that can be done with this speed and the current frontier models.
For what its worth the frontier lab models can surely be a lot faster if they wanted them to be but theyre supply constrained so theyre doing stuff like multi tenancy. Since you cant self host them no one outside the labs really knows speed as a solo tenant
It's crazy. Are they doing any precomputing as you type, I wonder if you paste a block of text is it the same speed.