logoalt Hacker News

WhitneyLandyesterday at 9:54 PM1 replyview on HN

If speed is a metric for you, tokens required to solve a problem affects that metric.

All else being equal passing triple the amount of tokens through a model to solve the same problem makes it slower.

Doesn’t mean this model is bad, and it has to be considered how cheap it is, but it’s a factor.


Replies

rat9988today at 1:57 AM

Still, seems useless to measure tokens when you can measure speed.