alt Hacker News> [others] Output tokens: ~5x more expensive than input tokens.
> [them] Output tokens: FREE (too cheap to meter).
I'm very confused by this.
Replies
quotemstr • last Tuesday at 8:19 PM
They're not doing autoregression, so all the outputs are computed in one big forward pass. Very cheap.
➕ show 1 reply
someguynamedq • yesterday at 10:45 AM
This is not confusing
CompleteSkeptic • last Tuesday at 8:26 PM
it's our output tokens that are free (under the system one / jev column)
The output tokens are just responses to your inputed questions and their probability. So relatively few output tokens. No unstructured text back in the response.