alt Hacker NewsCached tokens count towards the limit as well. For example, if your context window is 50,000 tokens, it takes 9 requests to reach that limit without generating a single token.
Replies
perching_aix • yesterday at 10:49 PM
then it's basically useless lol, wtf, this has to be a defect
Cached tokens counting toward the limit is ridiculous.