Assuming that's token cost upstream, and given the multiplication factor in tokens processed per query, that seems like maybe a few thousand requests per second at most? It's impressive but for a 50 person startup team expending millions per month, that seems about on par.

Would it be as impressive if the context were an email provider accepting thousands of message per second, or even one accepting thousands of messages per second and submitting them upstream for spam detection? The token count might even be higher in that case, but rightly or wrongly I think it would get a yawn on HN.

It says more about how far the industry has come these days in terms of scale on the one hand, but also on the other hand the huge blowup in data and processing for nominally simple requests. Nonetheless I'm sure the team is exceptionally skilled and it's certainly a laudable accomplishment.

To put that in perspective, if you assume a token is 4 bytes, that’s about 164 MB/s of traffic, which sounds a bit less staggering.