alt Hacker NewsWhy do you need an API key to tokenize the text? Isn't it supposed to be a cheap step that everything else in the model relies on?
Replies
weird-eye-issue • today at 5:53 AM
To prevent abuse? It's a completely free endpoint so I don't understand your complaint.
simonw • today at 4:16 AM
I'd love it if that API (which I do not believe Anthropic charge anything for) worked without an API key.
I'd guess it's because they don't want people to reverse engineer it.
Note that they're the only provider which doesn't make their tokenizer available offline as a library (i.e. the only provider whose tokenizer is secret).