alt Hacker NewsI discovered yesterday that the “amazing thing that comes out of OpenAI” is Sol, due to its token efficiency.
Dollar for tokens, Sol and Fable are the same price.
However, Sol uses (literally: in testing) around 10-100x less output tokens compared to Fable for the same task.
We run our frontier models nearly 24/7, so switching to Sol will save us around $500 per day.
And, due to less guardrails, Sol also performed better, and we lost less tokens due to guardrails shutting down sessions (I feel like it’s illegal to take $50 of someone’s token money and then shut down a session with guardrails before they get an answer, and yet Anthropic do it to us constantly… either take our money and commit, or trigger the guardrails immediately)
Replies
Evidently, Claude's tokenizer vocabulary size is ~15k[1]. On one hand, it's quite mind blowing. On the other hand, Anthropic models' token (in)efficiency makes a lot of sense in that light.
[1]: https://xcancel.com/magikarp_tokens/status/20878591737488549...
Sol is way cheaper than Fable by the token.
Wait isn't Fable like 2x more expensive if we compare under 272k tokens
We’ve literally saved tens of millions of dollars already (no exaggeration! already 8 digits) by switching to Luna for many workloads at my company.
The amount of workloads we can shift with an advisor model pattern continues to grow.
It’s seriously amazing.