logoalt Hacker News

boredumbtoday at 6:05 PM6 repliesview on HN

Not specifically Anthropic but why are we allowing billing to take place in tokens that are nebulous and fully controlled by the operators who have no aligned incentives?

If I have a user input and then sanitize and inject that into a prompt to do something, I have no idea how much that is going to cost at all and no real way to measure this properly. A parallel example is digital ocean or aws, i can go and measure/limit my compute/fs/memory/startup times/etc and while it can be impossible to get down to the last flop of money allocated - i can run things on a real budget with real constraints, opposed to an LLM where I have to .. prerun a sanitized user prompt through a tokenizer and then ask an LLM to guess what it may do and give token consumption estimates and then act on those in any sane manner for the user?

Perhaps i'm missing something to do realistic and static rails on things but I don't see a serious way at scale to use the token billing model handling things requiring a users free text input short of having to go pander to VC money to throw money at it until someone else figures it out.

*to clarify my rambling... We should be billed and given controls based on resource usage itself and not an opaque token concept on top of not being able to spin any knobs that control it's resource usage.


Replies

bob1029today at 6:12 PM

> the operators who have no aligned incentives

The model providers are quite aligned with concerns like customer retention. These arguments only work if there is no competition. We exist in a marketplace of black boxes. There's not just "the one" you must suffer. You have options. You can build your own too.

show 2 replies
demibabstoday at 6:07 PM

How else would they bill tho? Their operating cost is per token.

show 4 replies
eh_why_nottoday at 9:05 PM

One guess is that their "primary" target audience/market is the large corporations that get their employees unlimited tokens, and not the individual developer who may worry about spending and token accounting.

show 1 reply
daishi55today at 6:19 PM

For OAI and Anthropic at least you can set a spend limit per response. Also tokens are well-defined.

show 1 reply
m4rtinktoday at 10:25 PM

Seems like yet another instance of printing your own money & getting rich by screwing people forced to use them.

Goas back to factory towns, gift cards, game money or MtG.