alt Hacker NewsWe're going to need default hard budget caps on pretty much everything
Comments
It'd be nice if more than AI spend worked this way, autoscaling is almost a mixed blessing because unpredictable pricing can be worse than the cost savings...
Cloudflare also doesn't have any hard budget caps.
Uncapped usage is just corporate infinite scrolling in social media. Once you're hooked you just can't stop.
Counterpoint: if you can automate API calls on the client side, why can't you automate billing caps? If you want a machine that can run 24-7 and make money for you while you sleep (which let's face it is the motivation for a lot of AI takeup), isn't the onus on you to install cicuit-breakers?
Yes and no. I suspect many of the hard limits were set arbitrarily, and we'll see a relaxation of limits as people get frustrated with the limited use they get out of them. And some services will genuinely need to be re written to support higher rps or risk losing customers
These already exist, it's the most standard contract imaginable.
Budget cannot insulate you from uncontrollable price increases. Just make your business death more painful and prolonged.
The solution is to not give agents access to MCP servers.
The entire MCP ecosystem is ludicrous. You’re paying for inference for an agent to make the same decisions over and over again, and yet the actions they’re taking can be so easily written by those same agents into a bash script you can run again and again, deterministically and for free.
MCP is the problem. Having agents “use a product on your behalf” is the problem.
The pattern you’re looking for is that agents should write scripts that use products for us, and that doesn’t need a new protocol.
I'd support this provided we have the converse as well: if the customer doesn't pay their bill on time, the service gets shut down immediately. (Disclosure: I sell SaaS services to people who don't pay their bills on time).
This is nothing new. This has been defacto standard for companies using cloud, which has burst or semi-predictable spikes in usage.
But the idiocy is incredible, even allowing for this to be happen in a business is so infantile that the only hard cap that should be important is not to allow stupid people in the machine room.
What do people think of apps switching to a lower tier of model if your $$ threshold was exceeded?
We went from "it will be too cheap to meter" to "we need to meter this thing hard". Amazing.
Can we also put a hard cap on how much ram or gpus specific entities can get?
How does that work for storage or when you are in the middle of batch job that costs 1k and you get killed at 900?
That feature is called a subscription. Joking aside, it is needed on the API side
Why would your vendor want to make it harder for you to accidentally give them a million dollars?
One of the biggest benefits of not engaging with LLMs or any of this nonsense is you dont have to care about all these "self made" problems of the LLM-gliteratti.
Cheers!
You mean you want to curb business Gacha?
AWS, is a loot box... Tokens are just in game currency, and that sales person is just metrics that have identified your spending as making you a whale.
Your average CTO from the last decade turned a fixed cost into variable spending that looks like a mobile game.
This is another reason why cryptocurrency wins. Hard limits are baked in. There is no automatic charge like with a credit card or debit card or bank account. The payer has to initiate the payment.
the premise seems a bit faulty to me. why should we be giving next token predictors access to spend our money? like what great benefit do we get from this that we should allow them unfettered access, but with safeguards in the form of hard budget caps?
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[dead]
Wow, it only took AWS ... 24 years to develop this technology. Amazing progress, we're living in the future! /s
Could not agree more, this needs to be table stakes for any usage based service.
[dead]
This could have been written in 2006
If you own a resource that is desired and paid for by usage then that's your income and the open market determines the money thing (price)
What on earth is Simon whittering on about?
Ubicloud does not have hard budget caps, which I only realized this morning after moving all my CI over to them over the past few months. Fortunately I didn't learn the hard way.