logoalt Hacker News

dayboxyesterday at 7:44 PM3 repliesview on HN

> Never spend more than you budgeted

I assume that this is "$ spent on search + $ spent on LLM" < budget, but how do you handle the LLM spending more than you would expect on a request? Or is this handled by max_tokens and some form of pricing table? (and if so, how does caching play a role?)


Replies

waqasai123today at 5:08 AM

[flagged]

lajosdemeyesterday at 8:35 PM

[dead]