logoalt Hacker News

amelius • today at 11:26 AM • 2 replies • view on HN

Can't LLMs just "degrade" into decision APIs whenever possible/necessary?

Why does this need to be a separate thing?


Replies

remus • today at 11:40 AM

They generally use smaller, more specialised models which are substantially cheaper for similar performance e.g. https://strandsagents.com/blog/amazon-saves-millions-replaci...

mattmaroon • today at 11:37 AM

You could instruct them to behave like one but they’ll still eat a lot of tokens, probably more, even though you’ll save a small amount on the output. It’s a separate thing because of significant token savings.