logoalt Hacker News

int_19htoday at 9:03 AM0 repliesview on HN

I don't think you can ever make a case for it on cost savings in general. Inference is very obviously the kind of problem where things are cheaper at scale, and this is still true for smaller models.

The only reason to run locally is privacy.