alt Hacker News>Users began expecting something fundamentally different from software: not just tools that store and retrieve, but products that reason, generate, and respond.
Not really.
>Every inference call costs money.
Not really, either. If you buy your own GPU, rack it, and run an open model, there is no unit cost. This is just expensive hosting infra. You also pay unit costs for SaaS that your software uses (things like SMS etc).
> If you buy your own GPU, rack it, and run an open model, there is no unit cost.
No. There is economic opportunity cost (borrowing), energy cost, infra cost, depreciation / risk of failure with each unit of work, bandwidth, maintenance, and lots more. Small, but not zero, and often overlooked - especially the opportunity cost.