A few years is hardly forever and the state of the world here indicates a lot of low hanging fruit still exists.

An LLM can certainly be cheaper than grep, because it’s an approximation, while a grep is deterministic and must examine every byte in what can be a relatively complex state machine for a regex based grep. There are other scales to consider like the scale of your local hardware vs the highly multitenant and high end hardware of the hyper scale inference providers.

There are already high volume models for coding inference where the reasoning time is crazy low and cheap per token where it can build reasonably simple software so blindingly fast it isn’t implausible the bottleneck is the latency in tools and networks. I find them hard to use at times because I don’t have time to think through the next turn by the time it’s done.

Regardless I wouldn’t be surprised to see a world where tokens are so cheap it’s not worth metering them but charging licensing feels with meter tiers at the far horizons to prevent abuse, charge outliers. Subscription models already set this stage well.

The other side to consider is bountiful capacity will also drive tokens to near zero price. The data center build out is barely underway and as it materializes, as hardware efficiencies improve, as techniques and model science and technology improves, harnesses, methodologies , etc improve, the economics flip from load shedding to trying to keep the data centers utilized. The economics lead to the world where tokens are not a unit of measurement for cost for anyone other than the inference providers to manage their utilization.