If llama 3 70b were available for $400 today, people would make it work. They would have 4 of them working side by side or 1 of them working with a GPU on another model.

To me, it's like imagining if Sonnet 3 was burned into an ASIC 8 years ago and then never changed. It would still be revolutionary, and today we would have an entire ecosystem of tools and services built around it, likely surpassing some of our current workflows.

The frontier is a different beast, but it would likely mean competing on price.

for many purposes they're good enough now. If I had an opus 4.8 class model on a box next to me that could produce tokens at rates like 5000/s, i don't know if i'd need a new one for a long time. I think we might be underestimating how powerful very very fast LLMs could be, since they could iterate on tons of small variations on tasks. paired with deterministic guardrails that gate "doneness", you could loop on tasks for a long time having the agent try different strategies until the goal was reached, in ways that are just impractical now (and very expensive)

I think it makes more sense if you think about an ASIC pipeline which is updated on an annual basis, like Apple's M or A-series chips.

Sure, an ASIC model will always be behind (12 months? 18 months?), and a hosted flagship will be significantly better. But as time marches on, would I use a flagship model from last year if it came on a PCI card and cost $1000? Without a shadow of a doubt.