The AI being packed into everything are lightweight models that are not compute intensive at all to run.

It's the behemoth models (and their agents) that are actually constrained. There is not enough compute to meet the demand for people offloading their day job sized models (and the agent teams they spawn).