> at best it seems like we’re heading back to the “server under your desk” era of IT again

Maybe in the very long term. If companies go local, the efficient model is to buy some big hardware to share among developers.

I run local models. Even with 128GB unified memory systems or a 5090 or RTX 6000, the generation speeds X model quality X context length is still far behind what I get from my SOTA model subscriptions. I also pay a lot more for the locally generated tokens in electricity and hardware costs. I'm also limited in parallel requests to the local box. The list goes on.

I really like running local models, but for any given point in time it's more efficient to have a big central box aggregating requests and churning through them. So maybe companies buy $300K servers and try to split it among 30 users instead of trying to buy 30 x $10K boxes.

More likely, they rent time on cloud servers by the month so they can adapt the hardware when new models come out with new requirements.

Then some day in the distant future when hardware is cheap and plentiful again, it might make sense for us to go back to individual boxes under the desk.