> And over the next few years, everyone and their dog will have local GPU compute to train locally. People have home labs, the bar isn't that high, and eventually large text datasets will escape from Anthropic and other companies, allowing for comparable training.

You're vastly underestimating the scaling problem here.

Things that would need to be true for your statement to be valid:

   - Model intelligence doesn't increase with model size
   - Model intelligence doesn't increase with training set size
   - Novel architecture completely decouples model intelligence scaling from hardware scaling
   - Residential electricity is as cheap as industrial electricity
   - RAM and GPU supply outpace demand