"I am not sure how many people will run AI models locally. It still seems like a niche application to me."

Clip me :). You are currently living through the final stages of unrestricted computing in the hands of the 'public'. Our regimes are going to pull up the drawbridge in the name of 'safety'. Download the open models asap and prepare for an airgapped computing environment. That will be your frontier in not extremely neutered AI in the near future.

I am so hoping I'm completely wrong on this btw.