> The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages.

Most generic use cases do benefit from a model that has been trained broadly. When you don’t know the specific use case ahead of time, you have to have world knowledge ready to go. Even when coding it’s helpful to have all that knowledge on tap so the model can understand product intent and use cases for the product you’re building.

> As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles

The real expensive part of fine tuning is gathering a good data set. That’s been the hard part of training anything for a long time. If you’re lucky enough to have a neatly organized and clean data set then you can attempt it, but you need to be in a position to run evals and measure quality.

I’ve done it, but there are so many use cases where the engineering, data labeling, and ongoing quality review hours cost so much that it would be cheaper to continue using a frontier lab model that just works from the start.