alt Hacker News> People are realizing that what they need isn't more general intelligence, it's more specialization. A small but well tuned coding model...
It’s not quite as simple as that. Several studies have shown the opposite: models trained on more diverse knowledge tend to cross-pollinate across domains. So a more generalized model can actually perform better than a specialized one.
That’s why you’re not seeing tons of tiny models (one for Python, one for Pascal, one for Rust, etc).
Replies
The western labs are very AGI pilled, and their public models are distilled down from larger research-only models that are uneconomical to serve directly. They could (and probably will) start distilling models for more niche use cases eventually, but we're not there yet.
Problem is conflict of interest: the studies are mostly from the providers of the biggest models, or someone who received free tokens to do the research.
> more diverse knowledge tend to cross-pollinate across domains
Yeah, the cross domain transfer learning from RL is overstated by a lot.
This is definitely the position of the big ai companies.
But it doesn't match my experience. Qwen3.8 27b is clearly smarter at coding than MANY bigger models. gpt-oss-120b for example, is almost 4x the size, and performs way worse at coding tasks.
It's clear to me that you can build small models that work well at specific tasks.
Python vs Rust is probably too fine grained a way to build a model. Coding in general seems like a better target.
There will always be a place for large generalist models, no doubt. But I think that place is much smaller than the big ai companies are counting on.