Depends what you man by "more specialised". You wont train very good language understanding without alot of data. It probably uses the core tranformer stack from an LLM.

Classic classifiers are regularly just tuned general models; Training a CCN on ImageNet and tune it for cats and dogs gives better results than just training it on cats and dogs.

There is likley a small network used to tranform model output vector to probabilities, but that wouldn't be massive. Retraining that small network for specific task may beat jev; but that's bairly considered training by modern standards.