In a general sense, you see this crop up in SKILLS.md files and other places, as LLMs have to deal with broad contexts. People try and drill into some taxonomy in a naive fashion using plaintext as a directive, which is not particularly optimal.

I wonder if one could build a 'mixture of experts' at the model level that leveraged a variety of small models "within" a larger model...

because you have a different vector store for each categorization?

What if the question crosses categories?