You can do that by using sub agents and only giving specific MCP tools to the sub agents.

This pattern works well with specialized tool sets in general.