alt Hacker NewsThe harness that connects to a chatbot, API or voice interaction is the place to route requests to different systems. If you remember the early days of ChatGPT it explicitly said it was routing image generation to Dall-E after embellishing your request itself first.
Determining which tool to use should be a lightweight operation but I’m not expert enough to understand exactly how much lighter than a full LLM call just to recognize it needs a different tool or model.
For sure, I just know it’s tempting given the power of large transformers to throw things at an existing model.
For instance, OCR is something that can be done locally with no access to a GPU but people (including me) still often use cloud hosted multi-modal large language models for it.