You say this, but the reaction I get from colleagues is increasingly "why bother training new models when you can just ask an LLM?"

Doesn't matter the domain or the task, the answer from some researchers is "LLM" always.