I see it as analogous to CPUs and specialized coprocessors. You don't want your general-purpose CPU bit-twiddling your USB data lines or decoding ethernet frames or blitting pixels. Specialized hardware does it so much more efficiently. And yet, powerful general-purpose CPUs remain common. Partly that's because there's a lot of complicated stuff being run that just isn't suited for specialized hardware; try making a C++ compiler coprocessor and see how far you get. And partly it's because that powerful general-purpose CPU is always present and can be instantly reconfigured to do anything. You'd rather use a dedicated cryptography coprocessor than implement your crypto on the CPU, but doing it on the CPU allows you to implement new crypto algorithms quickly. What's better: doing AES on a coprocessor on the CPU? Well, that depends on whether you have an AES coprocessor.

LLMs are the powerful general-purpose CPU of the ML world. Using an LLM as a classifier is woefully inefficient compared to using a dedicated classifier. On the other hand, that LLM exists right now and you can hook it up in ten seconds. You could ask: why use an LLM when a dedicated classifier would be so much more efficient? You could also ask: why build a dedicated classifier when a sufficiently capable LLM is available off the shelf? Neither one is the right answer every time.