On my previous team at work, the data science team was using Open it's API for tagging of some natural language. The DS team spent weeks working on the prompt plus the outline of the pipeline before I was involved. It took maybe eight to 12 hours to run everything, and it was prone to API failures.

When I asked why we don't build some NLP tagging which would be cheaper and run in a fraction of time, the lead told me, "What if we need to use more unstructured data? GPT can handle that without any extra work."

The team has yet to make any other changes to the pipeline, but in the two-ish years since then, they've spent a ton of money and wasted countless hours on delayed processing because of it. Some day, maybe, they'll learn that traditional ML techniques are superior in virtually every way to just throwing data at an LLM to see what sticks.