We are coming an external tool as they 15xd prices on us. We have spent a lot of tokens and about a month, and are incredibly close to matching the external tool.

LLMs have made stuff like this much more possible.

Granted, we have an absurd amount of tech debt, but given our deadline that's an acceptable risk.

Note that the reason this works is because we can compare to the external tool, which is a really really good scenario for the current models.

In more greenfield areas they are less useful. Ultimately though, you can't outsource understanding, and that feels like the limiting factor now.