alt Hacker NewsSome teams I know started moving all linting and validation into LLMs and markdown files, the reasoning was that they asked agent to review changes that were later found to be buggy and agents flagged these bugs. And that’s great but my immediate worry was that: it finds it now, but will you check every new model for all these bugs? And what if you need to switch providers? Or your provider silently tweaks the model to improve their margins? The answer was “well also build deterministic linters for all these problems”, but you know that’s just wishful thinking - if you can get 99.9% done with just a little bit of fuzzy text and the next 0.1% is a lot of work it won’t happen. So you have model behaviour lock-in that will bite you suddenly when you least expect it. Or maybe LLMs will just keep getting better and cheaper? It’s all a casino right now.
Replies
Every LLM company must keep getting better and cheaper to compete. If you game out them running low on money, none of the outcomes actually result in you spending more or getting worse models, just switching to another one.
If you have 99.9% already - that's better than any pre-LLM deterministic linter.
> Some teams I know started moving all linting and validation into LLMs and markdown files
Sounds like those teams need better leadership. Deterministic tools should always be used along side LLMs to improve the results and reduce costs.