If I set a regular expression as watcher on a filesytem to notify me if any file changes and I write that in go and assuming regular expression isn't buggy nor its implementation - and then I write rules in a file (as regex) then there's snowball in hell of a chance that it would misnotify or miscategorize anything.

Are LLMs that super reliable in their output already with all the guardrails around?

Don't think so. Hence it is snake oil just like dozens of harnesses.

It might behave differently than specified and a human is required to validate every output carefully or else.

This is such a shame, finding where stuff is in a large codebase is my number 1 use for LLM. I hate it that it relies on grep so much, I can do grep better and faster myself.