I'm not sold; that's not to say I can't be swayed, but I've spent a lot of time building RAG solutions as a "one-size-fits-all" against codebases and I am not convinced embedding models or any of the myriad BERT implementations work well enough that they can beat out grep.

Most embedding/BERT models are far too small to truly grasp the nettles when an AI ask questions to them; and if you condition the AI to only search for really basic things, then... you're basically back to grep + regex.

You need to prune all manner of useless detritus that might gum up your matches like test files, documentation, and sidecars that semantically overlap or you're just going to match your tests. That's fine if you're building a document store for a business, but if you want a "one-and-done" solution for your codebase it's rarely good enough.

And if you do use a very large embedding model (which means it'll require running in the cloud as most company computers won't run it well), then... well, you're better off just using a language model as a subagent to scout ahead for you and elide the embedding model to begin with.