A lot of this is due the size of the corpus.

Grep falls apart for severely underspecified queries, which is the difficult part of web search. For any given query in web search there can be several millions of candidate results. You can get good results with FTS as well, but just finding phrase matches is inadequate, you need more ranking signals to find relevant results.

When Claude is looking for a function in your code base, it needs to sift through dozens of matches. This is not hard, and anything beyond grep is likely not worth the effort.