alt Hacker NewsWell, think that statement through a bit:
Grep reads through the entire file looking for patterns.
An LLM scans its neural net (in ways that I don't understand) which is kinda-sorta like having a huge index.
You can improve over Grep if you have an index; and the LLM has an index.
Thus, it's plausible that an LLM can be more efficient at reading its neural net (IE, index) than Grep reading the whole file.
But if the problem is literally grep (search this file you've never seen before), no index can pre-exist.
If you assume the file arrives ahead of time, can be indexed, and that this is worthwhile because we want to support multiple pattern matched retrievals, then sure it makes sense to consider indexed query schemes and upper/lower bounds. Each query could be faster as an inference if it doesn't have to re-scan the whole file.
But I don't think anybody, in good faith, can pretend that any LLM can digest a file faster than grep can. Particularly, if you admit the vector processing dedicated to doing the convolution kernel(s), you should also admit similar hardware could run a vectorized grep.