logoalt Hacker News

mdp2021yesterday at 1:09 PM2 repliesview on HN

> people vastly underestimate full text search

It is not psychological, it is fully justified: substring search cannot find synonyms, periphrases and mistaken neighbours.


Replies

andy99yesterday at 3:15 PM

Maybe I’m interpreting this differently but to me modern LLM+full text search means “agentic” - LLM gets to pick the search terms and iterate on them. The underlying LLM does know synonyms etc, better and more flexibly than an embedding model, and gets explainable feedback from failed searches.

show 1 reply
locknitpickeryesterday at 1:44 PM

> It is not psychological, it is fully justified: substring search cannot find synonyms, periphrases and mistaken neighbours.

It is, if people don't even stop to think if they need synonyms, periphrases, or mistaken neighbours.

As the blog post points out, more often than not you don't, particularly if your primary usecase is to search for technical keywords or codenames.

show 3 replies