logoalt Hacker News

hnfongyesterday at 6:55 PM0 repliesview on HN

Yes. Thank you for pointing this out.

I think there needs to be a linguist version of "what every programmer needs to know about (full?) text search"...

I'm not a linguist and I don't study languages, but I know enough to realize if a text search system is not designed for a particular language, it simply won't work. (As an example, to implement English search in a system for a hobby project, I had to import a US/UK spelling wordlist, and implement the Porter Stemming Algorithm. This is just for "one" language, and probably does not cover the other "English" dialects. Imagine doing a different workaround for every language in existence...)

RAG is actually a very language-agnostic way to work around those issues.