alt Hacker NewsThe former (it's well known that non-English languages don't tokenize well), but it's nice to put a concrete number to the impact.
The former (it's well known that non-English languages don't tokenize well), but it's nice to put a concrete number to the impact.
ty. so basically a formalization so others can build upon this idea.