logoalt Hacker News

koito17today at 3:29 AM0 repliesview on HN

How does this site tokenize text? Split on ASCII whitespace?

Inputting Japanese sentences of any length flags the whole sentence as "Dramatic Fragment: A standalone paragraph with ≤4 words".