logoalt Hacker News

ameliustoday at 4:55 PM0 repliesview on HN

Yes, OK. But all the tutorials start with explaining how a tokenizer works. This is not necessary. And in fact makes the message of why a tokenizer is necessary not come across as well.