logoalt Hacker News

numpad0today at 11:44 AM0 repliesview on HN

Tokenizer efficiency varying by languages, by as much as up to 15x, is very well known and established

  https://www.google.com/search?q=tokenizer+efficiency+by+language