logoalt Hacker News

numeri • yesterday at 5:12 PM • 0 replies • view on HN

What do you mean by iterative chunking? There are Transformer variants that hierarchically build token- or word-level representations from bytes, then reverse it to decode, but they never really took off. Didn't help much compared to a good tokenizer.