Yes, but doesn't the token change mean that?
You can train a tokenizer on old data just like you can train a model on old data.
alt Hacker News
You can train a tokenizer on old data just like you can train a model on old data.