logoalt Hacker News

clickety_clackyesterday at 10:11 PM1 replyview on HN

You can train a tokenizer on old data just like you can train a model on old data.


Replies

wongarsutoday at 12:25 AM

But you can't use an old model with a new tokenizer. Changing the tokenizer implies you trained the model from scratch

show 1 reply