logoalt Hacker News

serendip-mlyesterday at 6:18 AM0 repliesview on HN

The compression analogy is interesting. Another way of looking at it could be fine-tuning as "knowing what to leave out" - a 3B model for example tuned for a narrow task doesn't need the capacity that makes 70B good at many things.