logoalt Hacker News

mips_avatartoday at 5:23 PM0 repliesview on HN

A single 3090 will train qwen 0.8B just fine. While it’s not a very capable model any training technique you would want to master can be used to make real progress. And all the skills you need to learn how to do this can be learned watching Andrej Karpathy’s zero to hero series (shame he quit educational content and went to anthropic)