logoalt Hacker News

nialv7today at 9:05 PM1 replyview on HN

how are you running the 2.4t model locally if you don't mind me asking


Replies

jnwatsontoday at 9:41 PM

Oh I didn't; only a distilled 27B model ran locally. This was just to figure out if distilling caused the problem.