alt Hacker NewsEven simpler: Can GPT-2 anticipate and build Gwen/Deepseek? I think the answer is almost trivially "no", so I wonder what changed?
Replies
rowyourboat • today at 5:15 PM
That's not really a fair comparison, no? Modern LLMs are much more capable than GPT-2. We'ld need a modern LLM trained on exclusively old data, and that might be impossible
Lots of things changed, GPT-2 is small (1.5e9) and is also a base model, so it is only doing next-token/autocomplete rather than prompt-response like even the first ChatGPT-3.5 was doing.