alt Hacker NewsStrange how things evolve. When ChatGPT started it had about 2 years headstart over Google's best proprietary model, and more than 2 years ahead to open source models.
Now they have to be lucky to be 6 months ahead to an open model with at most half the parameter count, trained on 1%-2% the hardware US models are trained on.
Replies
rbtprograms • yesterday at 6:43 PM
it seems they believed that superior models would be the moat, but when deepseek essentially replicated o1 they switched to the ecosystem as the moat.
DeathArrow • today at 5:48 AM
>Now they have to be lucky to be 6 months ahead to an open model with at most half the parameter count, trained on 1%-2% the hardware US models are trained on.
Maybe there's a limit in training and throwing more hardware at it does very little improvement?
And more than that, the need for people/business to pay the premium for SOTA getting smaller and smaller.
I thought that OpenAI was doomed the moment that Zuckerberg showed he was serious about commoditizing LLM. Even if llama wasn't the GPT killer, it showed that there was no secret formula and that OpenAI had no moat.