alt Hacker NewsI wonder how the Chinese labs are training a 3 trillion parameter model on what has to be vastly smaller compute resources. If the U.S. compute advantage is persistent, it's hard to imagine that Chinese labs will be able to keep pace forever, as a matter of physics, but... so far they seem to be doing just fine.
Replies
kristofferR • today at 9:19 PM
The Chinese just saved the world economy by draining their absurdly enormous oil storage reserves nobody knew they had, wouldn't surprise me if they had lots of hidden compute too.
dopa42365 • today at 6:34 PM
It's not like same parameter count models are identical, so that doesn't appear to be an indicator for quality, or even compute requirements?
There seems to be more to producing a better model than brute forcing parameter count after all.
➕ show 1 reply
Or they just don't actually have any compute access restrictions of significance? Chinese companies can just go use those GPUs in neighboring countries that aren't export-restricted, like Malaysia. Like ByteDance openly did: https://www.tomshardware.com/pc-components/gpus/chinas-byted...
and Tencent is rumored to have done via Japan: https://wccftech.com/china-tencent-gains-access-to-nvidia-bl...
And that's not even considering just smuggling the GPUs in by eg buying them in Singapore.
AI-specific chips also seem to be on the easier side to design & create relative to high performance CPUs & GPUs, so there's no particular reason to expect Chinese domestic designs to continuously lag behind. They have access to the same fabs, after all