That's not happening without significant funding. The cost of the primary training run for frontier models is in the hundreds of millions of dollars, and is increasing with every iteration. The Astra pre-train likely cost close to a billion dollars.
The Chinese open-source models (which are 1-2 generations behind the frontier) cost in the low tens of millions to train. This is largely because of a lack of compute. They achieve comparable performance to frontier models via distillation.
alt Hacker News
That's not happening without significant funding. The cost of the primary training run for frontier models is in the hundreds of millions of dollars, and is increasing with every iteration. The Astra pre-train likely cost close to a billion dollars.
The Chinese open-source models (which are 1-2 generations behind the frontier) cost in the low tens of millions to train. This is largely because of a lack of compute. They achieve comparable performance to frontier models via distillation.