logoalt Hacker News

tristanj • today at 9:50 AM • 0 replies • view on HN

That's not happening without significant funding. The cost of the primary training run for frontier models is in the hundreds of millions of dollars, and is increasing with every iteration. The Astra pre-train likely cost close to a billion dollars.

The Chinese open-source models (which are 1-2 generations behind the frontier) cost in the low tens of millions to train. This is largely because of a lack of compute. They achieve comparable performance to frontier models via distillation.