logoalt Hacker News

LZ_Khanyesterday at 7:02 PM7 repliesview on HN

If you care about improvement of models, you would support the US labs here.

It costs hundreds of millions of dollars to train a frontier model. It's not just "scraping the web."

Distillation allows labs to replicate these results at 1/100th of the cost. This creates a prisoner's dilenmma which incentivizes labs to withhold their models from the public.


Replies

ElevenLatheyesterday at 7:06 PM

How much did it cost to produce all the data on the internet and every book ever published? Surely even the most conservative calculations put it at multiple years of planetary GDP. The same argument can be made to say that letting the big labs get away with pirating it will disincentivize people to publish anything.

show 2 replies
contravariantyesterday at 7:10 PM

If 'we' really cared about the improvement of models all of them would be public.

Anything else just proves someone prefers making money to improving the models.

falcor84yesterday at 7:15 PM

> incentivizes labs to withhold their models from the public.

Does it really? How would they get revenue if they withhold their models? And doesn't economics generally say that if it's easier for your competitor to catch up, you have a higher incentive to maintain your lead?

wpmyesterday at 7:04 PM

> If you care about improvement of models, you would support the US labs here.

I guess I don't care then.

show 1 reply
falcor84yesterday at 7:13 PM

I think that the bigger conversation to be had here is about the environmental damage - if by using distillation we can really train new models at 1% of the cost in energy, it is ethically imperative that we do this.

bigyabaiyesterday at 7:12 PM

This reads a bit like over-moralizing to me. US labs will continue improving their models because they have to make money in a competitive market. Chinese distillations have arguably improved the status-quo, with Qwen and R1 forcing GPT-OSS to be released to the public. American businesses are competing, and American customers are getting better products because of the competitive pressure on them.

Your purported "prisoner's dilemma" hasn't happened yet to my knowledge, instead we seem to see the opposite. The high-speed development velocity has forced US labs to release more often with less nebulous results. Supporting either side will contribute to healthier competition in the long run.

YetAnotherNickyesterday at 7:09 PM

> incentivizes labs to withhold their models from the public.

This is the only way they make money.