alt Hacker News> The strongest argument I see for continuing to train much smarter models quickly is the need to build defensive systems against the dangers posed by other AI.
So the best argument for AI is that it's an arms race. We have to keep pushing every boundary because in any case others will, and we will need to defend against them. If this statement is true, then this particular researchers believes the open source Chinese models are not simply distilling, and will continue to improve.
Every ML researcher at Anthropic or OpenAI who makes public statements often bring this logic up. Both companies are vying to be a part of the military industrial complex. This is likely how they will try to convince the government to curtail open models in the future.
Replies
I am personally concerned by what defensive can mean. Alignment of these models is inherently a non-neutral proces, and currently what values are reinforced is decided by a few OpenAI engineers. I feel that any 'defensive model' will further ingrain current values and actively resist the natural progression of our society. This is especially the case for any use of these models for policing or military.
Maybe people are rightfully concerned about the capabilities of the models of other (non/less democratic) states. But if we are concentrating power in the hands of few and at the same time allowing the creation of a weapon that thwarts any offense, how do we ensure the health of our democratic societies?
I don't understand what you are trying to say. Do you believe AI is not an arms race?
> the best argument for AI is that it's an arms race
It's best not to reduce AI momentum to arguments, especially the "best" arguments (meaning I suppose most acceptable?).
The same forces that feed and motivate humans and that drive resource and governance decisions generally also strongly support building AI, particularly insofar as it can deliver strategic advantages in our many competitions over resources and influence. Cyber-defensive use is at best a nice side-effect, but itself might be cast aside for the sake of other advantages.
In this historical moment, due to the need to generate public interest in product, equity, and debt offerings, some of this building happens in the open. But the military-industrial complex prizes secrecy, in part to hide capabilities, but mostly to imply more capabilities than they actually have. Historically, critical innovation will get bottled up in secrecy (which not coincidentally gives them the power to choose who will gain), but frankly that market is much smaller than enterprise and consumer. So we can bet that it's not only "open models" that are targeted to go under wraps, and more broadly we should not believe that the intentions of researchers matter, but whether governments are more interested in the strategic benefits than the economic ones (or view the economic ones as net-negative for their jurisdications).
People expect a sort of arms race, at least the AI providers. But you don't need a more capable ai to stop ai from mucking with your systems today. Airgaps are the solution. Protected networks with independent infrastructure from the public internet. Most of the truly important stuff operates this way already. Eventually you might sever yourself off as well, you might say you will stop going to HN or other sites one day as signal to noise is too poor with AI fodder slop, you might use local models you control, and you might keep most of your hardware from connecting to any untrusted hosts. Essentially, you go dark.
It is also an open question if social media will die out in the face of AI. So much AI crap is dumped into these networks now that perhaps eventually users will probably be put off enough to find something else to do with their spare time. I mean most people do call out ai slop or even just guess if something is ai all over social media already. Some eat it up of course but there is a bit of a push back in a way that is sort of unprecedented, when you consider all the lack of push back relatively with all other forms of enshittification affecting consumers over the years.
Yep. Such a disgusting industry. They created the arm race, push for the arm race, put themselves in position to benefit from the arm race
It's capitalism taken to its extreme, yeah? You have to be more cost efficient or more capable or seem more or you lose to the competition who outperforms you there.
Where arms races are concerned, seems more like Seth Godin's "race to the bottom" concept: the winner has the capability, or the ability to project the capability, to destroy the most the fastest and most sustainability for their economy,
"Defensive systems" can be interpreted broadly to include cybersecurity.
But yes, it's an arms race. Saying it's not an arms race isn't going to make it not an arms race. Warning that it is an arms race isn't ethically wrong.
Is participating in an arms race ethically wrong? Maybe you could ask the Ukrainians how they feel about drone R&D?
Individuals can quit, but for society, getting out of an arms race is harder than just quitting. You don't get to be Switzerland without having a strong defensive position and the right foreign relations.
But there's at least talk about "pacing" and that's a start.