logoalt Hacker News

dcrazy • today at 4:51 PM • 2 replies • view on HN

The only reason nuclear nonproliferation works is because the inputs are relatively rare and traceable. (Yes U238 is abundant but you can’t mine enough of it or buy the centrifuges to turn it into U235 without getting noticed.)

I have all the tools to develop my own ML training pipeline on my machine today. In order to choke off the inputs to “LLM proliferation,” we’d need to confiscate every computer, destroy all copies of the Internet, and throw out all books and other archived media. It’s not gonna happen.


Replies

BoppreH • today at 5:18 PM

> The only reason nuclear nonproliferation works is because the inputs are relatively rare and traceable.

There are only three companies in the world capable of mass-producing the chips used in SOTA AI training, they all use machines from a single company (ASML), and it also requires enormous power-hungry datacenters. Meanwhile, uranium can be dug up from the ground. You say that buying centrifuges will get you noticed, but why can't we do the same for SOTA ML hardware? It honestly sounds easier to control. And differently from nuclear weapons, both sides are afraid of rogue AIs.

This is a danger to our civilization, I'm sure we can figure something out.

> I have all the tools to develop my own ML training pipeline on my machine today.

Are you saying you can train the successor for Claude Opus 5.5 on your machine?

Already today you can get on a list by buying too many smoke alarms due to the americium content, or drug precursors, or a thousand harmless but suspicious products. I would be ok with a world where you likewise need a justification to buy thousands of GPUs at once.

tomku • today at 6:07 PM

Maybe we just have very different imaginative worlds, but after seeing the kind of crazy shit that people will believe thanks to online radicalization, I'm a lot less sanguine about the possibility that people will take a shot at your list of horribles. There are a lot of people out there who seem to genuinely believe that what's at stake is the very survival of humanity and the continued existence of truth, art, beauty and all the other good things. After enough exposure to that kind of pressure-cooker radicalization, something like a repressive world government that uses violence to ensure that only the "right" people have access to dangerous things like literacy and computation could seem like a small price to pay for saving us all. Sure, it'd suck in a lot of ways, but Altman and Amodei will have left the brave revolutionaries no choice.

I think it's very unlikely something like that would succeed, but I'm increasingly sure we're going to see serious attempts to do all of the things on your list, and they could do a lot of damage even in failure.