alt Hacker NewsI wonder if there would be an equivalent of Non proliferation treaty like Nukes?
I know it sounds crazy - but if there's even 0.1% chance that some models are so good that they can be used to hack into people's bank accounts - I, as the government, would not want that model to be publicly accessible. I would also request other countries to come to the table and sign this NPT(for AI).
Public will still have access to smaller models (like guns etc) up to Opus 4.8 etc but anything bigger than that is sooo good that it's dangerous. Nuclear also has benefits but the governments consider the worst when making policies rather than the best.
I am not touting Mythos as the god model but I wonder if the policy will move in this direction.
Replies
Bank accounts are currently regularly ""hacked"" by phish and other types of fraud. Current models are capable of helping with that.
Then there's monstrously stupid stuff like https://www.visa.com/en-us/solutions/intelligent-commerce , where visa place an AI inside the security boundary, pre-hacked for anyone who can prompt injection it.
> I wonder if there would be an equivalent of Non proliferation treaty like Nukes?
On top of this is just gonna be very hard to steal a nuke. So country cant just steal a nuke if they cant build one.
And even if some country do steal one nuke what of it? It really gives them nothing because they wont have parity for MAD.
Stealing model weight doesnt sound as complex though - once weights are out any small company can abliterate and run as many instances as they want.
Onbviously its not super simple, but certainly doable no matter how much effort LLM companies put into securing weights.
There is nothing that LLMs can do that humans cannot. If you are worried about bank accounts getting hacked, that's a problem to be solved on the banking side.
I'm sure many countries have learnt lessons from the NPT and would have the good sense to not agree to the same thing again. We've seen time and time again if you have nukes you can do what you want to the countries without them.
if models are good at hacking software they are equally good at patching it
Lmao, NPT for the slop machine, you know I have many bridges to sell you. Plugging the third world to internet was truly a mistake.
I'm pretty sure the days of any government signing any NPT with the US are very over. The trust is broken. I'd rather my government stockpile all the weapons of every sort at this point.
I think Dario's about to learn very quickly that making overly-hyped statements around safety to generate market interest is likely to result in regulatory reactions that don't just entrench his position as a market leader, but also severely curtail his business options.
You can't claim that your models are so good that they're basically weapons and then act shocked when the government starts imposing export restrictions on them exactly like they do for arms manufactures. Dario's recent blog post on "saving democracy" via regulatory capture and banning open weight models also mentioned export controls but just for chips, not what he's selling.