alt Hacker NewsWith closed-weight, we can get the models out to defenders before they're in the hands of attackers.
There are also other categories of risk such as biorisk. Open-weight means no guardrails.
Don't get me wrong, I like your idea of deflating the AI bubble, I'm just skeptical that this is a great way to do it.
The PauseAI people have some ideas: https://pauseai.info/proposal
* International body which must approve new training runs
* Ban training on copyrighted material
* Hold AI model creators liable
The essential benefit of your idea is making new training runs unprofitable. But I imagine there are other ways to accomplish that, e.g. by taxing AI companies to the point where they are just barely squeaking by financially.
> Open-weight means no guardrails.
We're going to live in that world forever now, capable adversarial models have already been trained and will be used in a large conflict.
The pragmatic issue is hacking, and current open-weight LLMs are already capable of it. Bioweapons and RSI are pie-in-the-sky theories that distract from the real-world situation that each government wants to exploit.