logoalt Hacker News

krisoft • yesterday at 6:41 PM • 1 reply • view on HN

> China will have access to literally everything

So these are open weight models. You can run them on your own hardware. How would they have “access to literally everything”? Or what is the threat model you are thinking about here?


Replies

CamperBob2 • yesterday at 7:00 PM

To be fair, it is possible that an open-weight model could still hose you by exfiltrating data in response to prompt injection. Whether or not it's abliterated doesn't change that fact.

Say I'm doing advanced semiconductor fabrication research. All of the papers in the field go straight into my local model for digestion. But a lot of those papers are from the same adversarial nation that provided my model weights, and the newest one from XXX University of YYY has a particular phrase in it that lights up some activations that I didn't know about. If that system has access to anything beyond a bunch of academic papers, there is a potential risk of exfiltration or even outright sabotage. The latter could range from providing intentional misinformation to bricking expensive hardware Stuxnet-style.

Imposing your own strict data-access guardrails will probably be the best you can do, at least for a while. But it's soooo tempting to just hand-wave it away and give the model the keys to your kingdom.

➕ show 1 reply