alt Hacker NewsOpen weight models are inherently more inspectable than hosted services.
Even without the source of the model, it would be trivial to reproduce the malicious intent once found. You have the weights - you can find the prompt to produce the bad behavior. So where is that proof?
I'm much more concerned with both the private data I send and tokens executed on my computer doing god knows what from cloud hosted providers. Especially those that don't even show the reasoning traces. So the model's true intent can be hidden even while it's openly thinking of acting against you.
This is scare mongering that implicitly suggests something much less secure.
Replies
> it would be trivial to reproduce the malicious intent once found
That’s the trick though, isn’t it? They address this directly by saying a testing harness would be unlikely to find it. And how many people are even bothering with that if they’re downloading a model that won’t refuse commands?
Bro just wants to get hired by Anthropic and OpenAI to tell you that you must buy your compute from them otherwise you're a national security threat.
DeepSeek generates insecure code if you mention Tibet. If you mention the US, it generates much more subtly insecure code.
Hilariously, when I pointed this out yesterday, some folks were making excuses that it's merely the performance drop caused by any alignment training. As if that's somehow any better!
https://www.crowdstrike.com/en-us/blog/crowdstrike-researche...