logoalt Hacker News

dgellowtoday at 10:19 AM0 repliesview on HN

I don’t think it has to be perfect or deterministic that way. It’s enough to have a bias towards implementing a backdoor in some circumstances. Something like, if the machine seems to be used in a Chinese environment, the model is biased towards missing some security issues, or towards implementing the type of bugs that can be used for an RCE, or similar.

Anthropic has done such checks at the agent level: https://cybersecuritynews.com/anthropic-claude-hidden-code/

Their excuse was defending against distillation attacks but you can see how that can be abused