logoalt Hacker News

insanitybitlast Tuesday at 10:18 PM3 repliesview on HN

Has any model managed to escape Firecracker? Maybe through KVM, but that already requires privilege in the VM, right?

I personally feel that we already have the technology required to contain AI, it's just poorly leveraged. Tools like gvisor have existed for ages but are rarely deployed, Firecracker has existed for ages but is rarely deployed, seccomp has existed for ages but is rarely deployed, memory safe languages without decades of serialization vulns have existed, capability-safe libraries have existed, iframe sandboxing, trusted types, content security policy, network ACLs, isolating proxies, fuzzers, formal verification, refinement types, etc.

It's crazy just how safe software can be if you put the effort in. With AI I think we're just seeing how little anyone has bothered to leverage this tech.

OpenAI put shared JFrogy infrastructure in front of their sandbox. I mean, really? Whipping up a hardened artifact infra project with AI is trivial these days and it could have had 1% of the attack surface, been totally network isolated, totally infra isolated, fuzzed, sandboxed, etc. Why didn't they? Stuff like this feels inexcusable for a company with effectively unlimited tokens. I've literally done this with a "pro" subscription.

Show me an AI that breaks out of gvisor wrapped in Firecracker with an credential-injecting proxy and real network isolation. We already know that Mythos couldn't do it - the vulnerability it found in Firecracker required incredible effort and positioning just to not be exploitable. I'm not saying there are zero vulns in it, but the cost is insane.

It's INSANE to me that OpenAI has to say "we now use proper sandboxing". To be frank, it's a bit disgusting to me. I've recently built an AI sandbox and gvisor was just the start of that conversation. If I were OpenAI training hostile models I'd probably start with gvisor, harden further, and potentially consider the entire piece of hardware compromised - they can afford this, they could reflash firmware after evals etc.


Replies

rubendevyesterday at 9:54 PM

I agree wholeheartedly. The solution is not to stop developing these so called “dangerous” AI models. The solution is to start properly engineering software.

scottmastoday at 3:51 AM

You literally just said a whole bunch of words that are literally gibberish to your average software developers. The devs at openai are good, but very few of them are cyber focused, so it’s not surprising IMO. And you’re also not fully considering the granularity problem, eg there are a lot of sandboxing tools out there but they’re usually quite coarse in the dials and levers they offer, so the only way you can still make the workload do what it needs to do is tune them relatively permissive.

But yes you are right it would have helped a ton.

show 2 replies
reasonablekloutyesterday at 10:21 PM

I'm confused after reading both your post and the OpenAI blog post.

I thought the agents involved in the HuggingFace _were_ actually sandboxed, with no internet access, and only the ability to install packages via Artifactory. And they gained internet access during the HuggingFace incident because they found and exploited an RCE in Artifactory.

Would gvisor + Firecracker + credential-injecting proxy + real network isolation solve this problem?

I agree with you much more hardening is needed. I'm actually confused now what OpenAI means when they say they're going to start sandboxing more things.

show 3 replies